[ale] Preferred Linux under Docker?

Tue May 16 07:17:10 EDT 2017

You win. Nice set of good uses. The python use case was the clincher as
I've got groups that each use different versions of Python and the main
code.

Time to learn how to toss containers around on the HPC.

On May 16, 2017 1:02 AM, "Ted W." <ted-lists at xy0.org> wrote:

> > I've yet to find a use case in our setup for it that wasn't either:
> > 1.  "Containers are cool, we need to use them!"
> > 2.  Incoherent string of buzzwords from someone who just got back from a
> conference.
>
> Of course every environment is different but I hear this sentiment a lot
> and I thought your comment made a good launching off point for another
> mini rant on how containers can be useful. I'll list a few examples
> below and try to explain in some detail why containers are a great fit
> in each situation.
>
> Bootstrapping a development environment:
>         Let's say I build an application in python and I post it on
>         GitHub for others to download and contribute. To build my Python
>         application you need several python dependencies and sometimes
>         these dependencies conflict with the versions installed on your
>         system from your package repository or that you have installed
>         manually to build some other Python application. Normally, you
>         would have to know how to setup a separate python environment
>         using a tool like pyenv to get an isolated environment. At best,
>         you already know how to do this.  At worst, I have to write a
>         bunch of extra documentation or a script to do it for you. I
>         have to then keep up with this and make sure it remains accurate
>         as the project evolves.
>
>         If I'm using Docker in this scenario, I can write a Dockerfile
>         that contains all of my dependencies and ship that in the same
>         repository as my code. I don't have to care about what operating
>         system you're running. I don't have to worry about what
>         environment variables you have set or what package versions you
>         have that might cause weird issues that will get reported as
>         bugs. Additionally, if someone contributes code to the project
>         that changes the build process, they are also empowered to
>         change the build environment because it's clearly defined in a
>         single text file. When someone later tries to build the
>         repository, after pulling down the updates, they run docker
>         build (or some wrapper script provided in the repository) and
>         the container builds with these new dependencies and the
>         application build Just Works because the developer knew exactly
>         what the build environment looked like.
>
> Deploying application updates:
>         You have an application you need to update. On a traditional
>         system, you have an operating system on a machine and you update
>         the application either through your package manager or through
>         some code deployment workflow. Once it's deployed, you find an
>         issue. With packages, you might be able to `yum downgrade` or
>         equivalent but what if it's a code change to your internal
>         application? What changed between the last version and this
>         version of the code? Can you easily revert these changes?
>         Maybe, if you had built it in to a package for the OS before
>         deployment.
>
>         If you're using containers to run your application, you build a
>         new version of your container which has the new version of your
>         code as well as any dependency updates. You shoot the old
>         container in the head and you start up the new container. If you
>         find a problem. You shoot this new one in the head and start the
>         old one up again. You can also use one of the many container
>         management platforms on the market (Swarm, Rancher, etc.) to
>         define a percentage of the containers running the application
>         which should be replaced by the new version to roll out the
>         application gradually.
>
> Security
>         Containers are just processes on a host OS that are isolated
>         using cgroups and kernel namespaces. If you don't want your
>         container to do something, you can set the appropriate
>         restrictions on the host operating system. There are even tools
>         emerging now to improve this experience. Liz Rice recently gave
>         a talk this past week at OSCON [1] about restricting the syscalls
>         your container can make to the host kernel.
>
>         As someone already pointed out in another thread, you do have to
>         be aware of where your containers come from. In production, you
>         should really build your containers from the scratch image or
>         directly from the official (Alpine|CentOS|Debian|.*) container
>         repository. **
>
>         [1] https://conferences.oreilly.com/oscon/oscon-tx/public/
> schedule/detail/56840
>         ** There are differing opinions on this
>
> On Mon, May 15, 2017 at 01:41:09PM +0000, Beddingfield, Allen wrote:
> > I've yet to find a use case in our setup for it that wasn't either:
> > 1.  "Containers are cool, we need to use them!"
> > 2.  Incoherent string of buzzwords from someone who just got back from a
> conference.
> > Allen B.
> >
> >
> > --
> > Allen Beddingfield
> > Systems Engineer
> > Office of Information Technology
> > The University of Alabama
> > Office 205-348-2251
> > allen at ua.edu
> >
> > On 5/15/17, 8:29 AM, "ale-bounces at ale.org on behalf of Jim Kinney" <
> ale-bounces at ale.org on behalf of jim.kinney at gmail.com> wrote:
> >
> >     I still see 'containers' as a blend between a chroot and a virtual
> machine.
> >
> >
> >     As an admin, user willingness to download any old container they
> found on the tubes that has their solution stuffed into it is a terrifying
> idea.
> >
> >
> >     On Mon, 2017-05-15 at 09:22 -0400, leam hall wrote:
> >
> >     On Mon, May 15, 2017 at 9:16 AM, Beddingfield, Allen <allen at ua.edu>
> wrote:
> >
> >     Since I'm a SUSE guy, I would automatically say SUSE/openSUSE, but
> I'm sure you will get that answer from each person with their preferred
> distro(s).
> >
> >
> >
> >
> >     "If I knew SuSE like you know SuSE..."   Sorry, couldn't resist.
> >
> >     Remeber those Sun boxes? This will be going on them. We now have a
> >     garage to deal with the sound issue.   :)
> >
> >     After a couple decades I'm not sure the current distro vendors get
> it.
> >     RH seems to be moving away from being on the hardware or a VM, and
> >     wants to go theri own way like Solaris. If I'm going to go somewhere
> >     might as well go all the way.
> >
> >     Leam
> >     _______________________________________________
> >     Ale mailing list
> >     Ale at ale.orghttp://mail.ale.org/mailman/listinfo/ale
> >     See JOBS, ANNOUNCE and SCHOOLS lists at
> >     http://mail.ale.org/mailman/listinfo
> >
> >     --
> >     James P. Kinney III
> >
> >     Every time you stop a school, you will have to build a jail. What you
> >     gain at one end you lose at the other. It's like feeding a dog on his
> >     own tail. It won't fatten the dog.
> >     - Speech 11/23/1900 Mark Twain
> >
> >     http://heretothereideas.blogspot.com/
> >
> >
> >
> >
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://mail.ale.org/mailman/listinfo/ale
> > See JOBS, ANNOUNCE and SCHOOLS lists at
> > http://mail.ale.org/mailman/listinfo
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20170516/b9d4dd56/attachment.html>