[ale] Would you mind critiquing a container build HOWTO?
Scott McBrien
smcbrien at gmail.com
Thu Jul 4 12:13:46 EDT 2024
I’m not familiar enough with Amazon Linux 2023 to provide information on what is or isn’t in /proc.
As to performance, the Red Hat perf engineering team regularly benchmarks RHEL, Amazon Linux, and Ubuntu. Except for Java workloads, they perform about the same on benchmarking. Java workloads do significantly better on RHEL because Transparent Huge Pages are enabled, where on the other two distros, they’re disabled.
I know that there’s a lot of FUD about “cloud optimized Linux” but the benchmark results show there’s no performance gain. My friend manages the team that builds Azure Linux, for him it’s about building a distro that has the smallest package footprint needed for the capabilities they need. Smaller images mean faster deployments and less need for updates because they track less software.
All that said, unless you’re mapping volumes to the container, I’d assume you’re looking at a FS overlay inside of the container (not the host). I’m sure the runtime passes through some stuff natively to the OS, in order for processes and activities to run inside the containerized environment without accidentally choking on some random lack of file or data. But really I wouldn’t try to compare environment in-container with the host, unless you’re explicitly trying to figure out which of these is passed through.
> On Jul 4, 2024, at 11:02 AM, Leam Hall via Ale <ale at ale.org> wrote:
>
> Hey Scott, since you spoke up I'm going to reply. You know more about this than I do, so feel free to get verbose. I need to keep learning.
>
> On both the host and the container, /proc/sys/kernel/dmesg_restrict is 0. Neither host nor container have a dmesg log, but that might be my lack of systemd knowledge. There are no volume maps, I'm just poking at the containers.
>
> Using the Amazon Linux 2023 image is part of my current learning plan. I figure the AWS folks know best how to tune the AMI to their hosts, though that might be an assumption. I'm also trying to get away from a bunch of different base images for a tech stack. If an n-tier app uses n different base OSes and configurations, support and insight become difficult. Most of my career has been enterprise-y, and we always have to secure and document things. Using RHEL, or any one OS, is better than just pulling whatever hub.docker.com puts as the default.
>
>
> Leam
>
>
>> On 7/4/24 09:00, Scott McBrien via Ale wrote:
>> I don’t use docker or Amazon Linux. Rather, podman and RHEL.
>> Amazon Linux 2 is based on CentOS7, where AL2023 is based on a smattering of CentOS Stream plus Fedora bits. So, in theory the Amazon Linuxes could use SELinux. If that’s the case, containers run with the context container_t which limits access and could explain the differential between accessing dmesg. Though this could also be explained by the container not having access to the host’s /var/log directory. (Are you using some sort of volume map to provide your container /var/log access on the host? If not, it’s looking in its own filesystem overlay for the content, and not finding it.)
>> IIRC Docker has made rootless containers the default.
>> Singularity was started as a project by Berkeley in conjunction with Lawrence Livermore, and predates docker. Its goal was to make portable environments for running HPC jobs. Greg Kurtzer (yes that one) commercialized it and convinced a PE firm to spin off a company (sylabs) to work on commercializing it. Though, things have not gone well…
>> -STM
>>>> On Jul 4, 2024, at 9:35 AM, Leam Hall via Ale <ale at ale.org> wrote:
>>>
>>> I forgot to add that dmesg fails on the container, even though my user on the host can see it.
>>>
>>> sh-5.2# dmesg
>>> dmesg: read kernel buffer failed: Operation not permitted
>>>
>>> Leam
>>>
>>>
>>>> On 7/4/24 08:19, dj-Pfulio via Ale wrote:
>>>> Does that mean that Docker doesn't still by default use privilege containers?
>>>> I didn't see that question answered.
>>>>> On July 4, 2024 9:07:37 AM EDT, Jim Kinney via Ale <ale at ale.org> wrote:
>>>>> That's why singularity started and RHEL did their version to also add in
>>>>> selinux. Container root should not be host root.
>>>>>
>>>>>> On Thu, Jul 4, 2024, 8:46 AM DJPfulio--- via Ale <ale at ale.org> wrote:
>>>>>
>>>>>> At the risk of showing my ignorance, has docker changed their default so
>>>>>> that using privileged containers is a hassle and not the default? That's a
>>>>>> huge reason I've avoided Docker completely.
>>>>>>
>>>>>>
>>>>>> On 7/4/24 07:18, Leam Hall via Ale wrote:
>>>>>>> And eventually I remember that docker run has a -d switch...
>>>>>>>
>>>>>>> Leam
>>>>>>>
>>>>>>>
>>>>>>> On 6/30/24 21:14, Mark Ulmer wrote:
>>>>>>>> Leam, I've run into the same issues... I just could not get
>>>>>>>> systemctl enable and start commands to work. Here is the final
>>>>>>>> docker file that works. I added yum update.
>>>>>>>>
>>>>>>>> FROM amazonlinux:latest RUN yum update -y RUN yum install -y
>>>>>>>> iproute sysstat procps-ng httpd EXPOSE 80 CMD
>>>>>>>> ["httpd","-D","FOREGROUND"]
>>>>>>>>
>>>>>>>>
>>>>>>>> I suggest also adding the docker run example to your HOWTO.
>>>>>>>>
>>>>>>>> docker run --name amzl_web -p 8080:80 docker.io/library/amzl_web
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards, Mark Ulmer
>>>>>>>>
>>>>>>>>
>>>>>>>> ------ Original Message ------
>>>>>>>>> From "Leam Hall via Ale" <ale at ale.org>
>>>>>>>> To "Atlanta Linux Enthusiasts" <ale at ale.org> Cc "Leam Hall"
>>>>>>>> <leamhall at gmail.com> Date 6/29/2024 9:09:08 AM Subject [ale] Would
>>>>>>>> you mind critiquing a container build HOWTO?
>>>>>>>>
>>>>>>>>> Hey container-savvy peeps, would you mind critiquing a short
>>>>>>>>> HOWTO (below) on getting an Amazon Linux container to run
>>>>>>>>> locally? I'm doing some AWS study and want to put together a more
>>>>>>>>> concise document that will let folks try out AWS without having
>>>>>>>>> to reroute through a dozen documents to resolve basic tasks.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>> Leam -- Site Reliability Engineer (reuel.net/resume) Scribe: The
>>>>>>>>> Domici War (domiciwar.net) General Ne'er-do-well
>>>>>>>>> (github.com/LeamHall)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> My hypothesis is that running Amazon Linux in EC2 would be more
>>>>>>>>> performant than other Linux versions because the AWS engineers
>>>>>>>>> could tune their OS distribution to their platform.
>>>>>>>>>
>>>>>>>>> To test, I began by building an Amazon Linux container locally.
>>>>>>>>> This would allow investigating the OS itself, and then knowing
>>>>>>>>> how to configure it for use and observability. This quickly ran
>>>>>>>>> into the issue of portablity, while Amazon does have a container
>>>>>>>>> image on the Docker Hub, it doesn't run in standalone mode and it
>>>>>>>>> is missing basic sysadmin tools like "ps".
>>>>>>>>>
>>>>>>>>> So far a kludge resolves the stand-alone issue, but I'd like to
>>>>>>>>> find a better solution. Here's the annotated Dockerfile and
>>>>>>>>> commands used.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> FROM amazonlinux:latest (1) RUN yum install
>>>>>>>>> iproute sysstat procps-ng httpd -y (2)(6)(7)(8)(9) # RUN httpd
>>>>>>>>> -k start (3) # RUN systemctl start httpd
>>>>>>>>> (4) ENTRYPOINT ["/usr/sbin/httpd"] (5) CMD
>>>>>>>>> ["-DFOREGROUND"]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> (1)
>>>>>>>>> https://docs.aws.amazon.com/linux/al2023/ug/base-container.html
>>>>>>>>> (2) Adding some observability tools and httpd to keep the thing
>>>>>>>>> up. (3) This just exits out. (4) This fails with:
>>>>>>>>>> [3/4] RUN systemctl start httpd:
>>>>>>>>> 0.976 System has not been booted with systemd as init system (PID
>>>>>>>>> 1). Can't operate. 0.976 Failed to connect to bus: Host is down
>>>>>>>>> (5) This ENTRYPOINT and CMD pair works. (6) iproute gives the
>>>>>>>>> "ip" command. (7) sysstat gives the sar, pidstat, vmstat,
>>>>>>>>> iostat, and mpstat commands. (8) procps-ng gives the "ps"
>>>>>>>>> command. (9) httpd is required to have a running process,
>>>>>>>>> otherwise the container shuts down.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Commands:
>>>>>>>>>
>>>>>>>>> Get the container image (https://hub.docker.com/_/amazonlinux).
>>>>>>>>> docker pull amazonlinux
>>>>>>>>>
>>>>>>>>> In the directory with the Dockerfile. Note the ending ".". docker
>>>>>>>>> build -t amzl_web .
>>>>>>>>>
>>>>>>>>> You must also start it with "&" to regain your terminal window.
>>>>>>>>> docker run amzl_web &
>>>>>>>>>
>>>>>>>>> Connect to the container. docker exec -it <container_name>
>>>>>>>>> /bin/bash
>>>>>>>>>
>>>>>>>>> _______________________________________________ Ale mailing list
>>>>>>>>> Ale at ale.org https://mail.ale.org/mailman/listinfo/ale See JOBS,
>>>>>>>>> ANNOUNCE and SCHOOLS lists at
>>>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ale mailing list
>>>>>> Ale at ale.org
>>>>>> https://mail.ale.org/mailman/listinfo/ale
>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>
>>>> _______________________________________________
>>>> Ale mailing list
>>>> Ale at ale.org
>>>> https://mail.ale.org/mailman/listinfo/ale
>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>> http://mail.ale.org/mailman/listinfo
>>>
>>> --
>>> Site Reliability Engineer (reuel.net/resume)
>>> Scribe: The Domici War (domiciwar.net)
>>> General Ne'er-do-well (github.com/LeamHall)
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> https://mail.ale.org/mailman/listinfo/ale
>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>> http://mail.ale.org/mailman/listinfo
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> https://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>
> --
> Site Reliability Engineer (reuel.net/resume)
> Scribe: The Domici War (domiciwar.net)
> General Ne'er-do-well (github.com/LeamHall)
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> https://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
More information about the Ale
mailing list