[ale] Allocation of memory

[alan at alanlee.org]

pthread_create will setup a new VMA (virtual memory area) for the stack frame
for the new thread, however most if not all of the page frames in that area will
point to null.  As the new thread executes and uses more and more of the stack,
those null page references will fault and Linux will on-demand map a free page
into that frame.  When that VMA is setup, Linux will, in addition, subtract the
number of page frames in that area from the total available pages in the system
(physical ram + swap space / page size).  This is so that if all memory users in
the system suddenly tried to use all of their pages, the system doesn't run
short.  You just have to be aware unless you lock those page frames, Linux is
free at any time to take any page and either swap it out or completely throw it
out of ram (if there is a file backing store like .text pages).

-Alan


On January 4, 2014 at 12:25 AM Alex Carver <agcarver+ale at acarver.net> wrote:
> Ok, I think I'm a bit fogged due to vacation (sure, that's the ticket)
> so help me clear something up as I fiddle with a little coding project.
>
> After reading and rereading and rerereading, the Linux Way (TM) of most
> memory allocation for things like pthreads and the like is to ask for a
> bunch of space and the OS will pseudo-grant that request with the caveat
> that anything that isn't really and truly used can get swiped away for
> something else.
>
> By example, a little program I'm writing spawns a thread with each new
> network connection using pthread_create. I'm watching the processes
> with top while testing (I'm trying to keep the program small and lean
> but still handle multiple operations). When it first starts up and no
> threads have been spawned, I see a resident size and a data (stack) size
> of some amount of kB. When a thread is spawned, the resident size
> increases slightly but the data-stack size jumps by about 8 MB per
> thread (the default request for pthreads on Linux). What I'm reading
> says that the 8 MB isn't really 8 MB, it's just a request and probably
> doesn't use all that much so the rest of the requested RAM that is
> unused while the entire program is running could potentially go off to
> another application. Have I read that right? When the threads close
> down and all that's left is the main process, the data-stack amount
> doesn't drop back down, it just sits there.
>
> I'm asking because I'm trying to make sure this program has a small
> memory footprint to start (which for the most part it does, resident
> size is under 600 kB and the initial data-stack size is only 300 kB) but
> I'll eventually run this program on a small machine so I don't want 8 MB
> to just vanish every time a thread starts up. It won't be many threads,
> maybe 10 at the most but still, I want to try to use the least memory
> possible just for fun. :)
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20140104/92ec2e24/attachment.html>