[ale] Linuxthreads question

[John Mills]

Christopher, ALErs - 

Thanks for the comments and suggestions.

On Tue, 24 Jun 2003, Christopher Fowler wrote:

> On Mon, Jun 23, 2003 at 10:13:53PM -0400, John Mills wrote:

> > but just to
> > make sure I use the 'linuxthreads' version in which each thread gets a
> > different PID and won't block the others, but in which the threads can all
> > share their parent's data segment where I will put my buffers. I
> > specifically expect that I can avoid any locking, collision, or 'deadly
> > embrace' problems by using multiple rolling buffers. 
 
> If you use mutliple buffers, you may encounter performance issues
> if you transfer from one to the other.  In other words, if you have
> a buffer that the writer writes into then when he is done you transfer
> it to a read buffer, your performance may go down the drain.  I faced
> a simlar issue using strdup() in a parser.  

I hope to avoid this by using multiple input buffers for each "writer" - 
enough of them so any "reader" can send off that _one_ most recent buffer 
before the "writer" could write all the way around and chew it up. I don't 
care if I lose full buffer slots, so long as I am not sending one when it 
is getting rewritten. If I find that won't work, I may have to use some 
other scheme - either 'copy out' as you described, or a channel mechanism. 
Those would both have either the performance problem you describe, or 
latency problems. I think I can just spend enough RAM to avoid it and have 
a self-culling system that keeps all "readers" independent of each 
"writer".

Thanks for the mutex sample, too.

> > The "thread" of my question was, how should I link to be sure I _do_ use 
> > 'linuxthreads' and get per-thread PIDs.
 
> I think on Linux it is libc '-lc'.  But, you'll need to make sure
> you use the 'man 3thr' interface.  I wrote a program native to Linux
> and needed to port to *BSD.  I should have use PThreads but I used clone
> directly. Just make sure you do not use clone() in your code and you'll have
> to use the 3thr API to do threading.  

Good point - I don't have a BSD requirement at present, but ... I have 
been following some discussion in freebsd-questions on using the fBSD port 
of 'linuxthreads' and will keep watching that. I know BSD is in a very 
different spot than Linux w.r.t. threads, but I think the situation is a 
bit fluid in that camp.

Actually there will probably be a MsWinXP-Pro port of the beast, and I 
expect these mechanisms _won't_ fall smoothly into MsWin.

> > Thanks for further comments.
 
> There is one other solution that will avoid threads all together.  You
> may want to just use fork() and each process has its own data segment.
> What you need to do there is place your buffers in shared memory then
> anyone (even debug programs) can access that data.  In those cases, you
> can then use locking make sure the data is reliable. I use this method
> much more than I use threads to share data. I create an API that any
> program can use to access the data and that includes locking code and
> maping code.  It is fairly simple to do.  I have an embedded device and
> *all* config info is up in memory.  Even user password data that is
> usually stored in /etc/passwd.  For that case, I just replaced libc
> getpwXXX() with my own implementation that would grab the data.  I also
> have a piece of shared memory that stores system management info.  I
> then created an API that even our snmp agent uses to grab data from that
> segment as well as config.  In those case, it is awesome to use IPC to
> do your shared data.

That was my first thought, roughly, and may still be a better approach. I
was troubled by FUD as to whether I could have a large enough number of
shared segments (8-10), then I found 'clone' and 'linuxthreads' which
seemed to do what I wanted. I could use one BEEGGG segment, but I wanted 
to allocate and attach each "writer"s buffer individually as a dedicated 
resource. 

Beginner's jitters since I'm trying to use quite a few 'classic'
mechanisms for the first time in this project.

Thanks again.

 John Mills
 john.m.mills at alum.mit.edu

_______________________________________________
Ale mailing list
Ale at ale.org
http://www.ale.org/mailman/listinfo/ale