Multithreading problems

[Jeff Warren]

On Wed, 2006-08-02 at 13:48, Marshall Crocker wrote:
> I've had quite an experience with using pthreads and uClibc and wanted 
> to ask some questions.
> 
> The embedded system I'm using is gathering data and relaying that data 
> over an evdo network.  I decided to write a multi-threaded app so I 
> would have one thread gathering data and the other transmitting the 
> gathered data.  My app uses pthreads and I initially tried using a mutex 
> for sharing data.  When that didn't work I went to semaphores.
> 
> This worked ok with the exception that the threads would quit working. I 
> thought maybe I was running into deadlock but I very carefully checked 
> my code and even wrote debug messages to output before a semaphore was 
> locked and after it was unlocked.  I didn't see any problems there but 
> that's when I realized I never linked to the pthreads library.
> 
> Linking to the pthreads library caused stack smashing apparently because 
> loading the pthreads library and then the curl library (which I'm using 
> to send data) caused the app stack to be overrun.  I disabled some 
> features in curl which stopped the stack smashing but now I had the 
> problem where curl would not work in a thread.  I could move the curl 
> code into main() and it worked ok but inside a thread it just would not 
> work. I straced my app and saw that a socket curl opens and tries to use 
> is closed when it's in a pthread but this doesn't happen when the code 
> is in main() (here's the curl email: 
> http://curl.haxx.se/mail/lib-2006-08/0009.html).
> 
> I did some digging in the uClibc emails and found out that the pthreads 
> library is  linuxthreads  and not NPTL which explains why mutexes 
> wouldn't work and also why I see several threads for my app instead of 
> one.  I also saw that NPTL work is being done for uClibc and should be 
> merged into the main NPTL tree.
> 
> So now for the questions:  the deadlock I was seeing could this be a 
> result of not linking pthreads? Actually, I am still seeing deadlock 
> even with -lpthreads so maybe it's a problem with my semaphores?  Why 
> can I compile without linking pthreads? Why does curl have problems in a 
> thread but not when it's in main()?  Am I wrong in thinking that I would 
> have fewer problems with NPTL?
> 
> I know most of these questions can't be answered without knowing a lot 
> more about my setup but is there anything major I'm overlooking? This is 
> my first try at writing a multithreaded app and considering that uClibc 
> threading is different from glibc there may be something I'm missing.
> 
> Thanks in advance for any help.
> 
        I don't know if this is of any help to you, but I use uClibc on
an
embedded arm target.  I probably do things considerably different than
the rest of the community.  I could not initially get the pthreads to
work using 0.9.26 and when I did get them working I found that the use
of time slices to not be very efficient.  
        I took what the pthreads was doing (cloning the process using
the same
memory space and file table) and I did that instead.  Here is the sample
code to start a new "thread":

int StartThread(void* pMessage)
{     
    long retval;
    void **newstack;
    
    newstack = (void **) malloc(STACKSIZE);
    if (!newstack)
        return -1;
    newstack = (void **) (STACKSIZE + (char *) newstack);
    *--newstack = pMessage;
    retval = clone(ServoThreadFunc, newstack, CLONE_VM | CLONE_FILES |
SIGCHLD , pMessage);
    LOG(LOG_WARNING,"Main Thread pid is %d, Servo Thread is
%d",getpid(),retval);
    pidThread = retval;
    return retval;    
}

        You can then essentially treat the new process as a thread. 
There is
still problems with gdb, if you set a breakpoint in your code that is in
another process, gdb will exit.  But doing it this way, you can do
remote debugging on your main "thread" normally, or debug a worker
"thread" by using gdbserver with the --attach 'pid' argument.  
        This "quick and dirty way" has worked well for me and the kernel
seems
to be more efficient in how it handles these as they are distinct
processes with no pthreads overhead.  You lose the benefits of the
pthreads implementation, but they were not important for my setup.

Regards,

Jeff Warren