cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: multithread curl. unexpected SIGSEGV

From: Azat Khuzhin <dohardgopro_at_gmail.com>
Date: Mon, 17 Sep 2012 17:20:38 +0400

Here is link on pastebin with normal formating.
http://pastebin.com/raw.php?i=8wmaB6MG

On Mon, Sep 17, 2012 at 5:18 PM, Azat Khuzhin <dohardgopro_at_gmail.com> wrote:
> On Mon, Sep 17, 2012 at 5:12 PM, Azat Khuzhin <dohardgopro_at_gmail.com> wrote:
>> On Mon, Sep 17, 2012 at 4:58 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
>>> On Mon, 17 Sep 2012, Azat Khuzhin wrote:
>>>
>>>> I have SIGSEGV when I run curl in 400 threads. valgrind doesn't detect any
>>>> memory errors.
>>>>
>>>> I can't post here all program sources, but here is some info:
>>>
>>>
>>> You shouldn't post "all program sources" - because we don't want them. You
>>> should write a new stand-alone example that repeats your problem and then
>>> post _that_ source code.
>>>
>>> If you can reproduce the problem in a separate application, chances are the
>>> problem is in your code.
>>
>> I don't think that this justified, but I try when I have more time.
>> Because this application crawl URL's and source code is not enough to
>> reproduce this error, also you need URL's (I think that ~5 million
>> rows should be enough.)
>>>
>>>
>>>> At all time program runs, only one valgrind message (valgrind
>>>> --show-reachable=yes --leak-check=full) :
>>>> ==30908== Thread 3:
>>>> ==30908== Conditional jump or move depends on uninitialised value(s)
>>>> ==30908== at 0x4C2B837: __GI___rawmemchr (mc_replace_strmem.c:1110)
>>>> ==30908== by 0x532A2AF: _IO_str_init_static_internal (strops.c:45)
>>>> ==30908== by 0x531E9D4: vsscanf (iovsscanf.c:44)
>>>> ==30908== by 0x530AB97: sscanf (sscanf.c:34)
>>>> ==30908== by 0x4074E9: parseUrl (crawler.c:28)
>>>> ==30908== by 0x407961: startRoutine (crawler.c:125)
>>>> ==30908== by 0x4E35B4F: start_thread (pthread_create.c:304)
>>>> ==30908== by 0x538D70C: clone (clone.S:112)
>>>
>>>
>>> ... this has nothing to do with libcurl but rather shows a problem in your
>>> application.
>>
>> I know, but it seems that this is not important error
>>>
>>>
>>>> And it caught SIGSEGV:
>>>> ==30908== Process terminating with default action of signal 11 (SIGSEGV)
>>>> ==30908== General Protection Fault
>>>> ==30908== at 0x50926C8: curl_slist_free_all (in
>>>> /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0)
>>>> ==30908== by 0x4031AF: curlSetOptionPC (curl.c:138)
>>>> ==30908== by 0x407CB2: startRoutine (crawler.c:185)
>>>> ==30908== by 0x4E35B4F: start_thread (pthread_create.c:304)
>>>> ==30908== by 0x538D70C: clone (clone.S:112)
>>>
>>>
>>> This could just as well be a thread/synch problem in your application. If
>>> you look at the curl_slist_free_all() implementation you'll see that it
>>> doesn't have a lot of logic that can go wrong.
>>
>> Every thread have it own curl handler.
>> At this moment I run my application under "valgrind --tool=drd" but
>> pthread_create is very slow under this tool,
>> so if I found some thing important/interesting I post here.
>>>
>>>
>>>> Also valgrind show that Curl_ssl_connect is called (how ? If I allow only
>>>> HTTP protocol)
>>>
>>>
>>> That seems strange yes and I can't explain it. Can't you just set a
>>> break-point and see how it gets there?
>>
>> It's a good idea. I give it a try and post here results when find something.
>
> Breakpoint 1, Curl_ssl_connect (conn=conn_at_entry=0x62f550,
> sockindex=sockindex_at_entry=0) at sslgen.c:199
> 199 sslgen.c: No such file or directory.
> (gdb) bt
> #0 Curl_ssl_connect (conn=conn_at_entry=0x62f550,
> sockindex=sockindex_at_entry=0) at sslgen.c:199
> #1 0x00007ffff7969895 in Curl_http_connect (conn=0x62f550,
> done=0x7ffff3986abf) at http.c:1333
> #2 0x00007ffff797bb14 in Curl_protocol_connect
> (conn=conn_at_entry=0x62f550,
> protocol_done=protocol_done_at_entry=0x7ffff3986abf) at url.c:3437
> #3 0x00007ffff797bdea in Curl_setup_conn (conn=conn_at_entry=0x62f550,
> protocol_done=0x7ffff3986abf) at url.c:5243
> #4 0x00007ffff7994074 in Curl_async_resolved (conn=0x62f550,
> protocol_done=protocol_done_at_entry=0x7ffff3986abf) at hostasyn.c:139
> #5 0x00007ffff7987ba1 in connect_host (conn=0x7ffff3986ac0,
> data=0x611770) at transfer.c:1952
> #6 Curl_do_perform (data=0x611770) at transfer.c:2087
> #7 0x0000000000403a34 in curlRequestGet (ptCurl=0x7ffff398ce00,
> pccUrl=0x7ffff398be00
> "https://www.google.com/bookmarks/mark?bkmk=http%3A%2F%2Fcmjornal.xl.pt%2Fdetalhe%2Fo-dia-em-imagens%2Fa-holandesa-laura-dekker-16-anos-tornou-se-ontem-a-mais-jovem-navegadora-solitaria-a-dar-a-volta-a"...)
> at gen/gen_curl.c:272
> #8 0x0000000000407a64 in startRoutine (arg=0x60e920) at
> crawler/src/crawler.c:143
> #9 0x00007ffff7bc7b50 in start_thread (arg=<optimized out>) at
> pthread_create.c:304
> #10 0x00007ffff76a870d in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
> #11 0x0000000000000000 in ?? ()
> (gdb) frame 8
> #8 0x0000000000407a64 in startRoutine (arg=0x60e920) at
> crawler/src/crawler.c:143
> 143 gotBytes = curlRequestGet(&curlPage, url);
> (gdb) p url
> $1 = "https://www.google.com/bookmarks/mark?bkmk=http%3A%2F%2Fcmjornal.xl.pt%2Fdetalhe%2Fo-dia-em-imagens%2Fa-holandesa-laura-dekker-16-anos-tornou-se-ontem-a-mais-jovem-navegadora-solitaria-a-dar-a-volta-a"
>>
>> Thanks for reply.
>>>
>>> --
>>>
>>> / daniel.haxx.se
>>> -------------------------------------------------------------------
>>> List admin: http://cool.haxx.se/list/listinfo/curl-library
>>> Etiquette: http://curl.haxx.se/mail/etiquette.html
>>
>>
>>
>> --
>> Azat Khuzhin
>
>
>
> --
> Azat Khuzhin

-- 
Azat Khuzhin
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2012-09-17