Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libcurl can't download FTP website in 7.57 or later version #2445

Closed
wncboy opened this issue Apr 2, 2018 · 6 comments
Closed

libcurl can't download FTP website in 7.57 or later version #2445

wncboy opened this issue Apr 2, 2018 · 6 comments
Labels
FTP

Comments

@wncboy
Copy link

wncboy commented Apr 2, 2018 •
edited

Hi All,

[I did this]
I use CURLOPT_WILDCARDMATCH to download FTP website, but libcurl will report "Unable to parse FTP file list", and this issue always can be reproduced if the folder contains a large number of folders/files.

I found that this issue appears in 7.57 or later version, and it should be caused by the following commits:

=============================================
commit 1d72b5b
Author: Daniel Stenberg daniel@haxx.se
Date: Wed Oct 25 18:19:44 2017 +0200

ftplistparser: follow-up cleanup to remove PL_ERROR()

commit f786d1f
Author: Max Dymond cmeister2@gmail.com
Date: Wed Oct 25 15:52:43 2017 +0100

ftplistparser: free off temporary memory always

When using the FTP list parser, ensure that the memory that's
allocated is always freed.

Detected by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3682
Closes #2013

=============================================

BTW, I'm using curl's Debian package.

[I expected the following]
This issue can be fixed.

[curl/libcurl version]
libcurl3:amd64 7.57.0

[operating system]
Linux 4.4.98.x86_64.1

Thanks,
@wncboy wncboy changed the title Curl can't download FTP website in 7.57 or later version lib curl can't download FTP website in 7.57 or later version Apr 2, 2018
@wncboy wncboy changed the title lib curl can't download FTP website in 7.57 or later version libcurl can't download FTP website in 7.57 or later version Apr 2, 2018
@bagder bagder added the FTP label Apr 2, 2018
@bagder
Copy link
Member

bagder commented Apr 2, 2018

Thanks for your report. Please tell us (with more details) how to reproduce this problem as otherwise we can't really work on fixing it.

@wncboy
Copy link
Author

wncboy commented Apr 2, 2018

Yes, we can use the following test program to reproduce this issue:
https://curl.haxx.se/libcurl/c/ftp-wildcard.html

Running ftp-wildcard in the console, and using the following command:
./ftp-wildcard ftp://ftp.scene.org/music/groups//*

curl_easy_perform function will return an error code, the code number is 87 (Unable to parse FTP file list)

Thanks,

@wncboy
Copy link
Author

wncboy commented Apr 9, 2018

Do you have any updates?

Thanks,

@bagder
Copy link
Member

bagder commented Apr 9, 2018

If there were updates, they would be posted here.

bagder added a commit that referenced this issue Apr 19, 2018
@bagder
ftplistparser: keep state between invokes
8349a4a 
Regression from f786d1f

Reported-by: wncboy on github
Fixes #2445
@bagder bagder mentioned this issue Apr 19, 2018
Closed
@bagder bagder closed this as completed in 8fb78f9 Apr 19, 2018
@bagder
Copy link
Member

bagder commented Apr 23, 2018

Re-opened since the fix for this (#2508) was reverted in 10b195d due to memory leaks.

@bagder bagder reopened this Apr 23, 2018
bagder added a commit that referenced this issue Apr 23, 2018
@bagder
ftplistparser: keep state between invokes
d709ec0 
Regression from f786d1f

Reported-by: wncboy on github
Fixes #2445
Closes #2508
@bagder bagder mentioned this issue Apr 23, 2018
Closed
@bagder bagder closed this as completed in abbc845 Apr 23, 2018
@bagder
Copy link
Member

bagder commented Apr 24, 2018

Had to revert again due to fuzzing problems with the landed code...

@bagder bagder reopened this Apr 24, 2018
bagder added a commit that referenced this issue Apr 24, 2018
@bagder
ftplistparser: keep state between invokes
6c1278a 
Fixes FTP wildcard parsing when doing over a number of read buffers.

Regression from f786d1f

Reported-by: wncboy on github
Fixes #2445
Closes #2519
@bagder bagder mentioned this issue Apr 24, 2018
Closed
@bagder bagder closed this as completed in e6c2236 Apr 24, 2018
@lock lock bot locked as resolved and limited conversation to collaborators Jul 23, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FTP
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bagder @wncboy