cURL / Mailing Lists / curl-library / Single Mail

curl-library

libcurl appends weird junk to downloaded file

From: kryptech <webmaster_at_kryptech.net>
Date: Sat, 18 Jun 2005 17:32:12 -0500

Hello, I have a probelm with libcurl. In a project I am working on, we
have to download files and then parse them, I use the following code to
download files:

/* download feeds to the disk
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 * GNU Library General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA.
 *
 * $Id: getfeed.c,v 1.7 2005/06/13 01:37:05 phreelance Exp $
 */
 
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <curl/curl.h>

#include "headers/error.h"
#include "headers/global.h"

#define CURL_RETURN_OKAY 0
#define CURL_TRUE 0

/*a libcurl callback that writes the data libcurl requires that the
 *function definition varible placement stay the same so dont chage it.
*/
static size_t write_data(void *ptr, size_t size, size_t nmemb, void
*data)
{
        int bytesize = size * nmemb;
        FILE *fp = (FILE *) data;
        
        fprintf(fp, "%s", (char*) ptr);
        
        return bytesize;
}

/*gets the raw XML into a file which can then be processed*/
void get_feed(pbString siteaddress, pbString diskfile) {
        if (siteaddress == NULL || diskfile == NULL)
                return;
        
        /*in case the file was already in use truncate it*/
        FILE *fp;
        fp = fopen(diskfile, "w");
        fatal_error_if(fp == NULL, E_FILE_NO_ACCESS, diskfile);
        
        CURL *curl;
        CURLcode opresult;
        
        curl = curl_easy_init();
        
        if (curl) {
                curl_easy_setopt(curl, CURLOPT_URL, siteaddress);
                curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
                curl_easy_setopt(curl, CURLOPT_FILE, (void*) fp);
                /* Using CURLOPT_FILE instead of the alias CURLOPT_WRITEDATA so that
this will work with older versions of libcurl.*/
                
                /*if in verbose mode display a little summary of the transfer*/
                if (progoptions.verbose == TRUE)
                        curl_easy_setopt(curl, CURLOPT_NOPROGRESS, CURL_TRUE);
                
                opresult = curl_easy_perform(curl);
                fatal_error_if(opresult != CURL_RETURN_OKAY, E_CURL_TRANSFER_FAIL,
siteaddress);
                
                curl_easy_cleanup(curl);
        } else {
                        fatal_error(E_CURL_NO_INIT, E_NO_APPENDIX);
        }
        fclose(fp);
}

(hope that appears well over email). So, it works well, but the
downloaded file has something like "0\n\n" appended to an otherwise
perfect transfer. For example, here is some output I get:
<?xml version="1.0"?>
<!-- RSS generated by SourceForge.net at Sat, 18 Jun 2005 22:24:25 GMT
on sf-web2

The content in this RSS feed was generated by SourceForge.net
regarding the SourceForge.net site or a project hosted on
SourceForge.net. Each project hosted on SourceForge.net is
responsible for their own content, file releases and support.

For a full list of available RSS feeds
https://sourceforge.net/docman/display_doc.php?docid=15483&amp;group_id=1

For terms of use:
https://sourceforge.net/docman/display_doc.php?docid=6048&amp;group_id=1

For content issues, comments and questions:
contact the project via the comment link or the resources on their
summary page.

For technical issues with this feed: contact the SourceForge.net team
at:
https://sourceforge.net/tracker/?func=add&amp;group_id=1&amp;atid=200001
(support is not provided via email)

In the event that you need to report abuse: contact the SourceForge.net
team at:
https://sourceforge.net/tracker/?func=add&amp;group_id=1&amp;atid=200001
(support is not provided via email)

-->
<rss version="2.0">
        <channel>
                <title>SourceForge.net: SF.net Project News: SourceForge.net Site News
(including full news text)</title>
                <link>http://sourceforge.net/projects/sitenews/</link>
                <description>SF.net Project News: SourceForge.net Site News (sitenews
project) - Project for the management of the SourceForge.net front page
site news block.</description>
                <copyright>Copyright and acceptable usage information for this RSS may
feed may be found at:
http://sourceforge.net/docman/display_doc.php?docid=6048&amp;group_id=1#licensing_and_other_terms</copyright>
                <lastBuildDate>Sat, 18 Jun 2005 22:24:25 GMT</lastBuildDate>
                <generator>SourceForge.net RSS generator</generator>
                <image>
                        <title>SourceForge.net logo</title>
                        <url>http://images.sourceforge.net/images/sflogo-88-1.png</url>
                        <link>http://sourceforge.net/</link>
                        <width>88</width>
                        <height>31</height>
                        <description>The world's largest Open Source software development
website</description>
                </image>
                <item>
                        <title>New Statistics Engine and Advanced Search Facility</title>
                        <description>SourceForge.net is proud to announce the launch of our
new project and site statistics system, Realtime Statistics for
SourceForge.net Subscribers, and Advanced Search for all site
users.&#60;br /&#62;[See the full press release] (&#60;a
href=&#34;http://sourceforge.net/forum/forum.php?forum_id=473918&#34;&#62;0 comments&#60;/a&#62;)</description>
                        <author>moorman_at_users.sourceforge.net (Jacob Moorman)</author>
                        <link>http://sourceforge.net/forum/forum.php?forum_id=473918</link>
                        <guid
isPermaLink="true">http://sourceforge.net/forum/forum.php?forum_id=473918</guid>
                        <pubDate>Thu, 09 Jun 2005 04:00:00 GMT</pubDate>

<comments>http://sourceforge.net/forum/forum.php?forum_id=473918</comments>
                </item>
                <item>
                        <title>SourceForge.net Surpasses 100,000 Projects!</title>
                        <description>Thanks to the continued support and interest from the
technical community, we've seen an amazing surge in projects posted to
SourceForge.net over the past several months. We think this is due in
part to the traction that Open Source is making in the enterprise as
more and more companies and organizations begin to migrate and adopt
Open Source technologies. SourceForge.net's mission is to fuel software
innovation by providing the most secure environment to foster
collaboration, development, and community among Open Source developers
and users. We're proud to be a catalyst for change in the technology
landscape, and want to thank all of our contributors and users who help
make us the largest collaborative development site in the
World.&#60;br /&#62;[See the full press release] (&#60;a
href=&#34;http://sourceforge.net/forum/forum.php?forum_id=473917&#34;&#62;0 comments&#60;/a&#62;)</description>
                        <author>moorman_at_users.sourceforge.net (Jacob Moorman)</author>
                        <link>http://sourceforge.net/forum/forum.php?forum_id=473917</link>
                        <guid
isPermaLink="true">http://sourceforge.net/forum/forum.php?forum_id=473917</guid>
                        <pubDate>Wed, 18 May 2005 04:00:00 GMT</pubDate>

<comments>http://sourceforge.net/forum/forum.php?forum_id=473917</comments>
                </item>
                <item>
                        <title>IBM Helps Drive Open Source Development</title>
                        <description>IBM has contributed more than 30 open source projects to
SourceForge.net and launched new online skills-building programs to spur
innovation, collaboration and development around emerging open source
projects.&#60;br /&#62;[See the Project List] [See the full press
release] (&#60;a
href=&#34;http://sourceforge.net/forum/forum.php?forum_id=473919&#34;&#62;0 comments&#60;/a&#62;)</description>
                        <author>moorman_at_users.sourceforge.net (Jacob Moorman)</author>
                        <link>http://sourceforge.net/forum/forum.php?forum_id=473919</link>
                        <guid
isPermaLink="true">http://sourceforge.net/forum/forum.php?forum_id=473919</guid>
                        <pubDate>Fri, 25 Feb 2005 05:00:00 GMT</pubDate>

<comments>http://sourceforge.net/forum/forum.php?forum_id=473919</comments>
                </item>
        </channel>
</rss>
0

I would be very, very, greatful to anyone that could help me out, email
me at webmaster_at_kryptech.net please.
Received on 2005-06-19