cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: [RFC] Cached memory pools

From: Jamie Lokier <jamie_at_shareable.org>
Date: Thu, 3 May 2007 03:58:08 +0100

Robert Iakobashvili wrote:
> On 5/2/07, Dan Fandrich <dan_at_coneharvesters.com> wrote:
> >On Wed, May 02, 2007 at 10:25:45AM +0300, Robert Iakobashvili wrote:
> >> The issue, that we have now is a certain performance degradation
> >> within tens of minutes upon a high CPU and low memory conditions .
> >> Shotting oprofile at such conditions is tricky, but we will try some
> >tricks.
> >> We are suspecting intensive memory defragmentation efforts by glibc,
> >> which impacts the performance.
> >
> >Have you tried tuning glibc's malloc?
> >http://www.gnu.org/software/libc/manual/html_node/Malloc-Tunable-Parameters.html
> >Or linking in an alternative allocator for comparison?
>
> Not yet.
>
> Very interesting option, Dan. Thank you for the pointer.
>
> Have you any experience with the tuning params
> for mallopt, that may may be recommended for libcurl?
>
> On a long run, however, we do believe in zero/near-zero run-time
> allocations, using cached memory.

Is it just memory allocation, or are you maintaining cached
pre-initialised data structures as well?

If it's just fast memory allocation, at the extreme end of generic
mallocs, there are mallocs which work by selecting pools at compile
time, when the allocation size is a compile-time constant, and using
the fastest strategies (like single-linked lists, or contiguous
sub-block allocations, or other things) for those fixed sizes.

-- Jamie
Received on 2007-05-03