Re: squid-2.4 release ?

Over the past several days I've had some time for performing benchmarks.
  I'll be posting the results in another day or two. I'm running into a
few issues of stability under load, as well as slight performance
degradation from 2.2STABLE5 (and obviously slower than reiser_raw, as is
expected).

The biggest concern is that I can consistently make Squid crash by
pushing it a little beyond it's load handling capability. This may be
simply an issue of CPU saturation, but the error I see (sometimes, other
times no error is presented) in the cache.log is one of a couple of
memory related errors. Either:

FATAL: logfileWrite: /var/log/squid/access.log: (12) Cannot allocate memory

or more commonly:

2001/01/07 03:09:31| comm_poll: poll failure: (12) Cannot allocate memory
2001/01/07 03:09:31| Select loop Error. Retry 1
2001/01/07 03:09:31| comm_poll: poll failure: (12) Cannot allocate memory
...2-9... Same error, repeated...
2001/01/07 03:09:31| comm_poll: poll failure: (12) Cannot allocate memory
2001/01/07 03:09:31| Select loop Error. Retry 10
FATAL: Select Loop failed!

Oh, and I also see a few of these in there too:

2001/01/07 03:09:17| comm_open: socket failure: (105) No buffer space
available

This is on a Datacomm-1 load at 100 reqs/sec. This is pretty well known
to be the limit on data throughput for two 7200 RPM IDE drives using the
standard FS. I've always been able to consistently get this performance
from less machines with the same number of drives (this is a 750MHz
Duron with 256MB RAM...the old test unit was a 500MHz K6-2 with 192 MB
RAM...same drives, however).

The async code does seem to be doing load shedding, and hit ratio goes
down towards the end of the run. But before the 4 hours are up, the box
invariably fails.

Another interesting item, is that the memory usage climbs towards the
end of the run. The Squid process goes from about 42MB during the first
couple of hours to as high as 90MB (usually more like 50-60MB) by the
end when it crashes. Available memory is still quite high at about 50MB
even after the process size has climbed, so we are not actually running
out of memory. I won't say it's a leak, but it does seem a little
greedy with memory as it begins to get overloaded, and since the errors
are always memory related, I guess there might be a problem there.

This is on a Squid-HEAD snapshot from about 5 days ago (from
squid-cache.org). I'll fetch todays snapshot for the actual published
benchmarks.

To be more clear about the tested system, the cache_dir total space is
4GB, and has been filled three or four times, as in all of my earlier
benchmarks. The compile is almost default, with async enabled, but
nothing else (I'm pretty sure). So memory usage should be quite
manageable on a box with 256MB. I suspect maybe there is a problem that
has been introduced in the past couple of months, because when I was
testing Henrik's new async stuff (when it was still from his async
branch) the performance was up by about 10%.

Anyway, I remember Henrik saying something about the system not having
enough contiguous free memory could also cause these kinds of errors.
Maybe some fragmentation issues have been introduced? (Is cbdata in
this squid-HEAD? Could there be added fragmentation with this new code?
  Nevermind. It looks like it hasn't been changed in the snapshot Squid
I'm running. I could have sworn I saw a message saying it was being
merged.)

If anyone has questions, or would like some specific tests or data, let
me know. I'll be posting the usual subsystest results and vmstat graphs
in a few days, but if any other information (debug out or debug levels,
etc.) are wanted, let me know.

Adrian Chadd wrote:

> Hi,
>
> What is left for a debut of squid-2.4-stable1?
>
>
> Adrian

                                   --
                      Joe Cooper <joe@swelltech.com>
                  Affordable Web Caching Proxy Appliances
                         http://www.swelltech.com
Received on Sun Jan 07 2001 - 18:36:05 MST