[httperf] Missing requests and connection resets with httperf and squid

Arlitt, Martin martin.arlitt at hp.com
Fri Dec 10 08:21:35 PST 2010


Hi Dave

I expect the issue in the second test is that persistent connections are not supported, but httperf is not recognizing the Connection: close header from the proxy.

One solution is described in a recent thread on the mailing list:

http://www.hpl.hp.com/hosted/linux/mail-archives/httperf/2010-November/thread.html

specifically, you can patch httperf to close the connection if so instructed by the server:

http://www.hpl.hp.com/hosted/linux/mail-archives/httperf/2010-November/000677.html

thanks
Martin

-----Original Message-----
From: httperf-bounces at linux.hpl.hp.com [mailto:httperf-bounces at linux.hpl.hp.com] On Behalf Of Dave Jenkins
Sent: Friday, December 10, 2010 8:09 AM
To: httperf at linux.hpl.hp.com
Subject: [httperf] Missing requests and connection resets with httperf and squid

Hi,

I'm new to the list, have recently been using httperf to benchmark apache vs squid+apache vs nginx. Generally I've been finding httperf a really useful tool but have run into problems when using it with the
squid+apache setup. Haven't found similar problems reported by others.

I'm using a sesslog consisting of 2 sessions:
(1) 10 bursts each consisting of one request for dynamic content followed by 0 to 10 static requests, 40 requests in total.
(2) 1 burst consisting of 60 static requests (page + css/js/images).
For simplicity, the tests below just request 2 sessions, so 100 requests in total.

Here's a successful sample run with apache alone.

Apache alone:
----8<----
$ httperf --hog --server mysite --wsesslog=2,1,mysesslog --rate=2 --verbose httperf --verbose --hog --client=0/1 --server=mysite --port=80 --uri=/
--rate=2 --send-buffer=4096 --recv-buffer=16384 --wsesslog=2,1.000,mysesslog
httperf: maximum number of open descriptors = 1024 reply-rate = 16.4 session-rate = 0.2 reply-rate = 1.6 session-rate = 0.0 Maximum connect burst length: 1

Total: connections 3 requests 100 replies 100 test-duration 12.294 s

Connection rate: 0.2 conn/s (4098.1 ms/conn, <=3 concurrent connections) Connection time [ms]: min 46.7 avg 4138.7 max 12294.2 median 75.5 stddev 7062.9 Connection time [ms]: connect 0.2 Connection length [replies/conn]: 33.333

Request rate: 8.1 req/s (122.9 ms/req)
Request size [B]: 125.0

Reply rate [replies/s]: min 1.6 avg 9.0 max 16.4 stddev 10.5 (2 samples) Reply time [ms]: response 51.1 transfer 2.1 Reply size [B]: header 310.0 content 13897.0 footer 0.0 (total 14207.0) Reply status: 1xx=0 2xx=100 3xx=0 4xx=0 5xx=0

CPU time [s]: user 3.79 system 8.50 (user 30.8% system 69.2% total 100.0%) Net I/O: 113.9 KB/s (0.9*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

Session rate [sess/s]: min 0.00 avg 0.16 max 0.20 stddev 0.14 (2/2)
Session: avg 1.50 connections/session
Session lifetime [s]: 6.2
Session failtime [s]: 0.0
Session length histogram: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 1
----8<----

- 100 requests, 100 responses, 0 errors.

Here's the same test on the same server but with apache moved to
127.0.0.1:80 and squid on external port 80 configured as an accelerator for apache:

----8<----
$ httperf --hog --server mysite --wsesslog=2,1,mysesslog --rate=2 --verbose httperf --verbose --hog --client=0/1 --server=mysite --port=80 --uri=/
--rate=2 --send-buffer=4096 --recv-buffer=16384 --wsesslog=2,1.000,mysesslog
httperf: maximum number of open descriptors = 1024 reply-rate = 4.6 session-rate = 0.0 reply-rate = 1.6 session-rate = 0.0 Maximum connect burst length: 1

Total: connections 43 requests 240 replies 41 test-duration 12.390 s

Connection rate: 3.5 conn/s (288.1 ms/conn, <=4 concurrent connections) Connection time [ms]: min 0.7 avg 302.7 max 1317.9 median 2.5 stddev 470.8 Connection time [ms]: connect 0.2 Connection length [replies/conn]: 1.000

Request rate: 19.4 req/s (51.6 ms/req)
Request size [B]: 309.0

Reply rate [replies/s]: min 1.6 avg 3.1 max 4.6 stddev 2.1 (2 samples) Reply time [ms]: response 75.9 transfer 3.3 Reply size [B]: header 498.0 content 21314.0 footer 0.0 (total 21812.0) Reply status: 1xx=0 2xx=41 3xx=0 4xx=0 5xx=0

CPU time [s]: user 3.85 system 8.54 (user 31.1% system 68.9% total 100.0%) Net I/O: 76.4 KB/s (0.6*10^6 bps)

Errors: total 41 client-timo 0 socket-timo 0 connrefused 0 connreset 41
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

Session rate [sess/s]: min 0.00 avg 0.08 max 0.00 stddev 0.00 (1/2)
Session: avg 40.00 connections/session
Session lifetime [s]: 12.4
Session failtime [s]: 0.0
Session length histogram: 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
----8<----

Note 240 requests and 41 replies, which should have been 100 of each.
And a connreset error for every reply. The squid access log only shows
43 requests (= number of connections). No errors are logged by squid.

On another squid+apache run with the same test, the result had:
Total: connections 100 requests 1087 replies 100 test-duration 13.070 s
Errors: total 97 client-timo 0 socket-timo 0 connrefused 0 connreset 97

- 1087 requests when there should have been 100.

I have not encountered any errors when browsing the squid+apache site with a browser.

Have also tried squid on port 80 accelerating apache on port 8080, similar problems occur in httperf tests but again, no problems observed with a browser.

Client: httperf-0.9.0-1.el5.rf from rpmforge on CentOS 5.5
Server: squid 2.7.STABLE3-4.1lenny1 / apache2 2.2.9-10+lenny8 on Debian Lenny.

Client and server are on the local network. Server is on a VM running on a Debian Lenny host.

Any help in tracking this down would be appreciated.

Thanks,

Dave
_______________________________________________
httperf mailing list
httperf at linux.hpl.hp.com
http://www.hpl.hp.com/hosted/linux/mail-archives/httperf/



More information about the httperf mailing list