[httperf] Re: Connection rate is very low
TTerrill at synacor.com
Mon Oct 22 05:31:11 PDT 2012
Based on your tests without the proxy responding at 27,000 requests, I think its pretty clear that there's a bottleneck in the configuration that has the proxy.
Have you tried lower rates? (side note, at that rate, you could only sustain it for 3-4 seconds before running out of TCP Ports (the 65535 max number). Eventhough you have FD set high, there's only a small limited number of ports available for transmitting.)
I'd say try a rate of 1000, then ramp it up.
To help with that, I'd recommend getting auto bench (google it) which is a perl wrapper that allows you to define start rate, stop rate, step rate so you can slowly ramp up load and see where the system breaks.
Also, are you running a tool like linux "sar" on each server at 1 second sample rates (since your test is only lasting 1 second (rate = 15000, num_connections = 15000))? This is critical to have this data so you can tell what the resource utilization is on each of your servers (proxy and web server AND load generator).
Senior Systems Performance Engineer
40 La Riviere Drive, Suite 300
Buffalo, NY 14202
From: Vikash Kumar <vikash.kumar at oneconvergence.com<mailto:vikash.kumar at oneconvergence.com>>
Date: Monday, October 22, 2012 7:57 AM
To: "httperf at linux.hpl.hp.com<mailto:httperf at linux.hpl.hp.com>" <httperf at linux.hpl.hp.com<mailto:httperf at linux.hpl.hp.com>>
Subject: [httperf] Re: Connection rate is very low
I am also getting TCP SYN flooding msg in dmesg of Haproxy host m/c:
[ 2965.742025] possible SYN flooding on port 80. Sending cookies.
Command cat /proc/sys/net/ipv4/tcp_syncookies is set to 1.
What can be done to stoop this message?
On Mon, Oct 22, 2012 at 5:13 PM, Vikash Kumar <vikash.kumar at oneconvergence.com<mailto:vikash.kumar at oneconvergence.com>> wrote:
I am doing httperf test but I am getting very less connection rate. Below is the command and O/P of the command:
httperf --hog --client=0/1 --server=220.127.116.11 --port=80 --uri=/1kb --rate=15000 --send-buffer=4096 --recv-buffer=16384 --num-conns=15000 --num-calls=1
Maximum connect burst length: 26
Total: connections 15000 requests 9614 replies 9589 test-duration 39.149 s
Connection rate: 383.1 conn/s (2.6 ms/conn, <=12651 concurrent connections)
Connection time [ms]: min 2.8 avg 4765.6 max 38926.4 median 3202.5 stddev 4236.4
Connection time [ms]: connect 9995.9
Connection length [replies/conn]: 1.000
Request rate: 245.6 req/s (4.1 ms/req)
Request size [B]: 64.0
Reply rate [replies/s]: min 0.0 avg 273.8 max 1157.2 stddev 430.1 (7 samples)
Reply time [ms]: response 931.1 transfer 0.2
Reply size [B]: header 174.0 content 733.0 footer 0.0 (total 907.0)
Reply status: 1xx=0 2xx=6553 3xx=0 4xx=35 5xx=3001
CPU time [s]: user 12.21 system 26.94 (user 31.2% system 68.8% total 100.0%)
Net I/O: 232.5 KB/s (1.9*10^6 bps)
Errors: total 5411 client-timo 0 socket-timo 5386 connrefused 0 connreset 25
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
All the request are going through Haproxy. Configuration of the Haproxy host m/c is:
OS : Ubuntu 10.04
Model name: Intel(R) Xeon(R) CPU E5504 @ 2.00GHz
HA-Proxy version 1.4.22
No.of CPU's : 8
My server and client m/c's are 8 cores. I am using lighttpd as server.
My FD_LIMIT is 400000.
I had increased number of sockets at optimum figure.
Why I am getting such a small conn. rate, req. rate and reply rate?
When the same test is done for back- to back connection ie. without Haproxy, then I am able to achieve 27K conn. rate, req. rate and reply rate.
Can anybody help me with this?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the httperf