[httperf] Httperf doubts

ALBERT HIDALGO BAREA abarea at tid.es
Mon Mar 28 02:42:05 PST 2011


Thank you very much for the answer.
I will try it. Seems that I got all wrong.

-----Original Message-----
From: httperf-bounces at linux.hpl.hp.com [mailto:httperf-bounces at linux.hpl.hp.com] On Behalf Of Jim Whitehead II
Sent: lunes, 28 de marzo de 2011 12:27
To: httperf at linux.hpl.hp.com
Subject: Re: [httperf] Httperf doubts

On Mon, Mar 28, 2011 at 11:08 AM, ALBERT HIDALGO BAREA <abarea at tid.es> wrote:
> Hi,
>
> I want to make some web server benchmarking. Until, now I have done it with
> AB (Apache Benchmark), but I read that httperf is better and with more
> options.
>
> The planned tests involve having different workloads and combine them. For
> example, make 1.000, 10.000 and 100.000 requests and with different number
> of concurrencies like 1, 10, 100, 250. To stress the web server and see how
> it performs in different situations.
>
> With AB is as easy as run:
>
> ab -n  10000 -c 100 -k http://example/example.html (the first number is the
> number of requests, and the second is the concurrency of the test, finally,
> the -k option is for keep-alive)
>
> or, ab -n 100 -c 1000 -k http://example/example.html
>
> But with httperf I try to configure it in the same way, but I can't.
>
> I run:
>
> httperf --server example --port 80 --uri /example.html --rate 10000
> --num-conn 10 --num-calls 1000 (I want to do 1.000 requests and with 10
> concurrent clients, or different connections. Also I set the rate high
> enough, as I want to check the performance of the web server).

It seems that you're using the wrong parameters here. what you've told
httperf to do is make 10,000 connections per second, but only make 10
connections. On each of those connections, perform 1000 requests. This
is *quite* different from what you're actually trying to do.

The man page describes quite clearly what these parameters mean:

  --rate: Specifies the fixed rate at which connections or sessions are created.
  --num-conns: The total number of connections to create
  --num-calls: The total number of calls to issue on each connection
before closing it

If you're trying to match what you're doing with ab, you'd need to
specify --num-calls 1, so that only one request is being requested
over each connection. Then it's just a matter of putting in the
numbers you want.

So, for 10,000 requests with a concurrency level of '10' (although
there isn't a 1-to-1 mapping between ab options and httperf options),
you'd do something like this:

httperf --server example --port 80 --uri /example.html --rate 10
--num-conn 10000 --num-calls 1

That is, make 10,000 connections at a rate of 10 connections per
second. On each connection make a single request for the specified
URI.

> Once the test is done, it returns all the parameters, but the Replay rate is
> 0.
>
> This is the post of the test:
>
> httperf --client=0/1 --server=example --port=80 --uri=/example.html
> --rate=100000 --send-buffer=4096 --recv-buffer=16384 --num-conns=10
> --num-calls=1000
>
> Maximum connect burst length: 1
>
> Total: connections 10 requests 10000 replies 10000 test-duration 1.195 s
>
> Connection rate: 8.4 conn/s (119.5 ms/conn, <=10 concurrent connections)

You can see here that your 'concurrency' is 10, not 10000 like you
seemed to intend.

> Connection time [ms]: min 1064.8 avg 1123.6 max 1193.0 median 1106.5 stddev
> 47.3
>
> Connection time [ms]: connect 0.2
>
> Connection length [replies/conn]: 1000.000
>
>
>
> Request rate: 8371.3 req/s (0.1 ms/req)
>
> Request size [B]: 70.0
>
>
>
> Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
>
> Reply time [ms]: response 1.1 transfer 0.0
>
> Reply size [B]: header 211.0 content 168.0 footer 0.0 (total 379.0)
>
> Reply status: 1xx=0 2xx=10000 3xx=0 4xx=0 5xx=0
>
>
>
> CPU time [s]: user 0.28 system 0.91 (user 23.7% system 76.3% total 100.0%)
>
> Net I/O: 3670.6 KB/s (30.1*10^6 bps)
>
>
>
> Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
>
> Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
>
> The body of the requested web page is 168 bytes (Hello world HTML page), and
> httperf gets the web page correctly, but as said before the Reply rate is
> always 0.

There weren't enough samples, the test finished in 1.1 seconds. This
is not really a valid test of anything, as it takes quite a bit of
time for a server to warm up to get valid benchmarks.

> I try another test to see what changes, and run with the following
> parameters (the difference with the previous one is that now it has 100
> connections):
>
> httperf --client=0/1 --server=example --port=80 --uri=/example.html
> --rate=100000 --send-buffer=4096 --recv-buffer=16384 --num-conns=100
> --num-calls=1000

This just makes the test longer, it doesn't do what you think it does.
It means now that you're making 1000 connections (ever, not
concurrently) and making 1000 requests on each of those connections.

> Maximum connect burst length: 1
>
> Total: connections 100 requests 91742 replies 91727 test-duration 21.989 s
>
> Connection rate: 4.5 conn/s (219.9 ms/conn, <=100 concurrent connections)
>
> Connection time [ms]: min 13924.6 avg 18382.3 max 21987.3 median 17822.5
> stddev 1983.7
>
> Connection time [ms]: connect 2492.3
>
> Connection length [replies/conn]: 986.312
>
>
>
> Request rate: 4172.1 req/s (0.2 ms/req)
>
> Request size [B]: 70.0
>
>
>
> Reply rate [replies/s]: min 4148.1 avg 4170.3 max 4193.6 stddev 18.6 (4
> samples)
>
> Reply time [ms]: response 15.9 transfer 0.0
>
> Reply size [B]: header 211.0 content 168.0 footer 0.0 (total 379.0)
>
> Reply status: 1xx=0 2xx=91727 3xx=0 4xx=0 5xx=0
>
> CPU time [s]: user 2.41 system 19.57 (user 11.0% system 89.0% total 100.0%)
>
> Net I/O: 1829.1 KB/s (15.0*10^6 bps)
>
> Errors: total 15 client-timo 0 socket-timo 0 connrefused 0 connreset 15
>
> Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
>
> Now it shows the Reply rate which is similar to the Request rate. But now
> appears some connection reset problems, maybe to lack of file descriptors or
> socket errors.

This is because you're requesting 1000 connections in the same time
period (instantly, for all intents and purposes) and your server
cannot handle this.

> The AB test with the same parameters: ab -n 10000 -c 100 -k
> http://example/example.html, returns similar request per second result
> (assuming that in AB request per second result is equal to Reply rate result
> in httperf).

Just to reiterate, this tests something completely different than what
you asked httperf to do.

> So, my questions are:
>
> What happened in the first httperf test? The test is too short that it can't
> get the Reply rate? It is strange, isn't it.

Not really, if you have a test that terminates in 1.1 seconds, there's
not going to be much statistically significant information contained
in the results. In this case httperf doesn't sample any of the replies
so there's no information to be provided.

> Why sometimes appear connection reset errors?

Client timeout means your server did not respond in time, and
connection reset is likely the server resetting the connection. Your
server is under too heavy load (1000 connections per second for 1
second) and cannot handle it.

> To perform the tests like in AB tool, I have to set --num-conn like the
> concurrency I want to set, and --num-calls like the requests to do?

See the reference I posted above, perhaps it will be useful to you.

--
Jim Whitehead

_______________________________________________
httperf mailing list
httperf at linux.hpl.hp.com
http://www.hpl.hp.com/hosted/linux/mail-archives/httperf/

Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at.
http://www.tid.es/ES/PAGINAS/disclaimer.aspx



More information about the httperf mailing list