[httperf] Httperf and WebQoS Session Thruput Testing

Craig Wills httperf@napali.hpl.hp.com
Thu, 02 Nov 2000 08:31:15 -0500


John -
     At one point I hacked the gen/sess_cookie.c code of httperf to handle
more than one cookie header being returned by the server.  The code handles
multiple "Set-Cookie" headers in the same response for inclusion in a
subsequent request.   The code also replaces the current value of a cookie
with a new one if the cookie name is already present.  The entire module
with my changes marked by "#ifndef CEW" (which should not be defined) is
below.  It is based on previous release of httperf before latest changes
announced by Martin.
				- Craig Wills
				  Associate Professor
				  Computer Science Department
				  Worcester Polytechnic Institute
				  100 Institute Road
				  Worcester, MA 01609
				  office (508)831-5622
				  fax (508)831-5776
				  cew@cs.wpi.edu
				  http://www.cs.wpi.edu/~cew


> 
> 
> -----Original Message-----
> From: John Mendonca [mailto:john_mendonca@hp.com]
> Sent: Wednesday, November 01, 2000 10:19 PM
> To: httperf@napali.hpl.hp.com
> Cc: trong_nygen@hp.com
> Subject: Httperf and Multiple Cookies
> 
> 
> David,
> 
> I was real happy to see your message about the version 0.8 of httperf which
> includes ssl. I feel that you continued interest in the software will help
> ensure its viability.
> 
> In the WebQoS lab we've been using httperf extensively for testing session
> thruput.   I've run into several problems for which I've hacked minimally
> satisfactory solutions.
> 
> 1) WebQoS uses a browser cookie to keep track of web sessions. Many
> applications, particulary those using ".asp" pages also set browser cookies.
> Httperf only remembers one cookie per "web session" with --session-cookie,
> and this messes up our webqos tests since our qoscookie gets overwriten by
> the application or asp cookie.  I've hacked a solution to this problem by
> adding the parameter
> 	--no-asp-cookies
> 
> but httperf still only supports a single cookie per session. Have you or
> anyone else implemented a more elegant solution that keeps track of multiple
> cookies.
> 
> 2) When using webqos in session mode (--wsesslog) one may specify a session
> idle timeout (--timeout) and/or a error code (--failure-status) which will
> terminate httperf test sessions.  WebQoS uses one error code to terminate a
> session, sadly IIS generates another error code when it determines it is
> overloaded. The problem is that httperf will only use one error code to
> terminate a session. To fix this problem I've added a --fail-on-error
> parameter which will end a test session on any http 5xx error code.  I would
> like to give this code back to you or have you implement a better
> alternative.
> 
> 3) any thoughts on a windows 2k implementation.  Yes I realize that it would
> be a rewrite to implement winsock overlapped i/o. we've toyed with the
> idea...
> 
> What do you think?  Do you interested in these updates or in implementing
> something more elegant.
> 
> John Mendonca
> 447-2924
> 
> 
> --
> To unsubscribe: echo unsubscribe httperf | mail majordomo@linux.hpl.hp.com


/*
 
              Copyright (C) 1998 Hewlett-Packard Company
                         ALL RIGHTS RESERVED.
 
  The enclosed software and documentation includes copyrighted works
  of Hewlett-Packard Co. For as long as you comply with the following
  limitations, you are hereby authorized to (i) use, reproduce, and
  modify the software and documentation, and to (ii) distribute the
  software and documentation, including modifications, for
  non-commercial purposes only.
      
  1.  The enclosed software and documentation is made available at no
      charge in order to advance the general development of
      high-performance networking and computing products.
 
  2.  You may not delete any copyright notices contained in the
      software or documentation. All hard copies, and copies in
      source code or object code form, of the software or
      documentation (including modifications) must contain at least
      one of the copyright notices.
 
  3.  The enclosed software and documentation has not been subjected
      to testing and quality control and is not a Hewlett-Packard Co.
      product. At a future time, Hewlett-Packard Co. may or may not
      offer a version of the software and documentation as a product.
  
  4.  THE SOFTWARE AND DOCUMENTATION IS PROVIDED "AS IS".
      HEWLETT-PACKARD COMPANY DOES NOT WARRANT THAT THE USE,
      REPRODUCTION, MODIFICATION OR DISTRIBUTION OF THE SOFTWARE OR
      DOCUMENTATION WILL NOT INFRINGE A THIRD PARTY'S INTELLECTUAL
      PROPERTY RIGHTS. HP DOES NOT WARRANT THAT THE SOFTWARE OR
      DOCUMENTATION IS ERROR FREE. HP DISCLAIMS ALL WARRANTIES,
      EXPRESS AND IMPLIED, WITH REGARD TO THE SOFTWARE AND THE
      DOCUMENTATION. HP SPECIFICALLY DISCLAIMS ALL WARRANTIES OF
      MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
  
  5.  HEWLETT-PACKARD COMPANY WILL NOT IN ANY EVENT BE LIABLE FOR ANY
      DIRECT, INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES
      (INCLUDING LOST PROFITS) RELATED TO ANY USE, REPRODUCTION,
      MODIFICATION, OR DISTRIBUTION OF THE SOFTWARE OR DOCUMENTATION.
 
*/
/* This module intercepts `Set-Cookie:' headers on a per-session basis
   and includes set cookies in future calls of the session.

   Missing features:
	- intercepted cookies are always sent, independent of any constraints
	   that may be present in the set-cookie header
*/

#include <assert.h>
#include <ctype.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <httperf.h>
#include <call.h>
#include <conn.h>
#include <core.h>
#include <event.h>
#include <session.h>

#define MAX_COOKIE_LEN	256

#define SESS_PRIVATE_DATA(c) \
  ((Sess_Private_Data *) ((char *)(c) + sess_private_data_offset))

#define CALL_PRIVATE_DATA(c) \
  ((Call_Private_Data *) ((char *)(c) + call_private_data_offset))

typedef struct Sess_Private_Data
  {
    /* For now, we support just one cookie per session.  If we get
       more than one cookie, we'll print a warning message when
       --debug is turned on.  */
    size_t cookie_len;
    /* We can't malloc the cookie string because we may get a
       ``Set-Cookie:'' while there are calls pending that refer to an
       existing cookie.  So if we were to malloc & free cookies, we
       would have to use reference counting to avoid the risk of
       dangling pointers.  */
    char cookie[MAX_COOKIE_LEN];
  }
Sess_Private_Data;

/* We need the call private data to ensure that the cookie gets set
   only once.  EV_CALL_ISSUE gets signalled each time a call is sent
   on a connection.  Since a connection may fail, the same call may be
   issued multiple times, hence we need to make sure that the cookie
   gets set only once per call.  */
typedef struct Call_Private_Data
  {
    u_int cookie_present;	/* non-zero if cookie has been set already */
    char cookie[MAX_COOKIE_LEN];
  }
Call_Private_Data;

static size_t sess_private_data_offset = -1;
static size_t call_private_data_offset = -1;

/* A simple module that collects cookies from the server responses and
   includes them in future calls to the server.  */

static void
call_issue (Event_Type et, Object *obj, Any_Type regarg, Any_Type callarg)
{
  Call_Private_Data *cpriv;
  Sess_Private_Data *priv;
  Sess *sess;
  Call *call;

  assert (et == EV_CALL_ISSUE && object_is_call (obj));
  call = (Call *) obj;
  cpriv = CALL_PRIVATE_DATA (call);

  if (cpriv->cookie_present)
    /* don't do anything if cookie has been set already */
    return;

  sess = session_get_sess_from_call (call);
  priv = SESS_PRIVATE_DATA (sess);

  if (priv->cookie_len > 0)
    {
      if (DBG > 1)
	fprintf (stderr, "call_issue.%ld: inserting `%s'\n",
		 call->id, priv->cookie);
      cpriv->cookie_present = 1;
      memcpy (cpriv->cookie, priv->cookie, priv->cookie_len + 1);
      call_append_request_header (call, cpriv->cookie, priv->cookie_len);
    }
}

static void
call_recv_hdr (Event_Type et, Object *obj, Any_Type regarg, Any_Type callarg)
{
  char *hdr, *start, *end;
  Sess_Private_Data *priv;
  size_t len;
  struct iovec *line;
  Sess *sess;
  Call *call;

  assert (et == EV_CALL_RECV_HDR && object_is_call (obj));
  call = (Call *) obj;
  sess = session_get_sess_from_call (call);
  priv = SESS_PRIVATE_DATA (sess);

  line = callarg.vp;
  hdr = line->iov_base;
  if (tolower (hdr[0]) == 's' && line->iov_len > 12
      && strncasecmp (hdr + 1, "et-cookie: ", 11) == 0)
    {
#ifndef CEW
	char sbCookie[4*MAX_COOKIE_LEN]; /* temporary work space */
	char *pch, *pchOldName, *pchOldNameEnd;
	int bMatch, newlen, lenNewName;
#endif
      /* munch time! */
      start = hdr + 12;
      end = strchr (start, ';');
      if (!end)
	end = hdr + line->iov_len;
      len = end - start;
#ifndef CEW
      if (priv->cookie_len > 0) {
	  /* ok, we got a cookie and already have one.  Could be new or old */
	  /* note we are not breaking apart the new cookie to see if more 
	     than one */
	  pch = strchr(start, '=');
	  lenNewName = pch - start + 1;	/* include equal sign */
	  pchOldName = &priv->cookie[8];
	  newlen = 0;		/* length of new cookie string */
	  bMatch = 0;
	  while (1) {
	      if ((pchOldNameEnd = strchr(pchOldName, ';')) != NULL)
		  ;
	      else if ((pchOldNameEnd = strchr(pchOldName, '\r')) != NULL)
		  ;
	      else
		  break;
	      if (newlen > 0) {
		  sbCookie[newlen] = ';'; /* add separator */
		  newlen++;
	      }
	      if (strncmp(pchOldName, start, lenNewName) == 0) {
		  /* match of old cookie value, replace */
		  if (DBG > 0)
		      fprintf (stderr, "%s: replacing existing cookie value\n", prog_name);
		  memcpy(&sbCookie[newlen], start, len);
		  newlen += len;
		  bMatch = 1;
	      }
	      else {
		  /* old cookie value not matched, just copy as is */
		  memcpy(&sbCookie[newlen], pchOldName, 
			 pchOldNameEnd-pchOldName);
		  newlen += pchOldNameEnd - pchOldName;
	      }
	      if (*pchOldNameEnd != ';')
		  break;
	      else 
		  pchOldName = pchOldNameEnd + 1;
	  }
	  if (!bMatch) {
	      /* got a new cookie, add in */
	      if (newlen > 0) {
		  sbCookie[newlen] = ';'; /* add separator */
		  newlen++;
	      }
	      if (DBG > 0)
		  fprintf (stderr, "%s: appending new cookie value\n", prog_name);
	      memcpy(&sbCookie[newlen], start, len);
	      newlen += len;
	  }
	  /* ok, now put it back where httperf wants it */
	  if (newlen + 10 >= MAX_COOKIE_LEN)
	      {
		  fprintf (stderr, "%s.sess_cookie: truncating cookie list to %d bytes\n",
			   prog_name, MAX_COOKIE_LEN - 11);
		  newlen = MAX_COOKIE_LEN - 11;
	      }
	  memcpy (priv->cookie, "Cookie: ", 8);
	  memcpy (priv->cookie + 8, sbCookie, newlen);
	  memcpy (priv->cookie + 8 + newlen, "\r\n", 2);
	  priv->cookie[10 + newlen] = '\0';
	  priv->cookie_len = newlen + 10;
	  if (DBG > 0) {
	      fprintf (stderr, "%s: got cookie `%s'\n", prog_name, start);
	      fprintf (stderr, "%s: cookie string: %s", prog_name, 
		       priv->cookie);
	  }
      }
      else {
	  if (len + 10 >= MAX_COOKIE_LEN)
	      {
		  fprintf (stderr, "%s.sess_cookie: truncating cookie to %d bytes\n",
			   prog_name, MAX_COOKIE_LEN - 11);
		  len = MAX_COOKIE_LEN - 11;
	      }
	  memcpy (priv->cookie, "Cookie: ", 8);
	  memcpy (priv->cookie + 8, start, len);
	  memcpy (priv->cookie + 8 + len, "\r\n", 2);
	  priv->cookie[10 + len] = '\0';
	  priv->cookie_len = len + 10;
	  if (DBG > 0) {
	      fprintf (stderr, "%s: got cookie `%s'\n", prog_name, start);
	      fprintf (stderr, "%s: cookie string: %s", prog_name, 
		       priv->cookie);
	  }
      }
#else
      if (DBG > 0 && priv->cookie_len > 0)
	fprintf (stderr, "%s: can't handle more than one "
		 "cookie at a time, replacing existing one\n", prog_name);
      if (len + 10 >= MAX_COOKIE_LEN)
	{
	  fprintf (stderr, "%s.sess_cookie: truncating cookie to %d bytes\n",
		   prog_name, MAX_COOKIE_LEN - 11);
	  len = MAX_COOKIE_LEN - 11;
	}
      memcpy (priv->cookie, "Cookie: ", 8);
      memcpy (priv->cookie + 8, start, len);
      memcpy (priv->cookie + 8 + len, "\r\n", 2);
      priv->cookie[10 + len] = '\0';
      priv->cookie_len = len + 10;
      if (DBG > 0)
	fprintf (stderr, "%s: got cookie `%s'\n", prog_name, start);
#endif
    }
}

static void
init (void)
{
  Any_Type arg;

  sess_private_data_offset = object_expand (OBJ_SESS,
					    sizeof (Sess_Private_Data));
  call_private_data_offset = object_expand (OBJ_CALL,
					    sizeof (Call_Private_Data));
  arg.l = 0;
  event_register_handler (EV_CALL_ISSUE, call_issue, arg);
  event_register_handler (EV_CALL_RECV_HDR, call_recv_hdr, arg);
}

Load_Generator sess_cookie =
  {
    "per-session cookie manager",
    init,
    no_op,
    no_op
  };

--
To unsubscribe: echo unsubscribe httperf | mail majordomo@linux.hpl.hp.com