IIS Stress Testing with WCAT and Log Parser

Server Setup:

  • Windows Server 2008 64-bit
  • IIS 7.5
  • WCAT 5.2
  • Log Parser 2.2

WCAT wasn’t as straightforward as I was hoping it would be and the instructions online aren’t very cohesive, plus there seems to be a few ways to use the program. Hopefully this will help anyone with similar needs. Please note that the last version of WCAT is 6.3, as of this writing, but due to non-technical reasons, we installed WCAT 5.2 instead. There are differences in how to use the tools when comparing 5.2 and 6.3, specifically the way the configuration uberfile (UBR) is formatted. The biggest hint is that if the file uses curly brackets “{}”, then that is for the latest 6.3 version. This writeup is for using the older 5.2 version. Adapting to the newer version should be similar but with differently formatted config files.

Ideally, a two-machine setup should be used because running the tests on the same machine will skew the results but if you are like us and didn’t have a choice in this situation with only one machine to use, then running it on one machine will be sufficient to get a rough idea.

Once installed, there are basically two steps in order to use WCAT. The first step is to generate the configuration file for WCAT to use and the second step is to actually use WCAT.

Example locations:

  • C:\Program Files (x86)\Log Parser 2.2\
  • C:\Program Files (x86)\WCAT Client\
  • C:\Program Files (x86)\WCAT Controller\

Step 1a

When first starting out, try out a simpler case where you don’t need Log Parser. Just start out with the following sample configuration file, which should be added to the WCAT Controller directory. Once you get the hang of this, try out step 1b.

config.ubr

[Configuration]
NumClientMachines:  	1        	# number of distinct client machines to use
NumClientThreads:     	100     	# number of threads per machine
AsynchronousWait:     	TRUE     	# asynchronous wait for think and delay
Duration:             	5m      	# length of experiment (m = minutes, s = seconds)
MaxRecvBuffer:        	8192K      	# suggested maximum received buffer
ThinkTime:            	0s       	# maximum think-time before next request
WarmupTime:           	5s      	# time to warm up before taking statistics
CooldownTime:         	6s      	# time to cool down at the end of the experiment

[Performance]				# http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/7898b860-462c-4846-a3a8-1179f287ad88.mspx?mfr=true
System\Context Switches/sec
System\System Calls/sec
System\Processes
System\Processor Queue Length
Processor(_Total)\% Processor Time

[Script]
SET RequestHeader = "Accept: */*\r\n"
APP RequestHeader = "Accept-Language: en-us\r\n"
APP RequestHeader = "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705)\r\n"
APP RequestHeader = "Host: %HOST%\r\n"

NEW TRANSACTION
classId = 1
NEW REQUEST HTTP
ResponseStatusCode = 200
Weight = 1
Verb = "GET"
URL = "http://www.testdomain.com/somepage.asp"

Step 1b

In our case, we wanted to use live server logs as the basis for our WCAT stress testing so we have to grab the website logs from the server. These logs are the same ones that IIS generates for every hit to the web server and the default location is C:\Windows\System32\LogFiles\W3SVC**** where **** is the ID for the website that you are investigating.

The first thing to do is that you’ll need two files: wcat.tpl and wcat.sql in the same directory where your Log Parser is installed. For example, if your Log Parser is installed at C:\Program Files (x86)\Log Parser 2.2\, add the following two files to that directory. You can add the files anywhere but just for example, this will make it easier.

wcat.tpl

<LPHEADER>[Configuration]
NumClientMachines:  	1        	# number of distinct client machines to use
NumClientThreads:     	100     	# number of threads per machine
AsynchronousWait:     	TRUE     	# asynchronous wait for think and delay
Duration:             	5m      	# length of experiment (m = minutes, s = seconds)
MaxRecvBuffer:        	8192K      	# suggested maximum received buffer
ThinkTime:            	0s       	# maximum think-time before next request
WarmupTime:           	5s      	# time to warm up before taking statistics
CooldownTime:         	6s      	# time to cool down at the end of the experiment

[Performance]				# http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/7898b860-462c-4846-a3a8-1179f287ad88.mspx?mfr=true
System\Context Switches/sec
System\System Calls/sec
System\Processes
System\Processor Queue Length
Processor(_Total)\% Processor Time

[Script]
SET RequestHeader = "Accept: */*\r\n"
APP RequestHeader = "Accept-Language: en-us\r\n"
APP RequestHeader = "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705)\r\n"
APP RequestHeader = "Host: %HOST%\r\n"

</LPHEADER>
<LPBODY>
NEW TRANSACTION
	classId = %UID%
	NEW REQUEST HTTP
		ResponseStatusCode = %STATUS%
		Weight = %TOTAL%
		verb = "%VERB%"
		URL = "%URI%"
</LPBODY>
<LPFOOTER>
</LPFOOTER>

The wcat.tpl file is used as a template for config.ubr. Note that the file is divided into three sections: LPHEADER, LPBODY, and LPFOOTER. They all function as they suggest. The wcat.sql will produce results that populate the body section. Each of the defined results in the wcat.sql statement will replace each respective value encompassed between percentage (%) signs (e.g., %URI% will be replaced with the URI generated from the wcat.sql statement).

wcat.sql

SELECT TOP 1000
   EXTRACT_TOKEN(cs-uri-query,0,'|') AS Query,
   STRCAT(cs-uri-stem, REPLACE_IF_NOT_NULL(cs-uri-query,STRCAT('?',Query))) AS URI,
   sc-status AS STATUS,
   cs-method AS VERB,
   COUNT(*) AS TOTAL,
   SEQUENCE(1) AS UID
INTO %outfile%
FROM %logfile%
WHERE (cs-method = 'GET') AND (cs-uri-stem LIKE '%.asp')
GROUP BY URI, VERB, STATUS, Query
ORDER BY TOTAL DESC

The wcat.sql file holds the query that will go through the given log files and using wcat.tpl as the template, generate the UBR file that we will later use for WCAT. Those of you familiar with SQL will notice the similarities. In our setup, we will pass in the input log files via command line but you can hard-code a specific file if you’d like in place of %logfile%. There are plenty of other queries that can be used and the Log Parser program includes a bunch of samples found inside the program’s installation directory under the Samples\Queries sub-directory. In this case, we’re querying for the top 1,000 ASP GET pages (as opposed to POST pages). Please note that it looks like WCAT doesn’t work with more than 1,000 results. Maybe the newer WCAT 6.3 will be able to handle a greater load.

Once you have those two files setup, copy whatever IIS log files that you want to evaluate into the same directory. Next, go to Start > Run… and type in “cmd” without the quotes to open up the command prompt. At the prompt, type in the following command:

Logparser file:wcat.sql?logfile=*.log+outfile=config.ubr -i:IISW3C -o:tpl -tpl:wcat.tpl

What this will do is run Log Parser using the wcat.sql as the query over all files with the extension .log in the same directory and output the results into a file called config.ubr. Notice that we specify wcat.tpl as the template. Move this config.ubr file into the same location as in step 1a, which is the WCAT Controller directory.

Step 2

To actually run WCAT, you’ll need to represent the controller and to represent the client(s). In our setup, we’ll keep it simple with one controller and one client. Open up two cmd windows (Start > Run… > cmd). In the first cmd window, cd into the WCAT Controller directory. Run the following:

wcctl.exe -z cls_currentlog.ubr -a hostname.domain.com

A lot of documents online use localhost instead of hostname.domain.com but that caused problems because we kept getting 404 errors. Depending on your setup, you might want to try either case.

In the second cmd window, cd into the WCAT Client directory and run the following:

wcclient.exe localhost

In this case, we didn’t have any issues using localhost, regardless of what was set when running the controller above.

At this point, WCAT should run and you will get on-screen feedback in the cmd windows of the progress. When it’s all done, you’ll get a log file in the WCAT Controller directory.

Results

At this point, if you open up the log file, you’ll see results, such as the following:

WCAT Version         = 5.2
ConfigFile           =
ScriptFile           =
DistribFile          =
PerfCounterFile      =
LogFile              = wcctl.exe.log
Author               =
Creation Date        =
Test Run Date        = Mon Mar 19 00:16:40 2012

Comment              =
Server [IpAddr]      =  [hostname.domain.com]
Clients              = 1
Threads              = 2500
Buffer Size          = 8388608 bytes
Duration             = 180 seconds (Warmup 5 seconds, Cooldown 6 seconds)

Results:

                        Data, Summary, Rate, 127.0.0.1

                   Client Id,          0,       0.00,      1,
                    Duration,        180,       1.00,    180,
          Total Transactions,       6178,      34.32,   6178,
              Total Requests,       6178,      34.32,   6178,
             Total Responses,      15484,      86.02,  15484,
                Total 200 OK,       6178,      34.32,   6178,
          Total 30X Redirect,          0,       0.00,      0,
      Total 304 Not Modified,          0,       0.00,      0,
         Total 404 Not Found,       9306,      51.70,   9306,
      Total 500 Server Error,          0,       0.00,      0,
           Total Misc Status,          0,       0.00,      0,
          Redirect Responses,          0,       0.00,      0,
           Avg Response Time,       6655,      36.97,   6655,
           Min Response Time,          0,       0.00,      0,
           Max Response Time,      25647,     142.48,  25647,
        StdDev Response Time,       4098,      22.77,   4098,
              Total Connects,     102138,     567.43, 102138,
            Avg Connect Time,          0,       0.00,      0,
            Min Connect Time,          0,       0.00,      0,
            Max Connect Time,         94,       0.52,     94,
         StdDev Connect Time,          2,       0.01,      2,
              Connect Errors,          0,       0.00,      0,
              Receive Errors,          0,       0.00,      0,
                 Send Errors,          0,       0.00,      0,
      Internal Memory Errors,       4059,      22.55,   4059,
           No Headers Errors,          0,       0.00,      0,
       No Status Code Errors,          0,       0.00,      0,
           Bad Status Errors,       9306,      51.70,   9306,
  Bad Response Header Errors,          0,       0.00,      0,
    Bad Response Data Errors,          0,       0.00,      0,
 No Redirect Location Errors,          0,       0.00,      0,
Bad Redirect Location Errors,          0,       0.00,      0,
                   Data Read,   77380938,  429894.10, 77380938,
                Header Bytes,    3702946,   20571.92, 3702946,
                 Total Bytes,   81083884,  450466.02, 81083884,
         Avg Header per Page,        599,       3.33,    599,
          Avg Bytes per Page,      13124,      72.91,  13124,

Performance Counters:

                                    CounterName,     Average,   Avg per Page

                    System\Context Switches/sec,         0.00,       0.00
                        System\System Calls/sec,         0.00,       0.00
                               System\Processes,         0.00,       0.00
                  System\Processor Queue Length,         0.00,       0.00
             Processor(_Total)\% Processor Time,         0.00,       0.00

             Files Requested,     102138,     567.43, 102138,
                  Files Read,       6178,      34.32,   6178,

          Total Negotiations,     102138,     567.43, 102138,
        Negotiation Failures,      82595,     458.86,  82595,
        Avg Negotiation Time,        929,       5.16,    929,
            StdDev Neg. Time,       2443,      13.57,   2443,
        Min Negotiation Time,          0,       0.00,      0,
        Max Negotiation Time,      26005,     144.47,  26005,

Per Class Statistics:

                   4 Fetched,       6178,      34.32,   6178,
                   4 Errored,      95960,     533.11,  95960,
                   4 distrib,      10000,      55.56,  10000,

In this case, we did a stress test involving one machine and 2500 threads stress testing one page (classid 4 in this case). The results provide a good aggregate stress testing view of the system and I found it easier to visualize if these results were converted into a spreadsheet along with a graph.

Hints

  • If you keep getting 100% 404 errors and zero 200 status messages, don’t use “localhost” but rather use the fully-qualified server hostname1.
    wcctl.exe -z currentlog.ubr -a hostname.domain.com
  • To perform POST operations, adjust the config file as follows. Note that the posted data is presented as parameter1 and parameter2.
    NEW TRANSACTION
    	classId = 1
    	NEW REQUEST HTTP
    		ResponseStatusCode = 200
    		Weight = 1
    		Verb = "POST"
    		RequestHeader = "Content-Type: application/x-www-form-urlencoded\r\n"
    		URL = "http://www.testdomain.com/somepagepost.asp"
    		RequestData = "parameter1=foo&parameter2=bar\r\n"

References

  1. http://stackoverflow.com/questions/1026320/wcat-problem
  2. http://technet.microsoft.com/en-us/library/bb742409.aspx
  3. http://msdn.microsoft.com/en-us/library/ms951774.aspx#usingwcat_topic9
  4. http://wcat.wikispaces.com/GettingStarted
  5. http://blogs.iis.net/mukhtard/archive/2009/03/16/troubleshooting-wcat-web-capacity-analysis-tool-problems.aspx
  6. http://blogs.iis.net/mukhtard/archive/2009/04/02/sample-wcat-script-files.aspx
  7. http://blogs.iis.net/chrisad/archive/2006/07/19/WCAT_3A00_–Easy_2C00_-Magical_2C00_-Stress-Testing-for-IIS-Web-Applications.aspx
  8. http://forums.iis.net/p/1150332/1873609.aspx
  9. http://forums.iis.net/t/1164314.aspx
  10. http://forums.iis.net/t/1150356.aspx
  11. http://stackoverflow.com/questions/114787/stress-testing-asp-net-iis-with-wcat
  12. http://blogs.msdn.com/b/alikl/archive/2008/03/09/stress-test-asp-net-web-application-with-free-wcat-tool.aspx
  13. http://blogs.technet.com/b/mscom/archive/2005/10/12/412394.aspx
  14. http://blogs.technet.com/b/mscom/archive/2005/11/03/performance-its-like-an-onion.aspx
  15. http://www.dotnetguy.co.uk/post/2010/04/17/using-wcat-and-perfmon-to-performance-test-your-aspnet-application
  16. http://www.techrepublic.com/article/talking-shop-test-your-iis-servers-performance-with-wcat/5031854
  17. http://theether.net/kb/100128
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s