|
Here are concise notes on using LoadRunner for performance testing. This is a companion to my pages on Vu Scripting, performance monitoring, performance tuning, and reporting.
As of November 28, 2007, the URL | Topics this page:
|
|
|
|
|
|
| Note: Links to documents that used to be here were removed after Mercury Interactive, Inc. lawyers demanded their removal. Although version 9.10 is now installed under "HP", "Mercury" remains under Program Files\Common Files, its \TDAPI\Client folder contains files TDCIntui.dll and tdclient.dll. These, the hidden folder C:\Config.Msi, the MacroVision folder (within Documents and Settings\All Users.WINDWS\Application Data), and many other files remain after uninstall. Over three thousand entries also remain within the Windows Registry after uninstall. |
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Application Product | Process Image Name | V9.0 | V8.0 Img KB | File Size | |
|---|---|---|---|---|---|
| | Launcher | LRLauncherApp.exe | 15,840 | 16,288 | n/a |
| |
Virtual User Generator | VuGen.exe | 23,980 | 12,436 | 2,334,769 |
| |
Controller | wlrun.exe | 61,312 | 13,076 | 5,681,215 |
| |
Load Generator Agent | magentproc.exe | 3,336 | 3,236 | |
| magentservice.exe | 3,496 | 65,536 | |||
| mdrv.exe | - | ||||
| |
Analysis | Analysisui.exe | 64,460 | 13,132 | 6,058,496 |
| |
Tuning Console | protune.exe | - | 3,403,833 | |
| perl5.8.0.exe | Interpreter | 20,535 |
| regtlb.exe | registers the batch automation type library | 30,720 |
| sed.exe | GNU sed (gsed) version 2.05 | 55,296 |
| wdiff.exe | Compares text files | 197,632 |
Alex Arbitman's
LR 7.8 Footprints.xls
reports that to run Web requires __ per process and __ per thread.
|
Quality Web Systems: Performance, Security, and Usability
(Addison-Wesley Professional; Aug. 2001)
by Elfriede Dustin, Jeff Rashka, Douglas McDiarmid
|
|
|
| Protocol | Server | Client Program | Parameter | Notes |
|---|---|---|---|---|
| Web | WebTours\StartServer.bat | http://localhost:1080/mercuryWebTours | ||
| COM/DCOM | (Operating System) | samples\bin\frsui.exe | ||
| Winsock | sockfrs.exe | samples\bin\flights.exe | Winsock WinSockWeb | |
| ODBC | (MS Access) | samples\bin\flights.exe | ODBC_Access | |
| CORBA | samples\CorbaSamples\server.cmd & samples\CorbaSamples\server.bat | samples\CorbaSamples\client.cmd & samples\Corbasamples\clientrecord.cmd |
Stuart Moncrieff's article on CORBA | |
| RMI | samples\RMISamples\server.cmd & samples\RMISamples\server.bat | samples\RMISamples\client.cmd & samples\RMISamples\clientrecord.cmd |
According to CPT11877.doc, JDK 1.5 users need to contact Mercury Support for a patch
to each specific LoadRunner version (7.6, 7.8 FP1 or 8.0). Otherwise, you'll get these messages:
The Java sample apps use the "flight32lr" User Data Source with Microsoft Access driver(*.mdb) in the USER DNS table in Data Sources(ODBC) of the VuGen's local machine.
Additionally, the sample Java servers must be operational prior to starting the client. This is done with the "samples\RMISamples\server.cmd":
Note the location of loadrunner class files I added to the default sample.
They are pre-pended to the existing classpath.
Note that there are no spaces in the file path.
The Zip file is equivalent to a JAR file in Unix systems.
Do not delete the black command window because the Java server runs within it.
CORBA and RMI Java clients are invoked with a command for Windows to start the java.exe program. This "samples\RMISamples\client.cmd" file contains:
Note that the RmiFlights.main class file name is passed into java for it to load.
When recording Java with VuGen, a different command — such as the sample clientRecord.cmd — needs to be invoked because VuGen needs to be invoked within the JVM sandbox:
Instead of web "Start recording",
Java VuGen scripts invoke Java functions within the Actions section.
"vuser_init" and "vuser_end" actions are not relevant within Java VuScripts.
Internally, the cjhook.ini file specifies which Java classes can hook in its [EXC_SYSTEM_CL] section. Java classes specified in the [SYSTEM_CL] section are not hooked.
The user.hooks file in LR \bin folder is a general format and cannot be used as-in. It needs to be copied.
Unlike Microsoft Office applications, Mercury has not programmed invididual components to be selectively uninstalled on its own.
|
|
|
|
|
|
|
|
|
|
|
|
Creating LoadRunner Vuser Scripts
online guide
Within the .usr file:
|
|
|
|
By default, VuGen creates a new user named after the current user name. For example, a user named "Tester" will have run results under
the C:\Documents and Settings\ folder for a user named Tester.LOADTEST.
VuGen automatically sets the Windows environment variable TEMP to %USERPROFILE%\Local Settings\Temp
so that results are written to that user's sub folder \Local Settings\Temp. The full path for user Tester would be
|
|
|
|
|
This table summarizes the typical settings for each
type of run scenario | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Menu | Setting | A. Speed | B. Contention | C. Overload | D. Longevity |
|---|---|---|---|---|---|
| Controller | # of Iterations | 1 only | Several | Infinite | |
| Run Options | Frequency of output: Sample once every | 1 second | 10 seconds | 1 minute | 5 minutes |
| Run-time Settings | Vusers | 1 only | max. licensed | # below "knee" | |
| Logging | For debugging | No | |||
| Think Time | None | Randomized Actual | |||
| Continue on error? | No | Yes | |||
| Network Speed Simulation | Maximum | No | |||
| Browser (cache) emulation | No | Yes | |||
| Content Checks | Yes | No | |||
| Schedule | Ramp-up: | Load all Vusers simultaneously | Yes | ||
| Initialize Before Run? | No | Yes | No | ||
| - | Interval (seconds) | 4 | >4 | 30 or more | |
| Tools > Options > Monitors | Server Resource Monitors: Data Sampling Rate | 3 seconds (Default) | 5 minutes | ||
When scheduled to Run until completion, the Quantity for a Scenario
is the number of vusers running one at a time.
When scheduled to Run for a period of time, the Quantity for a Scenario
is the number of vusers running simultaneously.
The time specified begins after the Ramp Up period, when all vusers have entered Run state. This specified time plus the time it took to get all vusers into Run state
becomes the total Elapsed Time of a time-limited run.
To run a specific number of vusers simultaneously, set the Parameter file in
the script (in VuGen) to Abort after reaching the end of file.
Scenario Run Time Settings make use of Run logic scripts plus:
|
|
If Run-Time Settings has the "Advanced Trace" checkbox selected before the script is run,
these lines will appear in the output log:
|
|
One of the most common headaches with load testing is running of hard disk space during a long run.
|
|
|
|
| Mode: | Scenario Duration: | Load Behavior: |
|---|---|---|
| Scenario Scheduling __ Schedule by Scenario: | Within Duration tab: __ Run Until Completion __ Run for ________ HH:MM:SS
| Within Ramp Up tab: __ Load all Vusers simultaneously (default) __ Start __ Vusers every ___ HH:MM:SS Within Ramp Down tab (if Run for limited duration): __ Stop all Vusers simultaneously __ Stop __ Vusers every ___ HH:MM:SS |
| Group Scheduling __ Schedule by Group (for each script): | Unknown duration | Defined per scenario group |
__ Initialize all Vusers before Run?
One way to determine appropriate ramp up time to specify is to set vusers to start simultaneously,
then look at the resulting rate Running Vusers drop off after processing
(such as 10 users within a 15 second span).
| ||||||||||||||||||||||||||||||||||||
| Transactions | Secondary | System Resources | ||
|---|---|---|---|---|
| Runtime: Running Vusers + # Connections | Error Statistics | UNIX Load Avg | Win Threads | |
| Transaction: Response Time (sec) | - | UNIX CPU Util | Win CPU Util | |
| Total Trans/Sec | per Second Hits + Pages Downloaded + Connections + SSL | UNIX Paging | - | |
| Throughput (bytes) | Network Delay | UNIX Disk Traffic | - | |
These are usually the most important relationships under study.
LR does not remember most scenario graph settings (4 graphs is the hard coded default).
So instead of building graphs from scratch, I start from opening and then changing my custom but standard scenario file.
Delete graph definitions you don't need to ever see.
LR collects data for graphs even if it is not displayed.
KB 26817: By default, the Controller online monitor shows a maximum of 20 measurements for each graph. To increase it, go to the LoadRunner\dat\online_graphs directory to modify the value of MaxDispMeasurments= in the file controlling each type of graph:
| Description | File Name |
|---|---|
| All | generalsettings.ini |
| System Resource Graphs | online_resource_graphs.rmd |
| Runtime Graphs | online_runtime_graphs.def |
| Transaction Graphs | online_transaction_graphs.def |
| Web Resource Graphs | online_web_graphs.def |
| Streaming Media | online_web_graphs_mms.def |
Default counters for the System Resource, Microsoft IIS, Microsoft ASP, or SQL Server monitors are defined in the res_mon.dft file within the LoadRunner/dat folder. Its values can be pasted from the [MonItemPlus] section within scenario .lrs files.
UNIX Resources and some other graphs are continuously updated even after the test is done.
So immediately after the scenario runs, right-click on the graph to freeze the values displayed
to lock in values associated with other graphs.
Installation tip: If the Web Resource Graph is blank, try re-registering .dll files by running MS-DOS Batch Files resister_controller.bat and set_mon.bat in the LoadRunner\bin folder.
Actions are executed sequentially in the order shown in Run-time Settings.
However, transactions that start and stop between two observations will appear
to be running simultaneously even if they were actually executed sequentially.
An observation interval of 4 seconds is the most often that you can set for Controller on-line graph
(to prevent too much CPU-intensive graphic refresh time from consuming the Controller machine).
But Analysis reports will show more granularity than on-line graphs
— down to 1 second.
|
| Called Service [Graph] | Protocol/Product | Template folder | Header | Notes |
|---|---|---|---|---|
| Custom / General | | lrc | lrun.h, global.h | |
| | General-Vba | - | ||
| | General-Vb | - | ||
| | General-Js | global.js | ||
|
| General-Java | - | ||
| E-Business Web Resource (eBusiness) |
| http, web, General-Js | as_web.h | |
| | FTP | mic_ftp.h | ||
| (Operating) [System Resource] |
| - | - | |
| [Network] |
| winsock WinSockWeb | lrs.h | |
| [Firewalls] |
| - | - | |
| [Web Server Resource] |
| - | - | |
| [Web Application Server] |
| - | - | |
| Client/Server [Database Server Resource] |
| Siebel_web | lrd.h
lrdtypes.h | |
| | Oracle_NCA | orafuncs.h | ||
| [Streaming Media] | | - | mic_media.h | |
| real | lreal.h | |||
| [ERP/CRM Server Resource] |
| Sapgui, SAP_Web | as_sap.gui.h | |
| | - | lrdsiebel.h | ||
| | baan | - | ||
| Distributed Components [Java Performance] | | - | - | |
| [Application Component] |
| com | - | |
| | dotNet | - | ||
| Enterprise Java Beans | | - | - | |
| [Application Deployment] |
| Citrix | ctrxfuncs.h | |
| [Middleware] |
MQMonitor avail. since Sep. 2003)
| - | - | |
|
| - | - | ||
| Mailing | | POP3 | mic_pop3.h | |
| | - | mic_smtp.h | ||
|
| IMAP | mic_imap.h | ||
|
| MAPI | mic_mapo.h | ||
| Networking | | DNS | - | |
|
| LDAP | mic_mldap.h | ||
| Application Services | | - | lrt.h | |
|
| - | - | ||
| Wireless | | I_Mode | - | |
|
| voiceXML | - | ||
|
| wap | - | ||
| | 327x (IBM mainframe) UNIX RTE (Remote Terminal Emulation) | rte | lrrte.h | |
| | F5 BIG-IP | - | - |