JMeter Load Testing This contains an analysis of JMeter, the popular open-source Java-based tool for load testing, in contrast with LoadRunner (which precedes it). This page aims to succintly describe how to install, configure, and use JMeter on Windows, minus the attitude and sales generalizations. See related pages on load testing tools, Testing

Topics this page: What is JMeter? Popularity Trends Country Popularity People Architecture Installation Folders/Env Vars Invoke Commands Properties Test Plans Test Elements JUnit Extending JMeter Custom Monitoring Your comments?

Site Map   About this site   Enter your search terms Submit search form Web   WilsonMar.com

What Is JMeter?

JMeter's home page is jemeter.apache.org. [ Wikipedia]

Like other software used to conduct load and stress testing, JMeter (like LoadRunner wire protocols) emulates what goes over the wire in and out between a browser and the server under test.

• JMeter does not process the Javascript applets embedded in HTML pages (such as dynamic AJAX calls), which LoadRunner TruClient scripts do.

JMeter is able to record traffic in native WinInet protocol as well as HTTP protocol, which may explain why OpenSTA response times stats vary from LoadRunner and JMeter for the same app/machine

Unlike commercial software such as LoadRunner, JMeter is built by a open-source community of unpaid volunteers. It was available from Dec. 2003 until 2011 at http://jakarta.apache.org/jmeter as a sub-project supported by the Apache SoftwareFoundation (ASF).

JMeter FAQ Wiki at Apache.org

Popularity Trends

The trend of jobs by posting aggregator Indeed.com dynamically calculates the trend of jobs posted for JMeter vs. LoadRunner since 2005.



LoadRunner, JMeter Job Trends

Loadrunner jobs - Jmeter jobs

The trend is that around 2014, more JMeter jobs were posted than LoadRunner jobs. THe extrapolation is for no LoadRunner jobs by 2015.

POP-UP: This trend graph by Google illustrates that around 2010 JMeter began to be searched more than LoadRunner.


Over the years, that spread has widened.

Countries

When placed side-by-side, Google's search stats by country (as of July 15, 2014) reveals differences by country:


Despite state-imposed limitations on use of Google search, more people from China (perhaps using proxy servers) used Google to search for LoadRunner than even India, where legions of outsourcing employees live. So interest in China must be much larger than indicated. That's not hard to imagine because 9 of the top 25 websites in the world are for China. Alibaba.com has more business than Amazon and EBay combined.

Curiously, search for JMeter in China is only a third of interest in India. Yet HP seems to be investing most of its LoadRunner marketing efforts in the US, when the United States account of just 17% of what China has. Click on "View Changes over time" control on the Google page and note that interest from China surged to the top during the 2nd half of 2013.


The disparity of general population size versus interest in JMeter is strongest in Japan, and not on the LoadRunner country list at all.

People Behind the Social of JMeter

The Ohloh website by Black Duck tracks contributions to JMeter core.

JMeter's maintainers include Mike Stover and others active on the
JMeter Mailing List. This group is the one issuing Twitter feed @ApacheJMeter.

The main contributor to JMeter core (since 2004) is Sebastian Bazley (sebb), a VP of the Apache Software Foundation.

Philippe Mouawad (@pmouawad) works in France at ubik-ingenierie.com (@ubikloadpack) which sells UBIK Load Pack Productivity extensions for Apache JMeter to load test GWT / GWT-RPC, AMF / FLEX, Applets and Spring. It answers many questions on Stackoverflow about JMeter.

While BlazeMeter is one of the sponsors,
Atlanbh is among the most prolific of acknowledged contributors to
Custom JMeter Plug-ins (Twitter @jmeter_plugins).

Loadosophia.org (Twitter @loadosophia). displays results of JMeter runs uploaded to its site from Google accounts.

Ophir Prusak of BlazeMeter presents YOUTUBE: Learn JMeter in 60 minutes in the YOUTUBE: JMeter Tutorials channel and YOUTUBE: BlazeMeter Support channel from BlazeMeter provides higher scalability and other features than what JMeter provides on its own.

Blog at QGIS about Jmeter v2.4 by Tomáš Pospíšek in 4.1.2011. A part of a series of blogs about JMeter.

Enjoy the ads with easy navigation to tutorials at Tutorialspoint.com

Alexander Fuhr's German website on JMeter by has screen prints which shows off JMeter's localization.

JMeter Architecture

JMeter is written in the Java language, with a generated Javadoc.

In early versions, Jmeter consisted of a Master system (the Jmeter GUI) which controls remote slave systems running jmeter-server instances which concurrently imposes load on a target server, network or object under test by emulating traffic to and from client software.

The JMeter GUI (ApacheJmeter.jar in the bin folder) is a multi-threaded Java class running Java Swing interfaces. It is invoked using jmeter.bat.

jmeter-server communicates with multiple remote injector Java RMIRegistry services.

Remote servers by default listens to port 1099.

A Test Plan is a container for elements which specifies the parameters for test runs.

Config. elements and Listeners can be on any level.

Samplers store monitor server metrics into .jtl (JMeter Test Log) files.

Unlike LoadRunner, JMeter doesn’t have built-in recording to generate code for playback. So Badboy is installed for that.

Each Thread Group simulates what LoadRunner calls an individual virtual user. Each thread is a unit of work that can be executed simultaneously or sequentially.
To each thread group can be added Logic Controllers and Elements.

JMeter has a multi-threaded architecture that enables Java developers to extend JMeter with custom plugins and functionality extensions, listed at code.google.com/p/jmeter-plugins/, described here.

JMeter presents graphs of run results (end-to-end performance under loads over time). Does LoadRunner provide wider flexiblity in formatting and more fine-grained analysis?

The jp@gc Graphs Generator from

DevX article by Javid Jamae and Kulvir Bhogal

Build Quality Apps with JMeter 5 page article (dated August 2002) by Peter Varhol

Download and Installation on Macs

Use Homebrew to also add plugins JMeterPlugins Standard, Extras, ExtrasLibs, WebDriver and Hadoop:

brew install jmeter --with-plugins

Open jmeter.sh where it was installed:

open /usr/local/bin/jmeter

To select Mac System Look and feel: Menu > Options > Look and Feel > System

Download and Installation on Windows 7

Identify the folder path where you will be downloading JMeter.
Example: C:\projects\Perf\
If you use the Eclipse IDE:
Example: ...\eclipse\workspace\

Download from the middle of this page a JMeter Binary file such as apache-jmeter-2.11.zip for Windows machines. Download .tgz for use on Linux or Apple Macs.

Installer files have grown over the years

2.3.2.zip created 6/10/2008 was 16.6 MB.
2.11.zip  created 6/10/2014 was 31.2 MB (6 years later).

If there is a difficulty, change the mirror listed.

Verify that the downloaded file contains all the bytes.
Since Microsoft does not provide an MD5 verification program with its operating system, you need to download one from fourmilab or pc-tools.net or slavasoft.

Click on the [md5] link associated with the JMeter file you downloaded. Highlight and copy what looks like this:

314099349e1b7edc5de7005d9311c5ad *jmeter-2.11.zip

There is no need to also download the JMeter Source files if you are not intending to add to it.

Folders and Environment Variables

• Unzip folder apache-jmeter-2.x.x into the folder you designated.

For reference later, after unzipping, we copy the zip file inside the expanded folder.

JMeter is a Java application. But JMeter ignores the environment variables CLASSPATH referenced by other Java programs.

• Determine the file path where java.exe and javaw.exe are installed. A common value is:
C:\Program Files\Java\jre1.6.0_07
• Specify the path in a new environment variable JMETER_HOME.

Files from JMeter are organized into folders common to most Unix distributions:

• bin contains .cmd and .bat Windows command files to invoke JMeter.
  bin also contains configuration text file jmeter.properties which stores settings such as IP addresses of remote_hosts setup to run JMeter.
  To enable JMeter to locate these files on Windows machines, create system environment variable JMETER_BIN to specify a path to the Jmeter bin folder. CAUTION: The path value must end in \.
• bin\testfiles is present if src was unzipped.
• docs\api contains static HTML generated from Jmeter java code packages starting from file \docs\api\overview-summary.html. This references files in folder CSS and images above its level. If you're low on disk space, this can be deleted since you can view them online here.
• extras contains sample jmeter-results-reports (in XML) and
  ant-jmeter-1.0.9.jar. To run JMeter test plan programatically using Ant, use this ant task script
• lib contains jar (java archive) files. JMeter automatically finds JMeter components in jars within the
  lib\ext directory (containing by default ApacheJMeter jar files). So jars (not zips) such as JDBC and new custom JMeter components should go there.

lib\junit contains a test.jar
lib\api contains geronimo jars if src was installed.
lib\doc contains a velocity jar if src was installed.
• printable_docs\usermanual contains user manuals. To save disk space, these files can be deleted if you refer to them online http://jmeter.apache.org/usermanual/.
  JMeter Proxy Step-by-Step
  jmeter distributed testing step_by_step user manual
  Include Controller Tutorial
  AccessLog Sampler Step-by-Step
  junit Sampler Tutorial
• printable_docs\Extending folder contains How to Write a plugin for Jmeter to extend Jmeter
• printable_docs\demos contains jmx files that specify demo test plans (in XML format). Custom test plans can be recorded via HTTP and HTTPS using badboy ($45 after 15 days).
• xdocs folder exists if src was installed.
• src folder exists if src was installed.
• test folder exists if src was installed.

jmeter.bat Invocation Commands

The JMeter GUI is invoked by specifying the jar file which java.exe JRE invokes from within the JMETER_BIN folder:

-jar "%JMETER_BIN%ApacheJMeter.jar"

This statement is within bin file jmeter.bat, which creates environment variable ARGS containing all parameters jmeter uses, which includes the PERM variable to specify JVM permanent space allocation.

If needed, system environment variable JVM_ARGS can be defined to specify additional java options needed by custom Java extensions, e.g. -Dprop=val

Additionally, run parameters read by the JMeter java program can be specified:

-v or --version displays the version.

-n specifies JMeter to run in non-gui (command line) mode.

-t precedes the name of the .jmx file containing the Test Plan.

-l precedes the name of the .jtl JMeter Test Log file containing sample results.

-r or --runremote runs all remote servers specified in jmeter.properties The default is the localhost:

remote_hosts=127.0.0.1

-R overrides jmeter.properties with a list of server IPs separated by commas.

If the site to be tested is behind a corporate proxy, add:

-H my.proxy.server -P 8000 -u username -a password

In the bin folder .cmd files jmeter-n.cmd and jmeter-n-r.com have these parameters pre-specified. The "-n" and "-r" in the name reflect the JMeter run parameters within each cmd file.

A jmx test file can be executed by dragging it from one Windows Explorer and dropping it onto the cmd file shown on another Windows Explorer window.

If you see a black screen flash and disappear, a pause command is needed.

Alternately, for the convenience of one-click invocation, we create a custom cmd file containing the jmx file to be executed. For example, custom invocation file SimpleTestPlan.cmd would contain:

jmeter.bat -t%JMETER_HOME%\printable_docs\demos\SimpleTestPlan.jmx
-Ljmeter.engine=DEBUG -X
pause

Java run parameters have no space between the tag (such as "-t") and the value.

-LDEBUG is equivalent to -Ljmeter.engine=DEBUG

-G defines a JMeter property sent to all remote servers.
-Jx=2 defines a local JMeter property named "x" with value "2" referenced within test plans as variable ${__property(x)}. This is used to define the threads, rampup, and loops for each group.
-D defines a Java system property. This is needed to get around connection exceptions
-Djava.rmi.server.hostname=192.168.5.6

-X exits the server after running a single test. This overrides the properties value to exit after running a single test:

server.exitaftertest=true

A full list of the parameter can be obtained by:

jmeter -?

jmeter.properties

See usermanual jmeter_proxy_step_by_step.pdf on setting up the proxy to record a HTTP session.

There are several reasons to edit the jmeter.properties file.

To exit after running a single test:

server.exitaftertest=true

The jmeter.properties file contains references to other properties files:

• system.properties
• user.properties

Test Plan .jmx files

Sample test plans from the installer in folder \printable_docs\demos are:

AssertionTestPlan.jmx
AuthManagerTestPlan.jmx
BeanShellAssertion.bsh
ForEachTest2.jmx
forEachTestPlan.jmx
HeaderManagerTestPlan.jmx
InterleaveTestPlan2.jmx
InterleaveTestPlan.jmx
JMSPointToPoint.jmx
LoopTestPlan.jmx
OnceOnlyTestPlan.jmx
ProxyServerTestPlan.jmx
SimpleTestPlan.jmx of Jakarta users
URLRewritingExample.jmx

Within a .jmx file, the first line identifies it as being encoded in UTF-8. So it's better to open .jmx files using a text editor such as WordPad, which knows to save in that format. (rather than Notepad).

<jmeterTestPlan version="1.2" properties="2.1">
This second line specifies which version of the JMeter API to which the file was defined.

All content sections are nested within hashTree tags. I don't know why.

Values in the TestPlan section are pulled from properties files.

The number within ThreadGroup.start_time and end_time (1211836583000 in v3.2) is (when the trailing "000" is removed), the Unix Epoch time for "Mon, 26 May 2008 21:16:23 UTC".

YouTube Video:
jmeter getting started uses a Mac to create a HTTP Proxy.

Jörg Erdmenger's vimeo:
Apache JMeter - The basics tutorial on contextual settings.
Apache JMeter Getting Started Tutorial
Apache Jmeter with a simulated login

My Sample .jmx

Download and open this template file I start with when creating a JMeter Test Case (based on this wiki page).

Test Elements

From: An Amazing Open-Source Performance Testing Tool – Jmeter written by Gallop, an Indian outsourcer.

Right click on a thread group to add elements, which are executed in this order within each controller:

1. Pre-Processors execute functions before requests are sent:

   HTML Link Parser, HTTP URL Re-writing Modifier, HTML Parameter Mask, HTTP User Parameter Modifier, User Parameters, Counter, BeanShell PreProcessor

2. Timers add delays between responses: Constant, Gaussian Random, Uniform Random, Constant Throughput, Synchronizing, BeanShell
3. Samplers (introduced with v2.1.2) are protocol specific. Protocols supported by JMeter samplers include:
   • HTTP Request
   • FTP Request
   • JMS (Java object request)
   • JDBC Request
   • LDAP Request Extended Ops
   • SOAP/XML-RPC Request
   • WebService (SOAP) Request
   • JNDI
   • POP Mail (not SMTP)
   • TCP

   The JUnit Sampler includes methods for "oneTimeSetUp" and "oneTimeTearDown". These should be thread-safe (not static methods).

4. Post-Processors execute functions after requests are sent.

   Regular Expression Extractor, XPath Extractor, Result Status Action Handler, Save Responses to a file, Generate Summary Results, BeanShell PostProcessor

5. Assertions determine if the application is returning the results expected: Response, Duration, Size, XML, BeanShell, MD5Hex, HTML, XPath, XML Schema
6. Listeners perform data collection, aggregation, and visualization (display):

   Sample Result Save Configuration, Graph Full Results, Graph Results, Spline Visualizer, Assertion Results, Aggregate Report, View Results in Table, Simple Data Writer, Monitor Results, Distribution Graph (alpha), Aggregate Graph, Mailer Visualizer, BeanShell Listener, Summary Report

   View Results Tree requires a large amount of memory and processing resources.

   • Logical controller elements tell JMeter when to send requests: Simple, Once Only, Loop, Random, Random Order, If, While, Switch, ForEach, Module, Throughput, Runtime, Interleave, Include, Transaction, Recording
   • Generative Controller elements communicate with applications.

"Parsing XML is CPU intensive. On a Pentium 4 2.4ghz cpu, 50 threads is the upper limit."

Data Parametization

JMeter Wiki page How do I parameterize my JMeter test cases? outlines Regular Expressions to define user id's and other variables.

JUnit Integration

JMeter scans jar files in jmeter/lib/junit (instead of /lib directory) for classes extending junit's TestCase class.

TestNG ???

Extending JMeter

JMeter's 9.8MB Java 1.4/AWT-based source can be altered to add plug-ins that reference custom objects such as those in custom JDBC apps.

Download the source.

Run Results Graphs

The JMeter GUI recognizes key combinations, which are faster than menu:

To stop threads after current work is done: Ctrl + comma
To stop threads immediately: Ctrl + period

http://code.google.com/p/jmeter-reports aims to improve JMeter's reports presentation of a performance tests execution for a better and easier analysis of the results.

Custom Monitoring of JMeter

JMeter collects its own metrics from servers under test (such as CPU utilization), so if servers already have monitors in place, it's better to run JMeter without its normal monitors, and have the monitoring software monitor how many thread groups (vusers) is running.

Here's how to instrument JMeter for the Intersperse console.

1. Discover jmeter mbeans using the Intersperse JMX Remote Agent.

2. Open the Jmeter.bat file and add a JAVA_OPTIONS environment variable that uses "-D" to specify several Java variables to JMeter at start-up:
   set JAVA_OPTIONS=%JAVA_OPTIONS%
   -Dcom.sun.management.jmxremote
   -Dcom.sun.management.jmxremote.port=8004
   -Dcom.sun.management.jmxremote.authenticate=false
   -Dcom.sun.management.jmxremote.ssl=false

3. After adding %JAVA_OPTIONS% in Jmeter.bat file, the command to invoke Java should look like this:
   %JM_START% %JM_LAUNCH% %JVM_ARGS% %ARGS% %JAVA_OPTIONS% -jar ApacheJMeter.jar %JMETER_CMD_LINE_ARGS%

4. Install the Intersperse JMXRemoteAgent into a separate directory on the JMeter machine:
   Launch the Intersperse Installer. Choose "Custom Install" and only select "JMXRemoteAgent".
5. Invoke the JMX Remote Agent webconsole and add information about the JMeter server. This is done instead of manually editing the JMXRemoteAgent install directory file intersperse.jmxremoteagent.jsr160.servers within .... JMXRemoteAgent\config\JMXRemoteAgent\ where "host" is the machine hosting JMeter:

   remote.1.server.url=service\:jmx\:rmi\:///jndi/rmi\://host\:8004/jmxrmi
   remote.1.server.name=JMeter

   Note: The "\:" escape characters are automatically converted.

6. Save the file and exit.
7. Start the Jmeter application using jmeter.bat.
8. Start the Intesperse JMX Remote Agent.
9. On the Intersperse Console, view the Jmeter Mbeans Thread count, by navigating to "Threading Mbean" along with other Mbean "OS Monitoring" (such as cpu usage and memory usage).

Issues and Criticisms

JMeter Java code is not "object oriented" in that it has a flat name and value pairs.

FunkLoad test is Python based. webunit.

Search in this site Search in all websites in Whatis.com in Wikipedia in Amazon.com

Next: LoadRunner

Your rating of this page: [Alt+R] Low High Your comments on this topic, please: [Alt+C] Publish this comment publicly [Alt+P] Privacy Policy

Your first name: [Alt+N] Your family name: [Alt+F] Your location (city, country): [Alt+L] Your Email address: [Alt+Y] Email me Updates [Alt+U]

[Alt+G] Top of Page Thank you!

Unless otherwise indicated ©Copyright 1996-2011 Wilson Mar. All rights reserved. | HTMLHELP | W3 | CSS | Atom