How I may help
LinkedIn Profile Email me!
Call me using Skype client on your machine

JMeter Load Testing

This contains an analysis of JMeter, the popular open-source Java-based tool for load testing. This succintly describes (minus the sales hype) how to install, configure, and use JMeter on Windows.

See related pages on load testing tools, Testing

 

Topics this page:

  • Why JMeter?
  • Architecture
  • Installation
  • Folders/Env Vars
  • Invoke Commands
  • Properties
  • Test Plans
  • Test Elements
  • JUnit
  • Extending JMeter
  • Custom Monitoring
  • Your comments???
  •  

    RSS XML feed for load testers RSS preview for load testers Site Map List all pages on this site 
    About this site About this site 
    Go to first topic Go to Bottom of this page


    Set screen What Is JMeter?

      tool JMeter's home page is jemeter.apache.org. [ Wikipedia]

      Like other software used to conduct load and stress testing, JMeter (like LoadRunner wire protocols) emulates what goes over the wire in and out between a browser and the server under test.

      • JMeter does not process the Javascript applets embedded in HTML pages (such as dynamic AJAX calls).

      JMeter is able to record traffic in native WinInet protocol as well as HTTP protocol, which may explain why OpenSTA response times stats vary from LoadRunner and JMeter for the same app/machine

      Unlike commercial software such as LoadRunner, JMeter is built by a open-source community of unpaid volunteers. It was available from Dec. 2003 until 2011 at http://jakarta.apache.org/jmeter as a sub-project supported by the Apache Foundation.

      JMeter's maintainers include Mike Stover and others active on the
      JMeter Mailing List.

      Although JMeter presents graphs of run results (end-to-end performance under loads over time), LoadRunner, provides wider flexiblity in formatting and more fine-grained analysis.

     

    Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen JMeter Architecture

      It consists of a Master system (the Jmeter GUI) which controls remote slave systems running jmeter-server instances which concurrently imposes load on a target server, network or object under test by emulating traffic to and from client software.

      The JMeter GUI (ApacheJmeter.jar in the bin folder) is a multi-threaded Java class running Java Swing interfaces. It is invoked using jmeter.bat.

      jemeter-server communicates with multiple remote injector Java RMIRegistry services.

      Remote servers by default listens to port 1099.

      Samplers monitor server metrics into .jtl (JMeter Test Log) files.

      Each Thread Group simulates what LoadRunner calls an individual virtual user.
      Each thread is a unit of work that can be executed simultaneously or sequentially.
      To each thread group can be added Logic Controllers and Elements.

      A Test Plan is a container for elements which specifies the parameters for test runs.

      Unlike LoadRunner, JMeter doesn’t have built-in recording to generate code for playback. So Badboy is installed for that.

      JMeter has a multi-threaded architecture that enables Java developers to extend JMeter with custom plugins and functionality extensions, listed at code.google.com/p/jmeter-plugins/, described here.

     


    Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Download and Installation

      Identify the folder path where you will be downloading JMeter.
      Example: C:\projects\Perf\
      If you use the Eclipse IDE:
      Example: ...\eclipse\workspace\

      Download from the middle of this page a JMeter Binary file such as 2.3.2.zip for Windows machines (16.6 MB created 6/10/2008).

    • Unzip folder jakarta-jmeter-2.3.2 into the folder designated.
    • For reference later, after unzipping, we copy the zip file inside the expanded folder.

    • We also download the JMeter Source file such as 2.3.2.zip for Windows machines (5.93 MB created 6/10/2008).

    • Specify the path in a new environment variable JMETER_HOME.
    • Verify that the downloaded file contains all the bytes.
      Since Microsoft does not provide an MD5 verification program with its operating system, you need to download one from fourmilab or pc-tools.net or slavasoft.

      Click on the [md5] link associated with the JMeter file you downloaded. Highlight and copy what looks like this:

      314099349e1b7edc5de7005d9311c5ad *jakarta-jmeter-2.3.2.zip

    Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Folders and Environment Variables

      JMeter is a Java application. But JMeter ignores the environment variables CLASSPATH referenced by other Java programs.

    • Determine the file path where java.exe and javaw.exe are installed. A common value is:
      • C:\Program Files\Java\jre1.6.0_07

      • Specify that path value when creating system environment variable JAVA_HOME

      Files from Jakarta are organized into folders common to most Unix distributions:

      • docs\api contains static HTML generated from Jmeter java code packages starting from file \docs\api\overview-summary.html. If you're low on disk space, this can be deleted since you can view them online here.
      • printable_docs\usermanual contains a User Manual. To save disk space, these files can be deleted if you refer to them online here.
      • extras contains sample jmeter-results-reports (in XML) and
        ant-jmeter-1.0.9.jar. To run JMeter test plan programatically using Ant, use this ant task script
      • lib contains jar (java archive) files. JMeter automatically finds JMeter components in jars within the
        lib\ext directory (containing by default ApacheJMeter jar files). So jars (not zips) such as JDBC and new custom JMeter components should go there.

      • lib\junit contains a test.jar
        lib\api contains geronimo jars if src was installed.
        lib\doc contains a velocity jar if src was installed.
      • xdocs folder exists if src was installed.
      • src folder exists if src was installed.
      • test folder exists if src was installed.
      • printable_docs\Extending folder contains How to Write a plugin for Jmeter to extend Jmeter
      • printable_docs\demos contains jmx files that specify demo test plans (in XML format). Custom test plans can be recorded via HTTP and HTTPS using badboy ($45 after 15 days).
      • bin contains .cmd and .bat Windows command files to invoke JMeter.
        bin also contains configuration text file jmeter.properties which stores settings such as IP addresses of remote_hosts setup to run JMeter.

          To enable JMeter to locate these files on Windows machines, create system environment variable JMETER_BIN to specify a path to the Jmeter bin folder. CAUTION: The path value must end in \.
      • bin\testfiles is present if src was unzipped.

    Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen jmeter.bat Invocation Commands

      The JMeter GUI is invoked by specifying the jar file which java.exe JRE invokes from within the JMETER_BIN folder:

        -jar "%JMETER_BIN%ApacheJMeter.jar"

      This statement is within bin file jmeter.bat, which creates environment variable ARGS containing all parameters jmeter uses, which includes the PERM variable to specify JVM permanent space allocation.

      If needed, system environment variable JVM_ARGS can be defined to specify additional java options needed by custom Java extensions, e.g. -Dprop=val

      Additionally, run parameters read by the JMeter java program can be specified:

        -v or --version displays the version.

        -n specifies JMeter to run in non-gui (command line) mode.

        -t precedes the name of the .jmx file containing the Test Plan.

        -l precedes the name of the .jtl JMeter Test Log file containing sample results.

        -r or --runremote runs all remote servers specified in jmeter.properties The default is the localhost:

          remote_hosts=127.0.0.1

        -R overrides jmeter.properties with a list of server IPs separated by commas.

      If the site to be tested is behind a corporate proxy, add:

        -H my.proxy.server -P 8000 -u username -a password

      In the bin folder .cmd files jmeter-n.cmd and jmeter-n-r.com have these parameters pre-specified. The "-n" and "-r" in the name reflect the JMeter run parameters within each cmd file.

      A jmx test file can be executed by dragging it from one Windows Explorer and dropping it onto the cmd file shown on another Windows Explorer window.

      If you see a black screen flash and disappear, a pause command is needed.

      Alternately, for the convenience of one-click invocation, we create a custom cmd file containing the jmx file to be executed. For example, custom invocation file SimpleTestPlan.cmd would contain:

      jmeter.bat -t%JMETER_HOME%\printable_docs\demos\SimpleTestPlan.jmx
      -Ljmeter.engine=DEBUG -X
      pause

        Java run parameters have no space between the tag (such as "-t") and the value.

        -LDEBUG is equivalent to -Ljmeter.engine=DEBUG

        -G defines a JMeter property sent to all remote servers.
        -Jx=2 defines a local JMeter property named "x" with value "2" referenced within test plans as variable ${__property(x)}. This is used to define the threads, rampup, and loops for each group.
        -D defines a Java system property. This is needed to get around connection exceptions
        -Djava.rmi.server.hostname=192.168.5.6

        -X exits the server after running a single test. This overrides the properties value to exit after running a single test:

          server.exitaftertest=true

      A full list of the parameter can be obtained by:

      jmeter -?

    Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen jmeter.properties

      There are several reasons to edit the jmeter.properties file.

      To exit after running a single test:

        server.exitaftertest=true

      See usermanual jmeter_proxy_step_by_step.pdf on setting up the proxy to record a HTTP session.

      The jmeter.properties file contains references to other properties files:

      • system.properties
      • user.properties

     


      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Test Plan .jmx files

      Sample test plans from the installer in folder \printable_docs\demos are:

        AssertionTestPlan.jmx
        AuthManagerTestPlan.jmx
        BeanShellAssertion.bsh
        ForEachTest2.jmx
        forEachTestPlan.jmx
        HeaderManagerTestPlan.jmx
        InterleaveTestPlan2.jmx
        InterleaveTestPlan.jmx
        JMSPointToPoint.jmx
        LoopTestPlan.jmx
        OnceOnlyTestPlan.jmx
        ProxyServerTestPlan.jmx
        SimpleTestPlan.jmx of Jakarta users
        URLRewritingExample.jmx

      Within a .jmx file, the first line identifies it as being encoded in UTF-8. So it's better to open .jmx files using a text editor such as WordPad, which knows to save in that format. (rather than Notepad).

      <jmeterTestPlan version="1.2" properties="2.1">
      This second line specifies which version of the JMeter API to which the file was defined.

      All content sections are nested within hashTree tags. I don't know why.

      Values in the TestPlan section are pulled from properties files.

      The number within ThreadGroup.start_time and end_time (1211836583000 in v3.2) is (when the trailing "000" is removed), the Unix Epoch time for "Mon, 26 May 2008 21:16:23 UTC".

     


      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen My Sample .jmx

      Download and open this template file I start with when creating a JMeter Test Case (based on this wiki page).

     


      .

      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Test Elements

      Right click on a thread group to add elements, which are executed in this order within each controller:

      1. Pre-Processors execute functions before requests are sent:

        HTML Link Parser, HTTP URL Re-writing Modifier, HTML Parameter Mask, HTTP User Parameter Modifier, User Parameters, Counter, BeanShell PreProcessor

      2. Timers add delays between responses: Constant, Gaussian Random, Uniform Random, Constant Throughput, Synchronizing, BeanShell
      3. Samplers (introduced with v2.1.2) are protocol specific. Protocols supported by JMeter samplers include:

        The JUnit Sampler includes methods for "oneTimeSetUp" and "oneTimeTearDown". These should be thread-safe (not static methods).

      4. Post-Processors execute functions after requests are sent.

        Regular Expression Extractor, XPath Extractor, Result Status Action Handler, Save Responses to a file, Generate Summary Results, BeanShell PostProcessor

      5. Assertions determine if the application is returning the results expected: Response, Duration, Size, XML, BeanShell, MD5Hex, HTML, XPath, XML Schema
      6. Listeners perform data collection, aggregation, and visualization (display):

        Sample Result Save Configuration, Graph Full Results, Graph Results, Spline Visualizer, Assertion Results, Aggregate Report, View Results in Table, Simple Data Writer, Monitor Results, Distribution Graph (alpha), Aggregate Graph, Mailer Visualizer, BeanShell Listener, Summary Report

        View Results Tree requires a large amount of memory and processing resources.

        • Logical controller elements tell JMeter when to send requests: Simple, Once Only, Loop, Random, Random Order, If, While, Switch, ForEach, Module, Throughput, Runtime, Interleave, Include, Transaction, Recording
        • Generative Controller elements communicate with applications.

     


      "Parsing XML is CPU intensive. On a Pentium 4 2.4ghz cpu, 50 threads is the upper limit."

      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Data Parametization

     


      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen JUnit Integration

      JMeter scans jar files in jmeter/lib/junit (instead of /lib directory) for classes extending junit's TestCase class.

      TestNG ???

     


      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Extending JMeter

      JMeter's 9.8MB Java 1.4/AWT-based source can be altered to add plug-ins that reference custom objects such as those in custom JDBC apps.

      Download the source.

     


      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Run Results Graphs

      The JMeter GUI recognizes key combinations, which are faster than menu:

        To stop threads after current work is done: Ctrl + comma
        To stop threads immediately: Ctrl + period

      http://code.google.com/p/jmeter-reports aims to improve JMeter's reports presentation of a performance tests execution for a better and easier analysis of the results.

     


      Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Custom Monitoring of JMeter

      JMeter collects its own metrics from servers under test (such as CPU utilization), so if servers already have monitors in place, it's better to run JMeter without its normal monitors, and have the monitoring software monitor how many thread groups (vusers) is running.

      Here's how to instrument JMeter for the Intersperse console.

      1. Discover jmeter mbeans using the Intersperse JMX Remote Agent.

      2. Open the Jmeter.bat file and add a JAVA_OPTIONS environment variable that uses "-D" to specify several Java variables to JMeter at start-up:

          set JAVA_OPTIONS=%JAVA_OPTIONS%
          -Dcom.sun.management.jmxremote
          -Dcom.sun.management.jmxremote.port=8004
          -Dcom.sun.management.jmxremote.authenticate=false
          -Dcom.sun.management.jmxremote.ssl=false

      3. After adding %JAVA_OPTIONS% in Jmeter.bat file, the command to invoke Java should look like this:

          %JM_START% %JM_LAUNCH% %JVM_ARGS% %ARGS% %JAVA_OPTIONS% -jar ApacheJMeter.jar %JMETER_CMD_LINE_ARGS%

      4. Install the Intersperse JMXRemoteAgent into a separate directory on the JMeter machine:
        Launch the Intersperse Installer. Choose "Custom Install" and only select "JMXRemoteAgent".
      5. Invoke the JMX Remote Agent webconsole and add information about the JMeter server. This is done instead of manually editing the JMXRemoteAgent install directory file intersperse.jmxremoteagent.jsr160.servers within .... JMXRemoteAgent\config\JMXRemoteAgent\ where "host" is the machine hosting JMeter:

        remote.1.server.url=service\:jmx\:rmi\:///jndi/rmi\://host\:8004/jmxrmi
        remote.1.server.name=JMeter

        Note: The "\:" escape characters are automatically converted.

      6. Save the file and exit.
      7. Start the Jmeter application using jmeter.bat.
      8. Start the Intesperse JMX Remote Agent.
      9. On the Intersperse Console, view the Jmeter Mbeans Thread count, by navigating to "Threading Mbean" along with other Mbean "OS Monitoring" (such as cpu usage and memory usage).

    Go to Top of this page.
    Previous topic this page
    Next topic this page

    Set screen Issues

      JMeter is not "object oriented" in that it has a flat name and value pairs.

      FunkLoad test is Python based. webunit.

     


    Go to Top of this page.
    Previous topic this page
    Next topic this page