Master Apache JMeter From load testing to DevOps.
Master Apache JMeter From load testing to DevOps.
Prefaced by Alexander Podelko
About the Book
Throughout the chapters, you will understand how to create meaningful performance tests, discover the richness of JMeter and learn how to use it efficiently and effectively.
From automatic recording that allows you to gain in productivity and realism in writing your scripts, through correlation that makes your test realistic and reusable across environments, the authors put you on the right track to realize your scalability testing campaigns.
They then explore the different protocols offered by the tool through examples drawn from their professional experiences.
Finally, they show how to integrate the tool into the DevOps approach, create professional reports and exploit the richness of its eco-system to integrate new protocols, enrich its monitoring or leverage its power through the use of the Cloud.
About the Authors
Packages
The Book
PDF
EPUB
WEB
English
25% discount - Pack of 3
This package is for 3 copies of the "Master JMeter : from load testing to DevOps" at a 25% discount.
PDF
EPUB
WEB
English
About the Contributors
Board director for the Computer Measurement Group (CMG), a worldwide organization of performance and capacity management professionals.
Over the last twenty years Alex Podelko supported major performance initiatives for Oracle, Hyperion, Aetna, and Intel in different roles including performance tester, performance analyst, performance architect, and performance engineer.
Currently he is Consulting Member of Technical Staff at Oracle, responsible for performance testing and tuning of Hyperion (a.k.a. Enterprise Performance Management and Business Intelligence) products. Before specializing in performance, Alex led software development for Rodnik Software.
Having more than thirty years of overall experience in the software industry, he holds a PhD in Computer Science from Gubkin University and an MBA from Bellevue University.
Alex periodically talks and writes about performance-related topics, advocating tearing down silo walls between different groups of performance professionals. He currently serves as a board director for the Computer Measurement Group (CMG), a worldwide organization of performance and capacity management professionals.
Apache JMeter committer, developer and Project Management Committee member
Long term committer in JMeter project and Tomcat.
He has a diploma in mathematics, but found working in IT more appealing.
He really likes to work with open source, as it gives him the ability to dive into the inner workings of those magic things.
Reader Testimonials
Software Testing Magazine
A must have for every software tester working with JMeter
The book provides an impressive amount of knowledge on how to successfully use JMeter, from your first steps with the tool to its integration in a DevOps approach. ... This book is naturally a must have for every software tester working with JMeter, but it will also provide valuable knowledge and solution to every software tester or developer concerned with the performance of her or his application. Read full review at http://www.softwaretestingmagazine.com/knowledge/master-apache-jmeter/
Deepak Shetty
Performance Expert and JMeter committer (http://theworkaholic.blogspot.com/)
This book fills in the details for many of the problems that JMeter users run into - which is practical advice on how to use the various components of the extensible JMeter ecosystem. This book is a very good resource to developers and testers who have some exposure to testing and are now looking to get into the more advanced JMeter features and using tools that JMeter integrates with.
Anthony Gauthier
Software Developer, Performance Engineer, ElasticSearch backend listener author (https://www.github.com/delirius325)
As an active member of the community & a seasoned JMeter user, I believe this book should be a must have for any JMeter user-beginner or expert. As a beginner, it will show you all of what you need to know to successfully establish meaningful test scenarios. As an advanced user, it will be a nice reminder of what really should be in your tests as well as making you discover some JMeter features you've never really used. I highly recommend it to anyone involved in performance testing/engineering.
NaveenKumar Namachivayam
Performance test engineer https://QAInsights.com
If you are getting started with JMeter, then this book is for you; if you are already experienced in JMeter, then this book is also for you; Even if you are an expert in JMeter, then also this book is for you. It teaches both: performance testing and engineering concepts clearly with examples. After reading this book, you can load test a website, database, web services, integrate your scripts with your enterprise CI/CD pipeline.
Table of Contents
- Copyright
-
Presentation of the authors
- Antonio Gomes Rodrigues
- Bruno Demion (Milamber)
- Philippe Mouawad (Philippe M.)
-
About the reviewers
- Felix Schumacher
-
Preface by Alexander Podelko
- Alexander Podelko bio
-
How to read this book
- I just want to discover Apache JMeter
- There is nothing and I have to do everything
- I am joining a performance team
- I need to do a load test from scratch
- I need script load test
- I need to help developers to make load test
- I need to analyse and presents results from a load test
- I need to make shift left strategy
-
Quick start with JMeter
- Our process
- Setup JMeter to record our browsing session
- Configure your browser
- Save our script
- Validate our script with only one user
- Setup our load test and launch it
- Run our load test and analyze it
-
JMeter overview
- JMeter overview
- Types of load tests supported
- Supported test protocols
- Reporting
- Conclusion
-
Why is it still important to make load test today?
-
Why performance matters?
- Better user experience
- Better SEO (Search Engine Optimization)
- Better conversion rate
- Reduce cost
-
What has changed in the last years and why it is not enough?
- Provisioning a server is easiest with Cloud
- More powerful server
- Autoscaling is easy with Cloud/Kubernetes
- Canary release deployment
- Blue/Green deployment
- Feature Flipping allows to quickly disable a faulty feature
- Design for failure
-
Other reasons why it is still important to make load test today
- Performance culture
- Don’t live with Broken Windows
- Put into production the healthiest application
- Fine tune as much as possible the architecture and application
- Understand the weak points of the application
- Reproduce a performance or multithreading issue detected in production to fix it
- Give business an idea of future response times when a new application goes live
- Anticipate Sales period
- Compare two versions of the same application
- Confirm/tune the alerting system
- Validate/improve monitoring (metrics, tools, dashboards, etc.)
- Find the soft limits of the AWS infrastructure
- Conclusion
-
Why performance matters?
-
Designing a test case
- Introduction
- Define your goal
-
Type of load test
- Smoke Test
- Performance/Load Test
- Stress Test
- Breakpoint Test
- Soak/Endurance Test
- Failover/Resilience Test
- Spike Test
- Other
- Summary
-
Load model
-
Closed model vs Open model
- Closed model
- Open model
- Why is it important to choose the right model?
-
Implementation in JMeter
- Concurrency user (Closed model)
- Throughput (Open model)
- Arrivals rate (Open model)
-
Arrivals rate model
- Poisson process with Precise Throughput Timer
- Constant arrival rate with Thread Group
- Constant arrival rate with Constant Throughput Timer
- Arrival rate with Open Model Thread Group
-
Think time
- Constant think time with Constant Timer
- Uniform distribution think time with Uniform Random Timer
- Gaussian/Normal distribution think time with Gaussian Random Timer
- Poisson distribution think time with Poisson Random Timer
- All think time we want with JSR223 Timer
-
Closed model vs Open model
-
Structure of a test case
- Ramp Up
- Plateau/Step
- Ramp Down
- Example: step load test
-
Identify critical business scenarios to script
- Frequent
- Vital/Business critical
- Technically risky
- Legally risky
- Brand image risk
- Summary
- How many virtual users do I need?
-
Identify Service Level Agreements (SLAs)
- Why SLA?
- Identify SLAs, use case 1: SLAs already exists
- Identify SLAs, use case 2: SLAs don’t exist, but business has a clear idea of them
- Identify SLAs, use case 3: SLAs don’t exist and business does not have a clear idea of them
- Best practices
- Implement them in Apache JMeter
-
Parameters to take into account when creating a scenario
- Vary user input data
- Reuse the same values
- Pre-calculate input data vs random input data
- Vary the type of users
- Vary and adjust think times
- Adjust the ramp up time
- Adjust the test duration
- Error handling
- User connection type
- Size of the scenarios
- Meeting point/rendez-vous
- Cookie Management
- Cache Management
- Conclusion
-
Important concepts in JMeter
- Scoping
- Elements execution order
-
How Timers scope works
- Rule 1: Timers are executed before each sampler in their scope.
- Rule 2: If there is more than one timer in the scope, all the timers will be processed before the sampler.
- Control the execution order of Timers
-
How Assertions scope works
- Rule 1: Assertions are executed after each sampler in their scope or to parent samplers
- Rule 2: Failed assertions cause the failure of the entire Transaction Controller
- Rule 3: Assertions can validate main sample and/or sub-samples
- Rule 4: Be careful with low performing assertions
- Our advice
-
How properties differ from variables
- How properties are created
- How variables are created
- How ${} differs from ${_P}
-
Preparing the test environment (injectors and tested system)
- Introduction
-
Prepare the injectors
- Never host injectors on the same servers as the tested system
- Calibrating your test
- Monitor injectors
-
Prepare Apache JMeter
- Use the latest version
- JMeter memory configuration
-
Prepare the network
-
IP Spoofing
- Setting up IP Spoofing:
- Avoid load testing behind a proxy
-
IP Spoofing
-
Prepare the tested system
- A tested environment
- An environment consistent with the target
- Datasets
- Take into account calls to third party services
- Disable protection systems
- Isolated environment
- License Management
- Summary
-
Monitoring the tested system
-
Monitoring/profiling tools and their features
- Overhead
- Metrics
- Logs
- Instrumentation
- Sampling
- Distributed tracing
- Memory monitoring
- Thread monitoring
- Exception monitoring
- Database monitoring
- Continuous Profiling/APM (Application Performance Monitoring)
- Network monitoring
- Events monitoring
-
Monitoring methodology
- USE method (Utilisation, Saturation & Errors)
- RED method (Rate, Errors & Duration)
- Four Golden Signals (Latency, Traffic, Errors & Saturation) method
- Our advice
-
Monitoring/profiling tools and their features
- Conclusion
-
Being productive with JMeter
- Introduction
-
Before starting the recording of the script
-
Choose if static resources are important or not
- Solution 1: Do not take them into account
- Solution 2: Let JMeter manage them
- Solution 3: Manage it ourselves
- Apply a naming convention for transactions/requests
- Modify or learn keyboard shortcuts
-
Choose if static resources are important or not
-
Have a script base quickly
- Pre-populate HTTP Request Defaults before recording the script
- Use the HTTP(S) Test Script Recorder element
- BlazeMeter Chrome Extension
- Use Tools / Import from cURL
- Import requests from HAR file
- Recording HTTPS website
-
Save time when working on the script
- Global Assertion
- Re-Use script parts
- Share an object between different Thread Groups
-
Get the most out of CSV files
- The different modes/options of CSV reader
- CSV file and JSON content
- Data coming from several CSV files
- CSV and Groovy
- Using Regular Expression Extractor on content with spaces and multiple lines
- Add headers to our HTTP requests
-
Waiting with While Controller
- Wait for the end of execution: example 1
- Wait for the end of execution: example 2
- Wait for three products to be added to the cart.
- Exit from While if the last action returns an error.
- The right Extractor at the right time
- Handle cookies
- Set HTTP status code 404 as success in JMeter
-
Debugging a script
- Using View Results Tree
- Capture errors
- Debug Sampler
- jp@gc - Dummy Sampler
- Log Viewer
- BlazeMeter Step-by-step Debugger
- The old-fashioned way in the console
- Use a third-party tool
-
Finalise a script
- Change the name of a transaction according to a parameter
- Mark a response as an error
- Execute an action based on the status of the previous one
- Use timeouts
- Conclusion
-
Load Testing a website
- A bit of theory
- Set up with JMeter
- Methodology
-
Put into practice with JMeter
-
Example 1: Find a owner in Spring PetClinic application
- Our scenario to script
- The recording
- Step 1: Work on “Go to the home page”
- Step 2: Work on “Do a search on the owner of a pet”
- Step 3: Work on “View the details of the owner”
-
Example 2: Add an owner/Display last owner in Spring PetClinic application
- Step 1: Add an owner
- Step 2: Display last owner
- Example 3: Technical tests with Byteman
- Example 4: Technical tests with JProfiler
- Example 5: Tricking captchas with a Java Request
-
Example 1: Find a owner in Spring PetClinic application
- Conclusion
-
Load Testing web services
- What we will learn
-
A few concepts
- REST (REpresentational State Transfer) web services
- GraphQL (Graph Query Language)
- REST vs GraphQL
-
Set up with JMeter
- Tips
- Methodology
-
Practice with JMeter
- Load test a REST web service: Spring PetClinic REST
- Load test a GraphQL web service: Spring PetClinic GraphQL
- Conclusion
-
Load Testing a database server
- A bit of theory
- Methodology
- Setup with JMeter
-
Putting theory into practice with JMeter
-
Example 1: Load test of a database
- Avoiding duplication using Include Controller
- Avoiding duplication using Module Controller
- Example 2: Studying indexes impact on performance
- Example 3: ETL
-
Example 1: Load test of a database
- Conclusion
-
Load Testing Message Oriented Middleware (MOM) via JMS
- What we will learn
-
A bit of theory
- Asynchronous messages
-
Decoupling
- Technology decoupling
- Geographical Decoupling (Location transparency)
- Time decoupling
- Back pressure
- Communication models
-
Types of delivery semantics
- At Most Once
- At Least Once
- Exactly Once
- Summary
- What is a message composed of?
-
Set up with JMeter
- Installation of MOM libraries
-
JMS Point-to-Point
- JMS Resources and JNDI Properties
- Communication style
- Priority
- Use non-persistent delivery mode?
- JMS Selector
- Example with Apache ActiveMQ Classic
-
Publish/Subscribe
- JMS Publisher
- JMS Subscriber
- Methodology
-
Put into practice with JMeter
- Example 1: Test the configuration of a MOM server with point-to-point
- Example 2: Test the performance of our consumer with JMS Publish/Subscribe
- Example 3: Testing the configuration of a MOM server with Publish/Subscribe
- Example 4: Test any MOM
- Conclusion
-
Performing a load test
- Introduction
- Methodology
-
Before the execution of the load test
- Testing your script/dataset with the graphical interface
- Check the injectors
-
Be careful with the use Extractors to avoid injection problems
- Regular Expression Extractor
- XPath Extractor
- Extractor performances
- Be careful with the use Assertions to avoid injection problems
- Be careful with the use of custom scripting to avoid injection problems
- Generate reports at the end of the test
-
Execute load test
- Running your test from the command line
- Running your test on the command line with Taurus
- Running your test from Apache Maven
- Running your test from Jenkins
- Running your test from Apache Ant
- Conclusion
-
Visualising, analysing and reporting the load testing results
- Introduction
- What we will learn
-
Prepare our analyse
- Have data to analyse
- Have RAW data
- Having the right granularity on the data
- Have enough data to analyse
- Have the right definition of metrics
-
Visualising
-
Visualising the results with listeners
-
View Results Tree
- Sampler result tab
- Request tab
- Response data tab
- Summary Report
- Aggregate Report
- Backend Listener
- Report Dashboard
- Particularities of the visualization of results for a distributed load test before JMeter 5.0
-
View Results Tree
-
Visualizing the results with PaaS load testing tools
- Redline13
- Blazemeter
- Visualizing the results with third party visualizing tools
-
Visualising the results with listeners
-
Analyse the results
-
Methodology
- Example 1: High response time and Low CPU/memory/network/io disk usage
- Example 2: High response time and High CPU usage
-
Some tips to analyse the results
- Check response time distribution
- Check for spurious events (errors and bad response time)
- Worrying About Your Tail
- Be careful with response time at beginning of the load test
- Have separate statistics for failed and successful response times
-
Some examples
- Is the application support increasing number of clients?
- Is errors having an impact in response time?
- Is Garbage Collector (GC) is optimal?
- Is my application is CPU bound?
- Is my application is memory bound?
-
Methodology
-
Reporting
-
Prepare the reporting
- Who is the target of the report?
- What is the purpose of the report
- Don’t expose sensitive information
- Methodology
- Structure our load test report
-
Choose the right chart types
- Don’t use pie charts
- Prefer bar graphs when we have sparse metrics
- Present errors with toplist
- Timeseries graphs and single-value summaries
- Consider box plot for response time
-
Choose the right key metrics
- Define the number of metrics to display in a time series line graphs
- Same key metrics in report and in SLAs/load model
-
Choose the right data aggregation
- Prefer percentiles over average
- View aggregated data by phase
- Don’t average percentiles
- Define the resolution/granularity of the measures
-
Choose the correct display for the chart
- Don’t forget to add clear labels, legends and units in graphs
- Axis Forced 0
- Make sure data can be easily seen
- Add annotations for important events
- Add Thresholds in graph
-
Prepare the reporting
- Conclusion
-
Integration of JMeter in the DevOps tool chain
- Introduction
- Why
-
Organization/team topology
- Load testing team in its ivory tower
- DevOps and load test team collaboration
- One performance tester integrated on DevOps team
- No performance tester
- Team of evangelist’s performance testers
-
Questions to ask before automation and Shift-Left strategy
- Question 0: Which tool?
- Question 1: How many times do we need to create and maintain scripts?
- Question 2: How can we stabilize the test environment (injectors and tested system) to allow us to compare results?
- Question 3: How long should a test take?
- Question 4: When do we need to execute automated load test?
- Question 5: Do we have someone to analyze the load test results?
- Question 6: When is it acceptable to slow down one part of the application to speed up another?
- Question 7: How to detect slow performance degradation?
- Question 8: How to evaluate the effectiveness of load tests?
- Question 9: How to choose threshold alerting and avoid false positive?
- Question 10: Can we have a production-like environment for exclusive use?
- Question 11: Do we need to inject huge amounts of test data before the test?
- Question 12: Do we need to wait for someone before the test?
- Question 13: How to load test new features?
- Question 14: How to maintain a realistic workload?
- Question 15: How to handle dependencies on external systems?
- Question 16: How to reduce organisational resistance?
- Question 17: How to justify the cost?
-
Setting up Shift-Left strategy
- Modification in JMeter script to implement Shift-Left strategy
-
Integrating JMeter in our software factory
-
Integrating with Jenkins using Maven
-
Ensuring the load tests are run on the correct nodes
- Configuring the load test using jmeter-maven-plugin
- Fail the build if error rate of load test is over a threshold
- Configure HTML report
- Configure memory used by JMeter
-
Ensuring the load tests are run on the correct nodes
- Enhancing integration with Jenkins using Performance Plugin
- Integrate JMeter with APM monitoring tools
-
Integrating with Jenkins using Maven
-
Automation
- Example 1: Non-regression testing of memory consumption with EJ JProfiler
- Example 2: Detecting a Memory leak during an endurance test with EJ JProfiler
- Example 3: Retrieving SQL queries executed during a load test with YourKit Java Profiler
- Example 4: Analysis of the Garbage Collector (GC) log file with GCViewer
- Conclusion
-
Old Stuff
- Introduction
- Load testing web application Java EE 6 Pet Catalog
-
Load test a SOAP/XML-RPC web-service
- WS-* web services
- In JMeter
- REST Web-Service: Discussion Forum
- Load test a REST web service: Customer Database
-
Example 7: Non-regression testing of Web APIs with Dynatrace APM AppMon
- Registering to Dynatrace using setup Thread Group
- Using the testId in other Thread Group
-
What’s new
-
2024/11/29 release
- Chapter “Designing a test case”
- Chapter “Load Testing web services”
- Chapter “Preparing the test environment (injectors and tested system)”
- Chapter “Quick start with JMeter”
- Chapter “Performing a load test”
- Chapter “JMeter overview”
- Chapter “Why is it still important to make load test today?”
- Chapter “Load Testing Message Oriented Middleware (MOM) via JMS”
- Chapter “Visualising, analysing and reporting the load testing results”
- Load Testing a website
- Chapter “Old Stuff”
-
2020/04/28 release
- Chapter “Designing a test case”
- Chapter “Preparing the test environment (injectors and tested system)”
- Chapter “Why is it still important to make load test today?”
- New chapter “What’s new” at the end of the book
-
2024/11/29 release
Other books by these authors
The Leanpub 60 Day 100% Happiness Guarantee
Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
Now, this is technically risky for us, since you'll have the book or course files either way. But we're so confident in our products and services, and in our authors and readers, that we're happy to offer a full money back guarantee for everything we sell.
You can only find out how good something is by trying it, and because of our 100% money back guarantee there's literally no risk to do so!
So, there's no reason not to click the Add to Cart button, is there?
See full terms...
Earn $8 on a $10 Purchase, and $16 on a $20 Purchase
We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.
(Yes, some authors have already earned much more than that on Leanpub.)
In fact, authors have earnedover $14 millionwriting, publishing and selling on Leanpub.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
