Agile Performance Testing Methodology

In the era of hyper-connected world, everyone wants high speed & perfection.  Agile methodology came into existence to solve few problems that exist in traditional software development methodologies & to increase agility in Software Development Life Cycle (SDLC) activities. Having realized the impact on the business, where thoughts about system performance came reactively at the end of functional testing, many organizations have started with agile performance testing methodology. But it is a big challenge to start assessing the system performance early in SDLC phase when the system is partially ready. This article discusses about how to bring performance testing into agile world and how to address some of key challenges to get good results.

The goal is to test the performance of the application functionality as soon as it is built and when it is ready for functional testing, so that both functional and Performance testing can be conducted in the same sprint. Benefits of early testing are tremendous as cost of poor quality (COPQ) is significantly brought down.

Performance Testing in Agile

Below diagram depicts performance testing methodology for Agile (Scrum methodology is considered). We can divide the whole performance testing efforts into three stages S1, S2 and S3, these stages are tagged to each stages of Agile scrum.

 

Agile Performance Testing Methodology

Stage 1. Unit Performance Testing

This stage can be started as soon as the development activity in the sprint is started. This stage referred as Unit Performance testing which concentrates on method or code level testing.

  • Understand the technology and functionally that is being developed (do not disturb)
  • Recommend on best performance practices of technology or high performance design patterns or server configurations (but do not insist).
  • Prepare unit level Performance test stories for required functionalities.
  • Once coding is completed, measure method level response time and resource usage requirements
  • Method level performance analysis can be conducted using profiling tools.
  • Use Test or Dev Environment for Performance Testing, since high load simulation is not required at this stage.
  • Try using open source tools/profilers for performance testing (keeping an eye on cost)
  • Measure and baseline the transaction response time results.

Key Metrics to be reported :

  • Code Profiling metrics
  • Transaction Response time (end user and method level)

Stage 2. Focused/Component Performance Testing

This is not a like system performance testing where load tests are conducted at production workload. Concentrate on a specific component (feature or functionality) level performance tests. Instead of waiting for complete code freezing of the application herein we can test the functionality that is recently built and design small load tests to test it for a low load scenario.

  • Load test specific functionality
  • Run repeated tests after each sprint
  • Measure the Response Time for a low & medium load conditions
  • Can use low end test environments to conduct the tests.
  • Do not waste time in enhancing scripts but concentrate on how to load test the functionality effectively & report performance issues quickly.

Key Metrics to be reported :

  • Transaction Response time of the functionality under test
  • System resource usages levels under the load conditions
  • Throughput and other performance metrics under load

Stage 3. Integration / System Performance  Testing

This is the phase where system level performance testing activities are carried out. These activities include planning for different types of tests including load, stress, endurance tests, etc for whole application. It is advisable to carryout Stage 3 activities during hardening sprint. Conduct performance testing for focusing on whole application.

  • Prepare a realistic workload model for the system under test
  • Setup performance monitoring at each of the tiers
  • Design and execute planned tests to discover bottlenecks at system level
  • It is advisable to carryout system performance tests on environment with production similar capacity or atleast where there is a 1 viable unit of production configuration for of the system tiers.

Key Metrics to be reported :

  • System performance, scalability & capacity characteristics including response time, system throughput in terms of peak users supported & peak transactions supported per unit time & server resource utilization levels at all tiers.

Advantages of Agile Performance Testing

  • Involving Performance testers during early sprints will help to build the system with high performance.
  • Early bottleneck detection & fixing with a huge savings on COPQ.
  • Avoid bringing up major performance issues during hardening sprint that will have great impact on system architecture & design.

About the Author 

 Irshad Ahmad Pallamajal is a passionate performance engineer with about 5 years of experience in carrying out end-to-end Performance Testing activities.  He has good experience in performance bottleneck analysis activities. He possess very good analytical & communication skills and he is involved in training  activities.

 

 

 

Recommended Posts
Comments
  • Pavan Gurram
    Reply

    Hi Irshad,

    Nice article. It would be great if you can explain this theory with some example (piece of work where you implemented this approach) so that it’s ease to understand your theory. Looking forward for your response. Thanks.

Leave a Comment