Implementing Performance Testing in a DevOps Pipeline
Successfully shifting performance testing and analysis activities left in the development cycle
Establish performance testing activities in an existing DevOps pipeline
A global manufacturing company had transitioned to DevOps, but was struggling to include performance testing practices in their pipeline. The IT organization had an established Performance Testing Center of Excellence that had thrived in the waterfall process. However, the legacy approach to performance testing, which demanded long lead times and was focused only on end of cycle execution, did not integrate well with their new agile processes.
Utopia introduced in-sprint performance testing and analysis activities with a focus on making performance part of everyone’s responsibility. Specific changes included:
- Creating performance based stories
- Unit and component level performance tests as part of continuous integration
- Tracking trends in performance-related system telemetry throughout the pipeline
- Utilized service virtualization to remove system dependencies in upstream environments
- Using performance engineers from the CoE to mentor the development teams on key performance related practices and tools.
The client’s performance testing CoE required long lead times and code freezes during test execution – all of which were incompatible with their DevOps approach.
Performance testing and analysis related knowledge had been isolated in the CoE for years leaving the development teams with inadequate performance expertise.
Production Performance Issues Reduced by 85%
The team was able to proactively identify performance issues when they were introduced versus waiting until the end of a release cycle. Performance related production issues were reduced by 85%
Performance related knowledge was disseminated to the development teams enabling performance to become part of everyone’s responsibility.
Performance Issues Detected and Addressed Proactively
The client was able to track performance trends at the unit, component and system level from build to build. Any changes that negatively impacted performance were detected in the sprint they were introduced. The result was an 85% reduction in customer reported performance issues over the first 12 months of implementing in-sprint performance testing and analysis.
The development teams developed performance related skills and were able to achieve the goal of making performance part of everyone’s responsibility.