Background
Her Majesty’s Courts and Tribunals Service (HMCTS) were directing the FullVideo Hearings (FVH) project to deliver a scalable service to meet the demands of the court service for a full video solution (i.e. where all participants are outside the court and connecting to a hearing via the internet) – all against the backdrop of Covid-19.
Performance testing the Video Hearings service was on the critical path for achieving demanding timescales.
The Video Hearings platform was already in production but subjected to very limited usage.
Limited manual performance testing by HMCTS had indicated that the performance of the application was poor. They had no automated way to determine the maximum concurrent figure before degradation of the service.
With the Covid-19 emergency, Video Hearings became an urgent focus and there was a desire to rapidly prove the performance capability of the platform.
Challenges
- Deliver a scalable service that could be handed over to HMCTS and meet the demands of the court service before the expected peak of Covid-19 infections.
- Client insisted on open-source test tools which could not fully support the requirements.
- The platform comprised of complex layers of technologies from multiple vendors.
- No direct access to all platforms requiring 3rd parties to collect vital performance data.
- Tests themselves needed to simulate both the granting access to a camera and microphone, as well as using that hardware during a video call. This was an issue as the tests were run on a virtual machine without any physical hardware.
- Scope changed throughout the performance test delivery phase as new issues were encountered.
Solution
- A two phased approach: Phase 1 -Extensive and detailed Discovery phase to fully understand the requirement, the technologies and the approach to performance testing followed by Phase 2 -Delivery phase where the test scripts, scenarios etc were created and executed and reported against.
- As timescales were very tight we increased the size of the team threefold to complete delivery within the six week timeframe. This also allowed us to react quickly to the many scope changes.
- Due to the limitations of open source tooling and the various complexities within the technology stack, we built scripts using multiple open source tools (JMeter, LoadRunner CommunityEdition and Selenium) in order to get full automation performance testing coverage.
- We built relationships with the various 3rd parties and included them in the performance testing activities, working with them to resolve issues that were raised.
Outcome
- We built a scalable performance test using open source tooling.
- We identified the performance characteristics of the platform.
- We identified performance issues and worked with a 3rd party to resolve these eg. database tuning.
- We handed over the whole test suite including documentation and test assets during an extensive knowledge transfer to the client test team.
- We enabled the client to take the performance testing forward in a repeatable and scalable manner thereby making an ROI – more testing with less resource.
This example demonstrates
- Testing Performance’s ability to tender competitively and deliver highly complex automated performance testing under very tight time pressures for a high profile project.
- How we work with third party suppliers to initiate and implement fixes for problems found during performance testing.