Monitor and improve
Plan to continuously improve
Continuous improvement means more than doing basic maintenance to fix bugs or errors. It's about responding to changes in user needs, technology and policy. Your service should remain relevant and fit for purpose. You should be proactive and retire your service intentionally rather than neglecting it.
Content, information and technology can change at a fast pace. So can business and user needs and how you measure success.
Once your product or service goes live, set clear review dates to assess, address and report on:
- how it's meeting your success measures
- if usability testing results show it still meet's user needs
- any changes in user needs
- any changes in policy
- performance issues.
Project scenarios: Plan to improve
Let's explore two project scenarios of an approach to planning, where a team’s service design goal is to increase the user’s uptake of accreditation.
A project team aimed to boost industry accreditations.
User research revealed higher accreditation levels were too complex. Lacking time and budget, they focused on the first level.
The project launched on time and within budget. Accreditation numbers peaked two months post-launch but dropped sharply, with minimal activity by six months.
Example scenarios
In the two scenarios below, the same team takes a different approach to their service design. Their approaches impact the value and outcomes of the project.
In example 2, notice how the team uses research and metrics to make decisions about the service and ensure its success.
Two sprints after launch, Scenario 1's team decided to release its resources to work on another project. This was because they had achieved their success measure, to be on time and on budget.
Shortly after, the number of completed accreditations tapered off sharply. The team hadn't planned or budgeted for further research after go-live. So, there was not enough resources to understand why the approach failed or how they could improve it.
The sponsors decided that the effort to restart the project was not worth the investment. The service continued to run but its success was limited.
Scenario 2's team did research with their users throughout the design and build phases. They wanted to understand how these users might engage with the service post-launch. They found that the users were a tight-knit community. It became clear that continued user uptake would rely on getting the community motivated to apply for accreditation through word of mouth.
For their post-launch sprints, they did user research to observe user behaviour. They found that users liked the format and idea of the service but became frustrated with its limits. This resulted in users recommending their community against using the service.
The team presented these insights and metrics to their sponsors. The sponsors gave the go-ahead for the full team to expand the service offering. This led to the service exceeding the previous performance benchmarks and becoming a self-sustaining success.
Allocate team resources
Know what resources you will need to make improvements. It's critical that you plan for resources and budget early on. You will need to maintain, iterate and improve the service throughout its lifetime.
If there's no trigger for a new iteration immediately after launch, you should scale back your resources.
Allocate budget and people to ensure you keep your service running and fit for purpose. You will usually need less resources to maintain your service, but a full team for significant improvements. For example, when you address a user pain point that you uncover post launch, which would result in changes to the service.
Make sure a team or person has oversight of the service to ensure:
- it is secure and stable
- privacy and security safeguards
- it's continually monitored.
Under-resourced teams will result in poor outcomes (in cost or quality) for improvement work. You should use the same design and build team for the iteration process. Otherwise, consider who you'll need to make the improvements.
Trigger points to monitor
Set up an ongoing monitoring process during the life of your product or service. Identify triggers that will let you know when something has failed or is no longer for purpose.
This will help you know when to act to make improvements and fix any issues with your service.
Trigger points can include:
- feedback from your users gathered through usability testing, surveys, reviews and so on
- a set amount of time
- a predefined increase or decrease in your metrics of success
- a drop in technical performance
- security patches.
Use evidence to improve
You will have set metrics to track the success of your service and established a baseline.
You can use tools like web and data analytics to track performance. They will show you how it's improved. Consider things like:
- rate of escalation in complaints
- response speed
- conversion rates.
Capture and act on feedback from your users to assess how well the service is meeting their needs. Look at customer comments on feedback pages, app store reviews, usability testing and user research after go-live.
Use the data and information you gather to support any decisions to improve or make changes Don't forget to test any new features or changes you make.
Keep up to date with changes
Keep up to date with any technology, policy or legal changes in your sector. This will ensure you provide customers with accurate, timely and relevant services.
Also assess whether your product or service could provide a simpler or faster way of doing things. If you don't, you run the risk of it being outdated and unusable.
Test your service
Test performance at regular intervals after go-live. This includes basic maintenance. Testing helps to identify improvements you need to make.
Follow your agency's standard templates and artefacts for testing. Also see Gov.UK and the 18F testing cookbook for guidance.
Maintain your service
With basic maintenance activities you will help to keep the service running as intended. Do things like:
- check that content is still accurate, current and relevant. If not, either update or archive it
- implement security patches when they become available
- practice vulnerability, penetration and security testing
- performance test to maintain speed, responsiveness and availability of your service.
Document and share improvements
Document any changes you make to the product or service. Tell your users, stakeholders and leaders about it. Send an Electronic Direct Mail (EDM), blog about it, or hold regular presentations.
Add any future improvement opportunities to your backlog.