Adding user tests to your agile process

In today’s competitive product landscape, we all want to get our products to market as fast as possible while staying ahead of the competition and preventing customer churn. Speed matters. And it’s why so many organizations have embraced agile processes.

The agile methodology has its benefits. But when performance is the priority, customer feedback can become deprioritized. While many teams may think it’s a corner that can be cut in the name of efficiency, doing so actually exposes organizations to considerable risk. If we release products that aren’t what our users want or need, we’ll waste far more resources than it would have taken to get the proper user validation early on. 

Many teams have released products, thinking they had a winner, only to find their customers weren’t interested after the launch. The financial loss associated with the sunk costs of development can be massive. Add that to the loss of trust a failed launch can have on customers and the impact is even more severe.

Thankfully, it’s possible for teams to take advantage of both the speed of the agile methodology and the confidence that comes with proper customer research. Organizations can reduce risk through user insights and feedback, showing how customers experience their product across touchpoints and channels. 

In this guide, we share how to conduct user testing when you’ve embraced an agile process for product development. 

We cover:

• What is an agile process?
• How user tests can help
• Why some teams get stuck
• How to conduct user tests in agile

What is an agile process?

An agile process is one that allows product, design, and development teams to rapidly respond to change. It’s a way of being adaptive and moving quickly based on user feedback. Unlike the linear and sequential waterfall methodologies of the past, the agile methodology emphasizes collaboration, customer feedback, and small, rapid releases. Teams typically work in 1-4 week sprints in which they plan, develop, test, and review a set of features. 

Using an agile process has become the go-to development methodology for reducing risk when shipping new products and features. But even though agile helps many organizations make huge workflow improvements, agile teams aren’t perfect. They can still create products that don’t meet user needs.

The way in which product, design, and development teams spend their time is extremely valuable. In a perfect world, developers would spend 100% of their time building new products and features. In reality, an estimated 50% of engineering time is spent doing rework that could have been avoided. Plus, fixing an error after development is up to 100 times as expensive as it would have been before.

So how might these mistakes be avoided? How do you reduce uncertainty and prevent the overall failure of the products and features your team ships? The answer is with user tests.

Why user tests are important in agile

A user test allows you to see how your product or feature will do in the wild. By trying it out on real human users, you’ll see where they succeed and where they get stuck. This information can help you make adjustments to ensure your product is as polished and valuable as possible.

These tests should be embedded throughout the agile process, not just something you do once the next iteration is complete. If you’re waiting to get user feedback until your developers have built the product, then it will be more challenging to fix any issues that you find. 

Challenges and solutions for user testing in agile

Teams can run into a number of issues when they try to embed user testing into their processes. We’ve outlined some of the most common challenges with potential solutions.

Challenge: Your website or product isn’t ready for a user test

Teams often share that their website or product isn’t good enough to be tested. They mistakenly believe that users can only give feedback on high fidelity prototypes or working websites. But you don’t need to present users with finished assets to gather useful feedback.

Solution: You can start getting actionable insights on very early designs—even hand drawn sketches, wire-frames, flat visuals, scamps, and prototypes. This type of early stage design research is known as directional testing, and can often be as simple as checking where people would click on a series of flat images, or running a think-out-loud study for the complete user journey. By carefully setting a scenario for users and explaining they’ll be using a low fidelity prototype, you can ensure feedback is useful.

Challenge: Not enough time to find participants and run user tests

You can’t wait to find and recruit participants if you want to be fast. But finding the right people is time-consuming. Conducting the tests can also use up a lot of time, as well.

Solution: With traditional sourcing approaches it can be a struggle to recruit enough participants within a sprint to conduct testing. This causes teams to put testing off further into the future when the product is in a later stage. By partnering with a solution like UserTesting, you can find users within minutes. 

Challenge: You’re not a researcher

Some team members don’t feel they’re equipped to do research. They might be designers, developers, or product managers as a primary discipline. 

Solution: Anyone can conduct research. In these cases, partner with an internal research team or a solution like UserTesting that can provide resources and templates to help you easily run tests yourself. 

Challenge: Worried about missing things

It’s tempting to try to capture everything with user testing and sometimes that’s not realistic. It’s natural to worry about missing something in the process.

Solution: Seek to answer only the questions that are most impactful, and consider a research backlog. Working in a week or two-week sprint should focus your mind on answering only the questions that will lead to the biggest impact. Focus on the outcomes you enable by answering fewer questions faster. For the questions that are simply too large for an agile approach, take a different approach. In contrast to directional testing, you can carry out foundational research on a separate track over weeks and months. This ensures the big questions are answered and given the time they deserve so you reach the right conclusions. Directional testing within agile ensures you are impacting your business every single week.

How to implement user testing in agile

Collaborate with your team

1. Get stakeholder buy-in and support

Engage your Product Owner, Designer, and Researcher to secure stakeholder support. If they’re not yet on board, highlight the risks of omitting UX research during design. Share client success stories to illustrate the benefits.

2. Run an enablement workshop

Conduct a workshop with your Agile teams and stakeholders to plan and structure research within sprints.

3. Decide on a research cadence

Determine whether a 7-day, 8-day, or bi-weekly sprint works best. UserTesting can assist in finding the optimal schedule.

Address Cadence Challenges

1. Agree on participant profiles

Pre-approve participant screeners for the year to save time and ensure consistent research.

2. Assign research responsibilities

Evaluate your internal UX research resources. If needed, UserTesting's Professional Services can provide on-demand researchers.

3. Start a research backlog

Establish a backlog for tactical research and larger studies, setting expectations for the types of research to be conducted during sprints.

4. Commit to the test cycle

Set target days for research deliverables and secure buy-in. Emphasize that testing can proceed with low-fidelity prototypes or flat images.

User tests in agile: How to test with parallel design and development sprints

Many leading organizations find that the most effective way to incorporate user tests into their agile development cycle is to have their UI/UX designers, user researchers, or product managers run a design sprint in parallel with their development team.

The purpose of the design sprint is to:

• Identify design problems through user research
• Validate design solutions through iterative testing
• Prepare user stories for the product owner to prioritize in time for the planning meeting (a.k.a. backlog review)

While your engineers are writing code for their current iteration, your design team can be identifying the problems and validating the solutions for your developers to work on in their next iteration. Some people refer to this parallel process as “iteration zero” because the design team is always one step ahead of the development team.

User tests in an agile design sprint

1. Conduct user research

On day one, the product owner, UI/UX designer, or user researcher digs into metrics from the live product's existing version to identify the problem areas. Then they run 3-5 user tests focused on those areas to identify what’s causing users to get stuck or confused. If they launch their user tests in the morning, they can have results by the time they return from lunch.

If you’re developing an entirely new product, use this stage to do exploratory research. Run user tests on your competitors’ products, or have your target audience show you how they currently handle the problem your product will solve.

2. Meet to discuss problems

On day 2, the product owner, Scrum Master, and UI/UX designer meet to discuss the problems that were identified through user research. Participants in this meeting should come prepared with metric data and video clips from user tests to use as evidence and guide the discussion. Once your team has defined the problems, start brainstorming potential solutions. The goal is to leave this meeting with a concept for your UI/UX designer to run with.

3. Test mockups and prototypes

Once a solution has been identified, your designer starts mocking up a solution. When they have something they can put in front of users to get feedback, they’ll run a user test and iterate on their design based on the feedback. They continue to repeat this process—moving from wireframes to low-fidelity prototypes and finally to high-fidelity prototypes—until they validate that the solution solves the problem as intended.

4. Prepare user stories

Once a design solution has been validated, the product owner and Scrum Master meet to get the user stories into the backlog and prioritized. Each card should include relevant design assets and video clips to support prioritization decisions.

Now that the user stories have been prepared, the product owner and engineering team will kick off the development sprint with an iteration planning meeting. At this point, the design team will repeat the same process to prepare validated solutions for the next development sprint.

User tests in an agile development sprint

1. Run an iteration planning meeting (i.e. backlog review)

On day 1 of the development sprint, the product owner and development team meet for the iteration planning meeting. The product owner presents the prioritized backlog, the engineers choose which user stories they’ll work on during the sprint, and those stories get moved into the sprint backlog.

2. Conduct a tactical meeting

Next, the product owner brings the development team together for another meeting where they look at the cards in the sprint backlog, break them down into the specific tasks they will do over the next iteration, and discuss tactical details.

3. Build software

Now that everything has been planned out, the development team starts building software for the length of the iteration cycle.

4. Validate solutions

Once a solution has been built, the engineer or product owner can run a user test to validate that it’s solving the problem as intended. Use the same test script as you did for the user test initially used to identify the problem. This will give you a benchmark to judge whether the solution has fixed the issue. It may also uncover new issues, which can be funneled into your product backlog for future iterations.

5. Complete an iteration review

Finally, hold an iteration review where the team demonstrates the software they built during the sprint. Share video clips from your user tests demonstrating that the solution you built has been validated and solves the problem.

Making user testing a part of your process

At this point, the design team will have gone through another sprint and prepped the next round of user stories. Both teams repeat their iteration cycles in parallel, with the design sprint always one step ahead, preparing validated solutions for the development team to deploy.

One of the best ways to get the insights you'll need to identify the right problems and validate solutions is to select the right tools and resources. Choose a solution that will help you rapidly understand how users interact with your product—including where they get confused, frustrated, or fail to make sense of your iterations.

Combining user testing with an agile process is the best way to deliver valuable products quickly. Rather than forgo user tests in the interest of speed, embed small and frequent testing into your process so you can confidently create products that users love.