User Testing: The Ultimate Guide

Meeting customers’ specific needs and solving their pain points are two goals virtually all companies share. People buy your products and services because they fulfill a need. User testing allows you to learn how your products perform.

Do your products meet customers’ needs and fulfill their expectations before putting them on the market? Learning this can save you time, money, and resources while helping you create an excellent product for your target audience.

In this guide, we’ll discuss the definition of user testing, methods you can use to test your products, and ways to implement user testing at your company.

Let’s get started.

Why is user testing valuable?

User testing is a simple way to gauge the point of view of your customers because it allows you to watch, hear, and review their interactions with your design.

While your employees, designers, and other people involved in creating your product, feature, or prototype might find it easy to use, your target customers may not. User testing allows you to see where your product falls short for your intended audience. It tells you where they get confused or frustrated.

Since user testing takes place before releasing your product, you can use this information to iterate on the product until you’ve created a user experience (UX) that you’re proud to stand behind.

You might be wondering how to perform user testing on your product. Well, every product, prototype, and feature is unique — as is every company — so you can employ various ways of user testing.

User Testing Methods

We’ve compiled this list of five commonly used user testing methods to help kickstart your product testing.

1. Usability Testing

Usability testing is when you provide a real customer or member of your target audience with a product, prototype, or feature so they can assess and test its usability. Usability refers to a product’s ease of operation, usefulness, and utility. Usability testing allows you to determine how intuitive your product, prototype, or feature is in the eyes of your actual customers.

You might think usability testing sounds like the definition of user testing we covered earlier, but it’s not. . Some people use the terms interchangeably. However, for the sake of this piece, we’re going to differentiate them and continue with the notion that usability testing is a method of user testing. Here’s why:

User Testing vs. Usability Testing

User testing covers the entire range of user experiences a customer has with your product, prototype, or feature. This includes all their perceptions, emotions, preferences, responses, and behaviors in response to that item, from when they obtain it to when they stop using it

Usability testing focuses on how and to what extent a customer uses your product to accomplish a specific goal. This plays a part in the user experience but is not the entire experience.

Here’s a visual to help differentiate between user testing and usability testing.

Image Source

If usability testing sounds interesting to you and your team, consider another factor: whether you’ll run a moderated or unmoderated usability test.

Moderated vs. Unmoderated Usability Testing

Moderated usability testing involves one of your employees sitting down with a real user, explaining what they want that user to do, and then listening to their feedback during and/or after their product use.

Unmoderated usability testing doesn’t involve any communication or interaction with the user. Instead, you’ll likely have a video recording of the user working with your product, prototype, or feature and then another video with a review of their interactions and their assessment of the item.

Moderated usability testing is great if you want to watch customers use the item in real-time to ensure you’re getting the information you need and want to obtain. On the other hand, unmoderated usability testing is a good option if you want to test a large group of real customers from several different areas simultaneously.

When to Use Usability Testing

Usability testing is ideal during the early to mid-design phase of a product, prototype, or feature’s life. That’s because you can ensure each aspect of your design works appropriately for your customers during development and before investing more time in its construction.

2. Surveys

Surveys are a great form of user testing if you want to obtain accurate, quantitative data from your customers about your product. For example, you might ask them to complete a survey about your new product or your latest update to an existing product. And since you create and ask the questions, you can be as broad or granular as you want.

Additionally, surveys are a simple way to obtain feedback because your participants can complete and submit them from anywhere, via any device. They’re also easy to send out to large groups of participants and pull data from once submitted.

When to Use Survey Testing

Surveys are ideal if you want to receive a wide range of responses regarding your product in a short amount of time. Surveys also make it easy to dive deep into details about specific features because you can ask your test subjects tailored questions of your choice. They can also help you quickly obtain, organize, and analyze quantitative and qualitative results to improve your product for your customers.

3. A/B Testing

A/B testing occurs when you split your test subjects into groups and test different product versions to determine their preferences. This way, you can understand which version best meets your customers’ needs and provides them with a better user experience.

A/B testing allows you to share different product versions with your customers and learn what works for them rather than guessing based on your buyer personas or information about your target audience. Just because you may think you know your customers well doesn’t mean your test subjects won’t surprise you in an A/B user test scenario.

When to Use A/B Testing

A/B testing is ideal if you want to get feedback about which version of a product is most popular among your target audience. For example, you might conduct an A/B test for a CTA button on one of your landing pages with slightly different wording. This way, you can learn which option gets you more clicks and conversions.

4. Focus Groups

Focus groups are where you sit down with a small group of real users (usually 10-12 people total) to discuss various attributes of your product, prototype, or feature. The group discussion is generally led by one of your designers, researchers, or other employees to ensure you’re receiving your desired type of feedback. A focus group discussion will typically last an hour or two and cover the test subjects’ concerns about a specific aspect of a product that the focus group facilitator brings up.

For example, suppose your company is testing a part of your new or updated software. In that case, you might have one of your web designers or engineers facilitate the discussion to ensure your test subjects provide you with details about their interactions with that specific aspect of the product. In addition, your focus group leader can ask questions about the updated portion of the software with your participants. Again, this cuts out all unnecessary fluff and makes for a straightforward and impactful discussion.

When to Use Focus Group Testing

Focus groups are a flexible form of user testing because you can discuss any point in your product lifecycle. They’re ideal for understanding what customers want and hope to get out of your product through a guided discussion.

For example, suppose you hold a focus group discussion. It’s recommended that you hold multiple focus groups, so you only chat with a small group of customers during a focus group discussion.

5. Beta Testing

Beta testing happens during the last stages of your product, prototype, or feature design. It’s when you provide your customers and target audience with the item you’ve created in a state that closely resembles what it’ll look like upon distribution. Beta testing is a great way to get a final thumbs up on your product from customers before going to market.

When to Use Beta Testing

Beta testing is ideal if you’ve pretty much finalized your product’s design and want to conduct one last review of it before distribution. This way, you can make any last-minute changes to enhance its UX if necessary and ensure it’s ready for your customers.

Before a beta test, other user tests should occur throughout your design process. That’s because this is only used for final and critical updates to the product — so you’ll want to get your customers’ opinions before the final stage of design. This way, once you reach the time for beta testing, you’ll likely only have minor changes that need to be made (if any).

Now that you understand the various user testing methods, let’s talk about how it works so you can begin testing your products, prototypes, and features.

1. Create a product, prototype, or feature.

The first step in user testing is pretty obvious — you’ll need a product, prototype, or feature to test. You may try a brand new product, a new product version, a sample, or a part you’re considering adding to an existing product.

2. Decide on a method of testing.

Once you’ve designed your product, prototype, or feature, it’s time to determine which user testing method you want to implement. To do this, consider which test method best suits the type of information you want and how quickly you’re looking to receive that feedback. This way, you can get the desired results that make sense for your company and the product being tested.

Once you’ve decided on your testing method, you can dive a little deeper. Consider which factors you’ll be looking at and testing for so you can ensure you have the type of feedback you want and need from customers. Here are some examples of things you can look for and incorporate during testing, no matter the method you’ve chosen:

Problem Discovery

Problem discovery is a process in which your customers and test subjects surface various issues regarding the usability of a product, prototype, or feature. You may have them do this while they complete a given set of tasks with the item. Additionally, you might use the system usability scale, a click test, or a heuristic evaluation to get more profound answers and results related to the problem customers have discovered.

Learnability

Learnability is when you observe how difficult it is for a test subject to complete a task with or within the given product. You can watch test subjects work through a specific task as often as they need to complete it. If the learning curve is shallow, they may only need to work through a task once. However, if the product has a more significant learning curve, they might need to do this in multiple attempts.

Competitive

Comparing two similar products, prototypes, or features can help you determine whether or not you’re building a better version of a product for your customers. The products being compared can be created by your company, or they can compare one of your company’s products to a similar one made by a competitor.

For example, you wouldn’t have test subjects compare new, high-tech running shoes to photo editing software — this wouldn’t be helpful for anyone involved. Instead, you’d have testers compare your photo editing software to the older version you initially sold them or that of a competitor, like Adobe Lightroom. This allows you to get honest feedback and context around enhancing your software’s user experience and improving its capabilities.

Benchmark

You can have a group of your test subjects provide you with a benchmark to build off of and improve your product from. Subjects will use your product, prototype, or feature and advise you on how to enhance it. Once you make those modifications, you have the subjects re-test to see how well you improved the item.

Eye Tracking

When you use video to watch a test subject’s eye movements throughout their use of your product, you’re using the testing technique of eye tracking. This is common for businesses testing software because you can easily see where that person’s line of sight goes on a screen.

3. Recruit people to complete the test.

Next, you’ll need to find and recruit people to complete your user test. No matter your chosen testing method, user testing calls for real customers, real members of your target audience, and individuals who fit your buyer personas and empathy maps to complete the test.

This allows you to get accurate data and results from the people who matter most — the ones who will be buying, using, and promoting your products. (We’ll go over how and where to recruit test subjects if you don’t already have a group.)

Learn about the power of using buyer personas and how to create one for your business.

4. Determine a location to conduct the test.

Once you have your product, method of test, and test subjects, you’ll need to determine the time and location of your test. Common areas for user tests to be conducted include the following:

• Community space
• Your office
• Outdoors
• Test subjects’ homes

This decision depends on your product, prototype, or feature, what it’s used for, and what its capabilities are. When choosing your test location, be sure to consider all of your subjects’ surroundings and any possible distractions they might encounter.

For example, if Airbnb was having users test their new, online vacation rental sign-up process, they might have their test subjects do this from the comfort of their own homes. This is most likely where their customers would be working through these forms in real life. Also, with the ability to watch users work through the form via webcam and a screen sharing software like Zoom, companies don’t need to ask test subjects to come into a community space, sit down with their computers, and work through the vacation rental sign-up process.

However, if Nike were conducting a user test for the newest version of their Metcon weightlifting shoes, they may conduct the test in a gym with equipment subjects can use to lift, squat, and box jump.

5. Decide how long the test will take to complete.

Have you ever been asked to complete a task you know fairly little about? Of course, one of the first questions you might have is, “How long is this going to take?”

That’s most likely going to be something your test subjects are wondering about your test, too. After all, they’re taking time out of their day to complete tasks you’re asking them to work through.

Different user tests will take different amounts of time. Try to stick around 30-60 minutes per test, per participant if possible. Longer tests seem to cause a drop-off in the quality of responses and feedback received from test subjects due to fatigue, boredom, and lack of interest.

If your test has multiple parts or requires participants to go back in and work through a test more than once, be sure you allow for sufficient time to do this — whether that means conducting a few short tests repeatedly or allowing your subjects to take breaks between tests. Either way, just be sure to communicate with your test subjects so they feel informed and cared for. Remember, these are members of your real target audience, so you want to leave a great impression on them.

6. Distribute the test.

Now it’s time to distribute your test. First, be sure your subjects have all the test instructions they need, including directions on how they are to provide you with their feedback and analysis.

Depending on your product, company, industry, and test method, the various directions and information you’ll communicate to your test subjects before and/or during the test will differ. However, let’s review some points that companies commonly examine before and upon distribution of the test:

• Make sure they know what you’re trying to achieve with the user test so they can focus on specific aspects of your product and/ or can understand why you’re testing the product.
• Ensure they know how long the test will most likely take to complete, so they’re aware of the schedule and what’s expected of them.
• Ask if they have any last-minute questions before taking the test.
• Make sure they have all the necessary equipment. For example, does their laptop have a webcam, and is screen share software downloaded?
• Ask if the participants need any unnecessary distractions removed from the area.
• Ensure the participants understand how you expect them to provide feedback and analysis after the test (verbal, written, or through a form).

7. Analyze the test results.

The final step is the most important: analyze your results. These results are what you conducted your user testing experiment for in the first place — to obtain critical feedback from real customers to improve your product and enhance its UX.

Consider everything your test subjects have to say, look at what the majority of them have concluded, and don’t be afraid to go back to the drawing board with minor or drastic modifications, if necessary.

Here are some standard user testing metrics that are reviewed upon the completion of a user test:

User Testing Metrics

Task Completion

Task completion is when a test subject indicates they’ve completed the test or its end goal.

Critical Errors

Critical errors prevent your test subjects from having the ability to complete your test.

Non-Critical Errors

Non-critical errors are made by the test subject yet don’t impact their overall ability to complete the test successfully. These errors typically mean a part of your product, prototype, or feature caused the test subject to be less efficient at a point in time.

Error-Free Rate

This is the percentage of test participants who can complete the test without errors, whether or not they made errors along the way.

Time On Task

Time on task is how long it takes your test subject to complete your test.

Subjective Measures

These are the opinions of your test subjects, such as their likes and dislikes, recommendations, personal feeling of satisfaction upon completion of the test, and simply their level of enjoyment while using your product.

Once you have your results and have associated them with the metrics listed above, you can analyze them in depth, look for patterns, and find trends among your participants.

You should also consider reporting the severity of the problems that surfaced throughout your tests. This will help you determine how imperative it is to rectify specific issues and decide how to prioritize them(if there are multiple).

Here’s a scale to help you determine this:

From here, you can compile your findings and recommendations and turn them into actionable takeaways to help you improve your product. Finally, you should organize these findings to share them with your team members, designers, and anyone involved in the product’s creation and development.

Some information to include in your list include:

• The number of people who completed the test successfully.
• The average amount of time it took test subjects to complete the test.
• Helpful participant comments, opinions, and recommendations.
• Specific details about the average point in time when most test subjects ran into issues.
• Positive things about your product — although user testing is used to identify issues with a product, it’s essential to highlight the great things about it, what your test subjects enjoyed, and what they liked about it.

How to Recruit Users to Test Your Products

How do you recruit test subjects representing your buyer personas and target audience to complete your user testing? Here are three main ways to do this:

1. Recruit existing customers.

A great way to recruit for user testing is by searching your current customer database for contacts willing to help. You can also check your email subscription list or look for customers who’ve left you reviews and ratings on sites like Google and Facebook.

2. Recruit through your network.

You can also recruit test subjects through social platforms like Twitter and Instagram and ask for referrals from your fellow employees and people within your professional network.

3. Recruit through user testing services.

If you’re working on a large-scale project and your business has the resources, you might sign up for a user testing service. These third-party sites connect your business with suitable test subjects for your specific situation. Here are a few popular user testing service sites to consider:

• Usertesting.com connects you with real customers, allows you to watch and hear them interact with your products in real-time, get their direct feedback, and share your results with your team.
• UXtweak.com allows you to conduct user testing using several methods and then efficiently recruit users straight within the platform. You can target specific user personas. The panel is global with strong coverage of countries.
• Validately.com offers recruiting services to help you find your ideal test subjects. It includes a dedicated testing platform to ensure obtaining results is simple and has automated reporting so you can analyze, manage, and organize your feedback.
• Verifyapp.com allows you to send out tests to gain real-time insight into your customer reviews to achieve actionable results.

Begin User Testing

User testing is a powerful tactic for all businesses because it provides you with real, actionable data that comes directly from your customers and members of your target audience. In addition, it allows you to ensure the design, UX, and all other aspects of your product, prototype, or feature, are ideal for customers before selling it.

So, prepare your new product, determine which testing method works for your specific situation, and recruit real customers to help you improve the UX of your offerings.

Editor's note: This post was originally published in April 2019 and has been updated for comprehensiveness.