How Many Users Is Enough?


The short answer? Five.

Here's the longer answer.

In talking about user testing, there’s a question that comes up a lot. What can one realistically glean from a few (say, 5) individual usability tests? After all, a small sample set is not statistically significant, and there's no assurance that those individuals are representative. Some people say they would only be comfortable with a large sample set, which makes usability testing a very time-consuming activity.

While Nielsen et al. have shown empirically that in many ways the optimal number of users per test is 5 or so, we like to provide a more quotidian response as well.

Map

Imagine you visit a foreign country, say China or Mongolia or Egypt, and find yourself in conversation with one or two locals. Can you learn anything from these conversations? After all, who’s to say they are representative of prevailing attitudes?

Of course, the answer is yes, you can learn something, because exposure to a different perspective is always educational, and often contains many “aha” moments. This is particularly true politically, because we approach global political issues through a lens shaped by our media, political environment, and so forth, which is quite different from other countries’. That is, we have different mental models. Similarly, we design websites with our mental models, which are never exactly the same as that of the target audience. After all, we are experts both in the domain and in the technology.

Even a few narrated videos can be instructive in showing us how well our mental models and information architecture map to what is expected by our audience. It may not seem that scientifically “valid” to make judgments based upon a handful of data points, but what we’re doing is not “scientific,” it’s softer and more creative and deductive.

Learn more about our demographically filtered user pool:

Get started
 

Related reading: