Why is an evidence-based practice so important?

The benefits of a user experience (UX) practice based on evidence are that it:

Allows you to identify a potentially successful methodology or design solution that has worked in the past, and may be applicable to your situation.
Saves you time, rather than making you start from scratch.
Increases your confidence in presenting and defending a method or design solution.

You can make your case and expect a higher chance of success.
Your recommendations are based on data, and not just on your opinion.
You are seen as a trustworthy and credible source.

Increases the chances of acceptance by management or a client who seeks support for decision making and asks:

Who says that (besides you or your organization)?
How do they know?
Does it still apply today and beyond?
Why should we do it or care?

Whenever possible, user experience (UX) practitioners should provide quantified, peer-reviewed design guidelines to support their design recommendations. According to usability.gov and others, many design guidelines lack this type of key information that is necessary in order to be most effective for organizations.

With that in mind, we’d like to propose seven steps for conducting solid evidence-based UX:

Clarify the question being asked regarding UX methods or design
Identify sources of research or best practice to help answer the question
Find available research or best practice
Review for credibility and applicability
Check to see if other research or practice has come to the same conclusions
Save copies of the materials along with links or citations for future reference
Communicate and apply what you have learned

Each of the seven steps is described in more detail below.

1. Clarify the Question

First, you should keep in mind that some UX questions are unanswerable or philosophical. An example of an unanswerable question is: “My end users are everyone. What web content works for them?”

In addition, some UX questions are so specific to a site or app that without your own research you will not find anything. For example, “Why is there such a high dropoff on step 2 in our online account setup process?”

Keeping those two caveats in mind, where should you look to try to answer the potentially answerable questions, such as, “Where do users typically expect to find the login functionality?”

On to Step 2!

2. Identify Sources

We recognize the role played by professional societies and associations that offer helpful publications online and offline, and often provide email alerts by subscription including:

American Psychological Association (APA)
Human Factors and Ergonomics Society (HFES)
Interaction Design Association
Society for Technical Communication (STC)
Special Interest Group Computer-Human Interaction (ACM - SIGCHI)
User Experience Professionals Association (UXPA, formerly UPA)

You should also seek out credible online UX research publishers such as UXMag and UXMatters.

Technology research companies such as Forrester and Gartner provide other resources that, depending on the question, may be useful as well.

Sources that we have recently found particularly helpful include:

Web sites and apps
Wikis and blogs
Research journals
Magazines
News sources
Books
Course materials
Conference proceedings
Peers or experts
You(your own or your organization’s research)

3. Find Research or Best Practice

In addition to publications, a general Web search will be revealing. Tips for search in general, such as on Google Search, as well as specific tips for Google Scholar, which focuses on research, include finding recent papers and locating the full text of an article.

4. Review

Next, you should evaluate the quality and applicability of a research or best practice article. Some considerations include:

Usefulness: Is the article relevant to the current research project?
Authority: Is the author an expert in this field? Where is the author employed? What else have they written?
Date of article: Is the article up-to-date, out-of-date, or timeless?
References: Scholarly works contain a reference section and / or bibliography of the resources that were consulted. The references in this list should be in sufficient quantity and be appropriate for the content.

One great way to characterize the articles you find is to use this rating scale from Research-Based Web Design and Usability Guides

Other tips for evaluating journal articles can be at Colorado State University's How to Evaluate Journal Articles.

Be sure to locate the original source of research or best practice if you can, since sometimes work may be unintentionally misrepresented in summaries by others, possibly in their attempts to be succinct.

Evidence based ux: identifying best practices and research

5. Check for Supporting or Related Research

Next, you should check for additional evidence as needed to answer a UX question, including:

Looking back in the original search results for related or later references to lend additional support
Trying related keywords or synonyms in a new search, such as “user experience” and “UX”
Looking for other articles in the same source, or by same author, and considering searching for the most distinctive author name to make it easier to find
Following up on the author’s university or organizational web site for related or more up-to-date information, such as a more recent blog post or article on the subject

6. Save Your References

Be sure to document articles, especially those that are found online, so that you and others can find them again, if needed (e.g., APA Style). Consider organizing and saving copies of key articles for ease of retrieval.

7. Communicate and Apply

Finally, apply what you have learned from the literature review, including the need to:

Be careful not to overgeneralize (Overgeneralization is the use of small and/or non-representative samples of real data to make an inference that is incorrect.) Your focus is on obtaining relevant insights. But don’t try to make the studies’ conclusions stretch too far beyond what the data will support.
Translate research results into practical language, implications, and recommendations
Use illustrative sketches or mockups in your presentations as a helpful way to convey the application of the research

Keep in mind that the lack of relevant studies may actually stimulate needed research into areas that will have an influence on the creation of designs for good UX. In other words, if you cannot find the research you need, then you may need to do the research yourself, or perhaps inspire a college or university to undertake the idea.

You should be aware that journals may be less likely to accept for publication the results of UX studies that generally confirm what is already known; the studies with more dramatic UX outcomes tend to be the more attractive choices for publication. So you may have to consider conducting your own research to supply the UX evidence you need.

Summary

For a solid foundation for your UX practice, you can:

Recognize the benefits of an evidence-based UX practice
Identify credible sources for research or best practice, and distinguish them from less credible sources
Interpret the questions being asked by a client or team member regarding UX methods or design
Identify sources of research or best practice to help answer the question
Evaluate sources and specific research for credibility and applicability
Generalize results as appropriate
Apply to design or methodology questions
Organize research results for future reference and collaboration with team members

Please share your ideas with us!