“It’s not rocket science”…user testing

Had an interesting lecture about user testing. This is something in my field of interest since it’s describing a phase in the design process that is more about the practical way of conducting things, and in this case how to test a design/prototype with users. The outcome of the activity is to analyze the data and implement the result in the design. The core question of the lecture was “when, why and how?” we do user testing. There were also a brief mention of data and how to use it the right way..

I recall touching on the subject of user testing in the course methods 1, were I critiqued the way we worked as a group in order to solve our usability issues. Basically to shorten the story we hadn’t got the opportunity to do things the right way. One method to do it (and which is mathematically calculated) is to test the design with 5 users during 3 different stages. Jakob Nielsen goes through the essentials in the article “Why You Only Need to Test with 5 Users”, which is an very interesting read and highly related to the field of user testing. Using that way will come as close it can to achieving 100% coverage, although there is always something to uncover as no design is perfect, this method provides the best cost-to-coverage ratio. As we are in an educational environment and no actual money is being spent on user testing, rather time, there will be no issues gathering more participants for the testing if it’s needed.

I’m curios to see if the formula is accurate enough were it’s stated that 85% of the problems will be uncovered in the first stage of user testing. The second testing will uncover 15% and the third one 2%. During each stage and refinement of the design there is bound to be a new introduction of some errors in the design by the designers but overall it’s an interesting claim which I will keep in mind when move to the testing phase in project 2.

Nielsen doesn’t specify which methods is used, maybe it is so that depending on which method is used and which setting the method is being used in will affect the success rate and possible errors uncovered? This is an interesting relation and something I will try to find some answers to during the user testing phase in project 2.

The lecture also brought up briefly some methods on how user testing can be done. Observation and think aloud is methods I have used before with good result. The methods were used in a project to analyze and format the user interface of a website. The end goal was to see how much more efficiently the website could be and improved in the overall experience.

Video analysis is a method I haven’t got the opportunity to use yet, maybe it will come in use this project? I have always looked at what’s the best method for the project and issue at hand and how efficiently data can be extracted. Take the example above with testing on a website, using video analysis seems cumbersome and time consuming. The think aloud method is more efficient and time-effective. It has more benefits over video analysis in this example. We learned that so much thoughts and opinions are in the mind of a person when browsing on an interactive web page. So it feels more appropriate to voice those thoughts out loud. And also voice them immediately than to continue do tasks and talk about possible issues and snags in the design in the end of the session. Most likely if the users have to remember and think back on all the faults, there will be some loss in the level of details in remembering all the errors. That’s why I really favor the think aloud process were the user thoughts and critic is unfiltered and voiced as the user is doing the tasks.

Leave a comment