A Cayugasoft client, the document management company DocuGain, which had only web interface, asked us to research the idea of a mobile app for them. This was a new business domain for our product and design team. Most legacy document management systems have no mobile versions, or very weak ones, so even competitive analysis didn’t provide much guidance. We decided to use the power of usability tests to validate the mobile app idea quickly (read: keep the budget low), and the concept of remote usability testing helped us a lot with this.
What you need to get prepared for usability testing:
In order to find the balance between the number of test customers who will participate in the test, and ensure your team has useful analytics, you need at least 8 – 10 respondents. The more the better, of course, but we’re trying to get useful input while keeping expenses down.
If you are going to test a mobile app, make sure you know what OS your customers use in real life: iOS or Android. Obviously, testing an iOS app with Android users (and vice versa) is going to impact your results (unless, for example, you intentionally want to check Android user behavior with iOS app).
There are many ways to build a prototype. For DocuGain, we used Marvel.
In our case we wanted to test usability with remote users, so we had to set up a test infrastructure that permitted us to get an impression of the user’s real-live experience using the app, including their emotions. Lookback.io does this well: you can see what the user is doing, and notice their reaction from the mobile device’s camera; later the test team can download and analyze the sessions offline.
Lookback.io has a 2 week free trial, so you can easily test it with your first customers for free!
Prior to the test sessions, it is important to write a script which contains a meeting plan and test tasks. It is crucial to ask the same questions, in the same order, for all test users – it helps to emulate the same situations for all people, and it will allow you to gather comparable feedback.
Plan your test session for 15-20 mins, if it lasts more, people tend to lose their attention, and this will influence your results.
The most important thing is to have a clear understanding why you need to perform usability tests, and the questions you need answered. e.g., if you are not sure about the flow for a certain feature, or the icon which is used to represent a function, be sure to include these in your test tasks, and watch how people behave while using the prototype.
You need to have a list of the metrics that you want to gather, prior to starting the test sessions. It is also important to keep track of the percent of user errors that occur during each test session, and the user journey while performing the tasks.
When all the test sessions were done, and the results analyzed, the team made a decision to change several user flows in the app which turn out to be unintuitive. We also decided to change the layout and some icons, which were confusing in some cases. As a result, DocuGain invested just 2 people from the product team for 2 weeks during usability testing, but saved the budget for 3 sprints of development by 2 developers and 1 tester. What is also important is that DocuGain was able to accelerate development of their mobile app by several weeks, and ensure the resulting app was on target with user expectations.
When doing usability tests, it is important to keep in mind that you are building your product for users, not your dev or product teams. By using a structured process for user testing and analyzing user feedback, you can accelerate the delivery of your product to market, and reduce development costs significantly.