Diary studies are a form of longitudinal research (research that takes place over a long period with the same participants). Typically, users self-report their activities at regular intervals to create a log of their activities, thoughts, and frustrations. It’s a useful approach for capturing organic feedback on activities that are repetitive, long, or unpredictable. Research questions that involve habits, prolonged processes, or perception change over time (likelihood to use, learnability, etc.) are prime candidates for diary studies. As with all longitudinal studies, the amount of time required for the study is important. If you can get a reasonable snapshot of behavior in an hour or a day, for example, diary studies may not be right for your questions—try an unmoderated study instead. While diary studies are more complex than typical usability studies, they can be done using the UserTesting platform.
Here are some examples of scenarios that might lend themselves well to diary studies:
Diary studies are also great for capturing information that you might not normally think to do with a remote, unmoderated test. For example, you could recruit users from the UserTesting panel and ask them to review a physical product like a fitness tracker, or you could ask them to play a Wii game for several days. Similarly, you can use a diary study to discover the habits and motivations that users might not be able to give accurate, nuanced, or complete enough answers about during a standard, one-time test.
Although they may seem similar, don’t confuse diary studies with iterative research. Many UX practitioners test the same product consistently and often (as you should!), and may even use the same participants if they so choose. This type of testing provides a snapshot of a user’s experience with the product. Diary studies, on the other hand, provide a look at what prolonged exposure to a product or process is like for the user. This can also include usability issues, but these studies are typically of a broader scope—looking for problems that affect or derail the overall process, rather than small annoyances or design preferences.
While diary studies are a bit more time-consuming than regular usability testing, their uses are diverse and fascinating. They provide a clear slice-of-life image of your users, their needs, motivations, and reactions. UserTesting customer and e-learning platform, Udemy, relied on a diary study to better understand how students utilize both desktop and mobile devices when using the platform. UX Researcher, Michelle Fiesta noted one advantage to the diary study methodology,
We wanted to observe our users over a longer period of time, noting behaviors as they happened to determine how/if these user groups were different, which wouldn’t have been possible with a one-off interview.
As with any customer research, the effectiveness of a diary study depends on proper set-up and study design. It’s important to note that diary studies can be more complex than remote, unmoderated studies. When possible, it helps to have direct contact with your participants, typically through customer interviews before and after the study is completed.
We recommend keeping the demographics and requirements as flexible as possible. This helps keep the recruitment schedule on track while giving your team more time to inform participants about the study length as well as what will be expected of them. Additionally, because diary studies require more time from participants, higher compensation than standard customer research will be required.
It’s important that the instructions that you send to participants are clear and explicit. Outline exactly how often you expect them to keep a diary, how long the diary should be, and whether you expect images, screenshots, survey responses, etc. Try to keep your instructions as clear and simple as possible. It can be helpful to provide a list of questions that you want participants to answer or keep in mind as they complete their diary. For example, you might want to ask participants what prompted them to start [activity] each day, what websites/apps they used to complete the activity, and what their experience was like each day. Because of how organic diary studies are, users may take your question list as a suggestion rather than a strict plan, so if there are questions that you must have answered, be sure to make that explicit as well.
At the beginning of the diary study, you can schedule a live interview session (30 or 60-minute sessions) to help set expectations and get to know your participants. Establishing this connection can help participants stay committed and responsive throughout the study while also helping reduce participant dropouts. Additionally, it can be helpful to schedule follow-up interviews with participants once their diary study is complete. This gives both you and your participants the opportunity to expand on their overall experience and address any questions that may have arose during the study. This approach is especially helpful if you have a limited number of participants, as it will give you additional insight and detail to complement the findings from the diary studies.
If you’d like to learn how UserTesting can help you understand your customers through on-demand human insights, contact us here.