Peanut butter and jelly.
Coffee and cream.
Calvin and Hobbes.
Gin and tonic.
Sometimes, great things come in pairs.
And if you’re research-minded like myself, there’s nothing better than pairing quantitative data with qualitative insights.
While UserTesting’s platform was originally designed for qualitative research, our internal Research Team has learned that the tool can pair well with many quantitative methodologies to create powerful findings for your site or app.
Today, I’d like to share an unusual-but-awesome pairing: tree testing and unmoderated, remote user testing.
Tree testing lets you show users a menu structure in its most basic text form so that you can test your site’s organization and terminology---without worrying about visual cues or design elements influencing users. For this study, I chose TreeJack (a handy tool made by Optimal Workshop) to run my tree test.
For the pairing, I decided to use the website of my undergraduate alma mater, Troy University (www.troy.edu). Troy is a medium-sized school in southern Alabama with multiple locations, a rich online education program, and some outstanding alumni (but maybe I’m biased).
It’s no secret that university and government (and even some e-commerce) websites often suffer from way-too-complicated navigation. Menus can get massive and unwieldy, and ultimately, users end up struggling to locate the links that they need to accomplish basic tasks.
Clean information architecture is one of the keys to improving the usability of any website, and tree testing is a great way to find out whether your current navigation is getting the job done.
Like most websites for higher education, Troy University’s site suffers from an excess of information. Many links in its main navigation menu are duplicated multiple times. Application prompts appear on the homepage and within at least three of the site’s sub-menus. The academic programs (areas of study) are linked even more frequently. And some menu links actually re-route users to new subdomains like admissions.troy.edu. On the subdomain, some menus and links are duplicated, depending on whether you’re looking for undergraduate, graduate, or international student information.
So it got me wondering: just how clear are the menu options on Troy’s website? And could tree testing combined with user research help me answer the question?
To set up the tree test, I had to recreate the site’s menu structure. To do this, I started on the homepage and navigation menus to create a list of the possible links that users can take to dig deeper into the site. Each link represents a different “branch” of the tree, which then divides into smaller branches, and smaller ones...and you get the picture.
Above is the menu that drops down when you click “Academics.” The next branches of the tree are Undergraduate, Graduate, Colleges & Schools, and Academic Programs. As you can see, there are even MORE links under each of those branches.
Recreating Troy University's navigation in Treejack
It took much longer than I thought it would! Some information is buried four or five clicks deep into the site, and every one of those links needs to be put into the tool. There are several sitemap generator tools available online (and most are free), if you prefer not to get too mired down. However, I enjoyed the exercise of exploring the site’s different trees.
Once the site’s tree of links was set up, I had to pick a few common tasks that a user might want to accomplish on the site. I chose the following three:
The next step was to distribute the tree test, and this is where pairing with UserTesting becomes valuable.
TreeJack generates a simple URL that can be distributed to users however you like; I made it the starting URL in the User Testing order form, provided some explanation to users that they’d be doing something a little different for the test, and then launched the test.
The tests filled in under an hour, and I got to watch the users take the tree test while thinking out loud. This clued me in to some important misconceptions that I’ll discuss in just a moment.
For this study, we asked 10 users to complete the tree test while recording themselves. Typically, tree tests are run with 30+ users, but if you use the think-aloud methodology and record with the UserTesting tool, you can capture a lot of great insight with a smaller sample size.
When it comes to understanding tree test results, there’s some terminology that needs to be understood (but hopefully it’s pretty intuitive):
“Direct success” means that users read the task, navigated the map without hesitation, and found the right answer without getting distracted by incorrect links.
“Indirect success” means that users tried different branches within the tree before finding the correct answer. Perhaps they clicked on two or three different branches, looking for keywords in the sub-menus or deliberating about which label best met their needs….but they found the right link eventually.
“Failure” means that users indicated they would complete a task using an incorrect link.
“Skip” means that users gave up on the task entirely. However, you can turn off the users’ ability to skip tasks within TreeJack, and I utilized that option.
Troy’s website provides four ways to access this information through the menu, and so I expected it to be the easiest task to accomplish. As it turns out, I was right:
Task 1 had a 70% direct success rate.
TreeJack made it super easy to tell how well users did at each task, and as you can see, 7 of the 10 users in my study found the Dothan campus map without difficulty. The pietree they offered also showed me where users went off the rails: one user wandered into the “Academics” section of the web site, and another into the “Online Learning” section.
The pietree shows the different routes (correct and incorrect) that users took to find the campus map.
As a student of Troy University, I often walked across the central quad, never thinking very hard about the possibility of my graduation date being immortalized in brick beneath my feet. Family members, however, love the opportunity to honor their recent graduates with a gift like this!
It all starts with finding the order form online, however, and if my tree test is any indication, family members of Troy Students are much more likely to leave the site and purchase a Visa gift card or a copy of Oh, the Places You’ll Go! by Dr. Seuss as a gift for their grad.
This task only had a 10% direct success rate.[/caption] As you can see, only 1 of my 10 users found the correct link on their first try, and only 2 more found the correct link at all.
There was only one correct path to take from the home page: click on “Giving,” then “Make a Gift Online.” It’s not a very complicated flow, so why such a low success rate?
This pietree shows the convoluted routes users took to find the right information.
As the pietree above indicates, many users visited the “Alumni” menu instead, and funneled down to “View All Available Ornaments”, which was the only link that remotely relates to a commemorative gift.
I was a little puzzled by this activity: why would users think that “Alumni” was the right menu for this task? Then I watched the videos on the User Testing platforms, and got my answer: in their minds, a fresh graduate IS an alumnus.
In addition to this tidbit, I learned that users were the least confident in their selections during this task. Like the user above, four others admitted that they were not at all sure that the link they selected would really get the task done.
Most parents and students who are thinking about college have one basic question: how much is this going to cost me? I figured that this task should be somewhat easy to complete, because it’s one of the first things you want to know about attending a new school. To my surprise, this was the most difficult question for users to correctly answer.
This task had the highest failure rate: 80%.
Only one user found the information right away, and only one other found it by trial and error. The other eight users ended up... all over the place!
This pietree indicates that this task had the most confusing navigation.
This was the pietree with the most branches, meaning that users went down wrong paths most frequently when searching for this information. As I watched the user videos of this task, I noticed that, more than for the previous tasks, users were haphazardly clicking on options, sometimes expanding multiple items within a list and discarding them. Looking at the pietree, it’s easy to understand their confusion. The “Future Students” and “Academics” paths both seem likely to contain graduate tuition info---but only one path actually does.
This probably isn’t a huge surprise, but the more places a site links a particular piece of content, the more likely it is that users will find it. Task 1 had four possible correct answers, while tasks 2 and 3 could only be accomplished by getting to one specific link. So partly, good navigation is a numbers game.
However, don't take this to mean that all information should be found in every menu. That would be overwhelming---and not helpful at all! Instead, websites like Troy University's could benefit from crosslinks on pages where users are likely to become confused. For example, placing a link to tuition information on the "Graduate" page in the "Academics>Graduate" route in the example above could have helped the users find it much more easily.
Even more important than providing crosslinks is using clear terminology to begin with. There’s a lot of jargon on Troy’s website that’s specific to the scholastic world: undergraduate, graduate (which has two meanings), admissions, academic programs... and if you’re not privy to the meaning behind those terms, it’s easy to think that admissions costs and fees are the same thing as the costs and fees for the whole semester.
After watching the videos, I thought that I probably could have provided a little bit more context about the unique testing situation in which I was putting my users. A simple heads-up that they’d be testing a university’s site structure might have made their first glance at the menu a little less jarring.
On the other hand, the lack of context for tasks meant that users weren’t in a particular mindset when they approached the site’s content. Troy’s main navigation menu is definitely designed for people with different mindsets; their first link is titled “Future Students”, and there are also links specifically for “Military”, “Alumni”, and “Student/Faculty & Staff” visitors. But in between those links are topical options, like “Giving” and “Locations”, which could be for just about anyone.
Some navigation options in this menu are for certain people; others are for everyone.
This led me to my biggest conclusion from the study: maybe it’s worth separating areas that are targeted to specific users from areas that discuss specific topics.
For example, Drexel University’s website has separate navigational bars for user groups (Current Students, Faculty & Staff, Alumni, and Parents) and topical content (The Drexel Difference, Academics, Research, Campus Life, and About Drexel). Some other well-designed college websites do this as well.
Drexel's site provides separate navigation options for different user groups.
I like to think so. While I was able to make some intuitive leaps based on the quantitative data provided by TreeJack, I thoroughly enjoyed watching users go through the tasks within the tool; seeing not only where they click, but where they hover, and hearing the reasons why one link makes more sense than another. It helped me separate organizational issues from terminological ones, and it gave me an inside look at users’ confidence, all of which leads to additional research ideas in the future.
If you're looking to improve your site's navigation, I would encourage you to give tree testing a try. You can try Treejack here. Happy testing!
Get our best human insight resources delivered right to your inbox every month. As a bonus, we'll send you our latest industry report: When business is human, insights drive innovation.