By Eric Jeitner and Linda Salvesen

Even as many higher education institutions in New Jersey continue to open, it seems that both virtual and hybrid learning are here to stay, at least to some degree. One important library process that has been greatly altered by the pandemic is usability testing—arguably, an operation more important than ever, as many of our patrons continue to rely heavily on our libraries’ virtual services. Since the authors recently navigated this environment to conduct usability studies at each of their respective libraries, despite the challenges presented, they wanted to share their experiences so you could try it at your library!

For reference, both studies occurred within the 2021-2022 academic year: Eric Jeitner conducted a website usability study in Fall 2021 at Stockton University’s Richard E. Bjork Library, while Linda Salvesen conducted a discovery usability study in Spring 2022 at William Paterson University of New Jersey’s David & Lorraine Cheng Library.

This collaboration was initiated since both are from libraries that are members of the New Jersey Academic Library Network (NJALN), which is a consortium of academic libraries within NJ that implemented the same ILS together and now maintain a shared lending network. Implementing an ILS as a group has become more common recently, and it provides an opportunity for partnership between institutions.

Similarities between these two usability studies include the fact that each of the respective institutions required IRB approval before these studies could begin–this ensured that subject privacy and confidentiality of data was strictly maintained, including anonymizing all data and recordings. Both studies also utilized Zoom, where the platform’s screen capture capabilities were used to record both video and audio.

In terms of methodology, both studies also utilized the Talk Aloud Protocol paired with task-completion exercises. Subjects were given a list of scenarios that required them to interact with the online library tool being evaluated. Great care was taken in designing the questions as well as not leading the participants’ thinking or subtly directing them towards successful task completion. Outside of the tasks, subjects were asked about their library instruction history and provided the opportunity to ask the interviewer debriefing questions pertaining to the tasks.

In both cases, recruitment was the biggest challenge, by far. While flyers were created and displayed around each campus, posts were made on social media or in campus announcement emails, and incentives were offered, the biggest drivers of participation were personal connections and the spread via word-of-mouth.

There was one fundamental difference between the studies conducted at each institution, and that was the study environment itself. At WPUNJ, Linda chose to conduct the usability study completely virtually, based on both the continued uncertainty surrounding the pandemic and the virtual usability testing success stories that were starting to appear in the literature. This had a few benefits, including streamlined scheduling and a more comfortable testing environment. In contrast, Eric conducted the entirety of his testing in a library conference space with on-site student subjects. This gave greater control over the study environment, but masking and social distancing proved to be challenges. In both cases, it was difficult to read any kind of nonverbal communication that usability researchers usually like to analyze.

Overall, each librarian gained new insights from their respective studies, which they are sharing with both their user education and website maintenance teams. They also hope to continue fostering a usability testing culture in their libraries so that their libraries can be more proactive in regularly evaluating their web presences and following an iterative design philosophy.

Eric Jeitner (eric.jeitner@stockton.edu) is the User Experience Librarian at Stockton University's Richard E. Bjork Library.

Linda Salvesen (salvesenl@wpunj.edu) is the Systems Librarian at William Paterson University's Cheng Library. Feel free to contact us with any comments/questions about usability testing!