USABILITY TESTING
Validating a design
Scholastic was able to correct adoption-blocking mistakes before launch
The Problem
Scholastic Classroom Magazines was redesigning its magazine websites in an effort to bolster its digital offerings, and it wanted to make sure the new sites worked for teachers and students.
The Solution
This is a situation that called for usability testing. In this case, as a practical matter we focused on the teacher preparation step rather than in-classroom use, so we invited teachers to open the website and use it to plan a lesson.
Project at a glance
-
I was the sole researcher
-
I worked with stakeholders of the dozen-plus individual magazines
-
Tests included a handful of teachers for each magazine, for a total of about 50, and were conducted
-
The process unfolded over a period of two months
Deep Dive
My philosophy of usability testing is to think of the it more broadly as an information gathering exercise in which the main goal is to keep an open mind throughout. It is mission-critical to treat the interviews with flexibility and sensitivity, and to listen closely to the unspoken. A script is of course a requirement to align the team on goals and discuss the questions already in their minds, and through the interviews there needs to be data-gathering consistency to get a sense of the severity of issues, but in my experience, the most valuable feedback is the one you didn't expect. Flexibility is the only way to get it.
​
In this case, we had a product that was meant as a teaching aid — a digital supplement to the printed magazines that Scholastic is known for — for use in the classroom with school students, as well as teachers, and we would need to deal with permissions for getting into a classroom. After some attempts, we decided it would be more expedient, and we wouldn't lose much in the way of data, to run the tests with individual teachers, since the use of the site began with them, and we could trust their view about how it would work in a class setting. Also, doing so would free us from the need to go into the classroom and we could run remote tests, which opened our recruiting options considerably.
​
Once we figured out the plan, the logistics were typical of usability tests. We contacted teachers from magazine subscription lists. Recruiting and scheduling was conducted by editorial teams, who already had relationships with teachers. In the remote sessions, we had the teachers follow think aloud protocol as they used the site, based on tasks we devised that simulated classroom prep. The editorial and web teams gathered in a conference room for viewing sessions, and after each test we debriefed — this approach is far more impactful than what results from reading a report, even though in the end a report was delivered as well.
The Outcome
The tests successfully answered the questions that site designers and editorial teams had about how their design decisions would play, including some that were found to pose significant barriers to adoption. The teams settled on a number of updates in then end. In one case, the home page top content area was significantly reworked to clarify intent and to be more engaging.
​
The site met with considerably improved response after the updates and went on to see healthy adoption. The usability test had staved off an adoption disaster.
Contact me if you are looking for more detail. I can provide it privately.