Usability Testing Content

For the past three years, I’ve been run­ning usabil­ity tests on soft­ware appli­ca­tions. Most of the tests have been to mea­sure and iter­ate on designs and inter­ac­tions.  How­ever, lately many of my projects my team has taken on have been related to *test­ing con­tent*.  I real­ized that I needed a refresher on how to test con­tent, and wanted to share with you what I’ve been review­ing. High­lighted below are some of the meth­ods iden­ti­fied in the UX com­mu­nity to help eval­u­ate whether the con­tent on your Web sites is use­ful for your cus­tomers and prospects.  Not sur­pris­ingly, test­ing con­tent is actu­ally very sim­i­lar to test­ing design/interactions.

Con­tent Test­ing Meth­ods:

Iter­a­tive Test­ing. Kevin O’Connor and Colleen Jones’s 2010 IA Sum­mit talk: “Test­ing Con­tent: Early, Often, & Well” is a great resource on test­ing con­tent (here is the audio). Not sur­pris­ingly, she talked about how impor­tant it is to test con­tent iter­a­tively, just like we test designs and inter­ac­tions.  She will often start with a base­line test of the con­tent that is on the live site, then fol­low that up with con­cept test­ing, and then do a val­i­da­tion test.  What was inter­est­ing was how she focused the *pro­to­col* on three key con­tent ques­tions:  Can users find and read the con­tent they need? Do they under­stand the con­tent? Can and will they *act* on the content?

5 Sec­ond Tests. Chris­tine Per­fetti wrote an excel­lent arti­cle: “5-Second Tests: Mea­sur­ing Your Site’s Con­tent Pages.”  Using this method, you’d basi­cally show par­tic­i­pants a web page for 5 sec­onds, take it away, and then ask them for their ini­tial impres­sions.  Chris­tine says you can ask them to describe back to you what they saw.  She says users make impor­tant judge­ments in the first moments they visit a page.  This method helps to uncover those judge­ments.  Also, it can mea­sure whether the calls to action on the page are appar­ent enough.  She describes a case study with the Amer­i­can Red Cross, where 5 Sec­ond Tests helped them iter­ate on the design of their dona­tion page, to make sure users knew what all the dona­tion options were (i.e., donat­ing air­line miles, stock, clothes, etc.).

Inher­ent Value Tests. Chris­tine Per­fetti and Jared Spool talk about this method in a cou­ple of arti­cles, and in a pod­cast.  They say you’d want to run an Inher­ent Value Test when your team needs to know how well a Web site com­mu­ni­cates the inher­ent value the design­ers have put into it, and whether new cus­tomers under­stand the true value of the ser­vice.  How it works:  recruit two user groups, in the first phase, you recruit exist­ing users, and inter­view them about what they like about the prod­uct (what they find valu­able).  This will help to iden­tify fea­tures loyal cus­tomers miss.

Eye Track­ing, Gaze Plot­ting, and Web Ana­lyt­ics. I talked with a our Tech­ni­cal Writer, Mau­reen Lau, who had recently attended the Nielsen Nor­man Group’s course on “Writ­ing for the Web.”  I learned from her that the Nielsen Nor­man Group tests a lot of con­tent with Eye Track­ing, Gaze Plot­ting, and Google Web Opti­mizer.  Eye Track­ing tests how long they look at a par­tic­u­lar area (the more red, the more they looked at it).  Gaze Plot­ting, tests where their eyes jump to.  She said you go through a sim­i­lar process with con­tent as you do with inter­ac­tions (paper pro­to­typ­ing, iter­at­ing, try­ing to under­stand users’ needs, etc.).  She said Eye Track­ing and Gaze Plot­ting are great to see where peo­ple look, and what catches their eye, how­ever you need to inter­view peo­ple at the same time, to see if they are actu­ally com­pre­hend­ing what they have looked at. They will use the Google Web Opti­mizer to roll out an AB test, and see which con­tent “per­forms” bet­ter.  She told me about how read­ers typ­i­cally read in an F-shaped pat­tern.  Finally, she said we want to cre­ate tasks around the con­tent. Since users often search for key­words, they sug­gested get­ting a bet­ter sense of what SEO key­words we use, and test­ing to see if they look for those key words, where they’d expect to look for it, etc.

Test­ing Con­tent Con­cepts. Colleen Jones wrote an excel­lent arti­cle called “Test­ing Con­tent Con­cepts” that walks through the spe­cific pro­to­col, probes, and ques­tions to ask par­tic­i­pants about when test­ing con­tent.  Colleen says a “Con­tent Con­cept” is a mockup or draft of your con­tent.  She says if your run a usabil­ity test on your con­tent, and your con­tent is not work­ing well, you should fix the prob­lems and test again until it does. Colleen rec­om­mends test­ing in three lev­els of fidelity:  con­tent only, con­tent in a wire­frame, con­tent in pol­ished design.  What I found most inter­est­ing in her arti­cle was her rec­om­men­da­tion to focus obser­va­tions on how usabil­ity par­tic­i­pants work with the con­tent. I also liked her rec­om­men­da­tion of mea­sur­ing how suc­cess­ful peo­ple are at read­ing, under­stand­ing, and remem­ber­ing key mes­sages in the con­tent. Colleen dis­cusses the biggest chal­lenge in test­ing con­tent:  mea­sur­ing how the con­tent has influ­enced the par­tic­i­pant to take action. She says “don’t press too hard, and thus, end up with mis­lead­ing ratio­nal­iza­tions.” One idea Colleen has that could work well for us, is to con­duct a clos­ing ques­tion­naire  that asks whether par­tic­i­pants would now make a deci­sion that dif­fers from their typ­i­cal deci­sion, and to rate how well the con­tent informed their decision.

I’ve never been so con­vinced of how crit­i­cal it is to test not only how to nav­i­gate to con­tent and how to find con­tent, but to test the con­tent itself.  It is the con­tent itself that dri­ves influ­ence and calls read­ers to action.

If you have any com­ments, ques­tions, or alter­na­tive meth­ods to test con­tent, please leave a com­ment below!