Tag Archives: usability studies

In a pre­vi­ous life, I was a Library and Infor­ma­tion Sci­ence mas­ters stu­dent, and part-time ref­er­ence desk librar­ian. I worked in the Bio­med­ical library, con­nect­ing med­ical stu­dents and pro­fes­sion­als to jour­nals, arti­cles, research, stud­ies, books, knowledge…I attended med­ical library con­fer­ences and stud­ied how to help med­ical pro­fes­sion­als with “evidence-based prac­tice.” As a
result, I became adept at help­ing peo­ple find evi­dence to base their prac­tice on in med­ical research lit­er­a­ture. That skill has cer­tainly served me well as a user expe­ri­ence researcher as I work closely with design­ers and stake­hold­ers, and now unsur­pris­ingly, find­ing and deliv­er­ing evi­dence to inform design deci­sions is my absolute favorite part of this won­der­ful job.

I’m in the thick of user research on my cur­rent project, and along the way I’ve been remind­ing myself of all sorts of best prac­tices I learned from the bril­liant Joan Kaplowitz, an Infor­ma­tion Lit­er­acy Librar­ian.  Using her advice, I do every­thing I can to make sure the peo­ple con­sum­ing the infor­ma­tion I present, are get­ting what they need and inform­ing the design ques­tions they have.

My hope is what works well for me, using Joan’s prin­ci­ples, will help you too when you have to com­mu­ni­cate your results to a team of stakeholders.

1. Engage your stakeholders

“Active learn­ing is bet­ter than pas­sive recep­tion for reten­tion and trans­fer of learn­ing.” Joan Kaplowitz

In my expe­ri­ence, the best way to engage stake­hold­ers, and have them actively learn usabil­ity issues is to involve them in the research.  Hav­ing them come to observe usabil­ity test ses­sions and con­tribute their obser­va­tions to an affin­ity dia­gram is a very effec­tive way for them to retain the feed­back they hear, and see trends across users.  One strat­egy I learned from attend­ing the Nile­sen Nor­man Group Usabil­ity Boot­camp in 2007 was from Kara Per­nice: have your observers take notes on post-its (quotes, obser­va­tions, one issue per post-it), and then change color post-its for dif­fer­ent par­tic­i­pants. Then you can start to group issues together with stake­hold­ers, and see usabil­ity trends across users (col­ors). Your dia­gram of groups then can turn into your report, tran­scrib­ing your wall of insights into a pre­sen­ta­tion slide deck or any other format.

2. Keep it light­weight

“Less is more. It is bet­ter to teach a few things well than to over­whelm learn­ers with so much infor­ma­tion that they become frus­trated, anx­ious, and unable to retain any­thing from the instruc­tion.” Joan Kaplowitz

The best way to describe this prin­ci­ple as it applies to report­ing usabil­ity find­ings was best artic­u­lated by Todd Wilkens from Adap­tive Path: “The effec­tive­ness of your research report is inversely pro­por­tional to the thick­ness of it’s bind­ing.”  You will be so much more effec­tive in mak­ing change hap­pen, and get­ting peo­ple to take action on your find­ings, if you don’t over­whelm them with new infor­ma­tion. Your goal is to inspire your stake­hold­ers to address the usabil­ity issues, and have design­ers solve the usabil­ity prob­lems.  Then, val­i­date those changes were suc­cess­ful with another round of testing!

3. Get them talking

“The instructor’s voice should be the one heard least dur­ing teach­ing.” Joan Kaplowitz

The best pre­sen­ta­tions I’ve ever given have been because I met with stake­hold­ers ahead of time to get a sense of what they learned from watch­ing usabil­ity ses­sions. Get design­ers to tell you what they learned, because it doesn’t mat­ter so much what is in your head, as it does what is in theirs. Before giv­ing your final pre­sen­ta­tion to the larger group of stake­hold­ers, talk with your key stake­hold­ers, and have them tell you what they learned, observed, and took away from the ses­sions.  Review the take­aways you intend to present, and invite them to add to it. Your final pre­sen­ta­tion will be far more effec­tive at inspir­ing change to happen.

4. Grab them in the first 5 minutes

“You win or lose your audi­ence in the first five min­utes. Talk to your learn­ers as they enter the room. Pro­vide them with some­thing to do and/or think about as peo­ple are arriv­ing. Engage them from the start.” Joan Kaplowitz

Some­times the begin­ning of the find­ings pre­sen­ta­tion is the most chal­leng­ing, because all your great insights are com­ing later in your slide deck.  How­ever, the begin­ning of a pre­sen­ta­tion ALWAYS sets the stage for the mes­sage you are giv­ing. Get to the pre­sen­ta­tion room early, and build rap­port with each per­son who comes through the door.  Great them with a hello, and say their name (peo­ple love to hear their own name).  The goal before the pre­sen­ta­tion starts is to get “audi­ence mem­bers” on your side, because in my expe­ri­ence, these are the folks that that are most likely to help fend of heck­lers in your defense! Also, you want a strong intro­duc­tion.  You def­i­nitely don’t want to stum­ble through your first 2 min­utes of speak­ing.  Here is a basic intro­duc­tion that I use often — but the more cre­ative you can be the better!

“Great, let’s go ahead and get started. Today we are going to talk about_____________. We have about 45 min­utes for the dis­cus­sion, and I’ll leave some time at the end for ques­tions, but if you have any ques­tions along the way, please feel free to ask. First I want to ask… who had a chance to observe the ses­sions?  Great, well if there is any­thing you would like to add that you remem­ber observ­ing, please feel free…”

5. Demon­strate each key issue

This advice I learned on the ground floor at the UCLA Bio­med­ical Library ref­er­ence desk.  Stu­dents, nurses, patients, etc. would come to the desk ask­ing how to use the library data­bases.  The most effec­tive way for them to learn how to use the data­bases wasn’t just telling them, but actu­ally demon­strat­ing it to them. In fact, it was best if you could hand the key­board to them, and have them drive them­selves.  In an usabil­ity find­ings pre­se­n­a­tion, you really want to re-create the issues that par­tic­i­pants had dur­ing the ses­sion.  Stake­hold­ers tend to be quite visual, and greatly ben­e­fit if you show them what you mean.  Take full screen shots, and sim­u­late the expe­ri­ence of using the tech­nol­ogy using basic ani­ma­tions such as fad­ing in.  This seems to be the best way to re-tell the story, and have your find­ings really sink in with stakeholders.

6. Keep track of your effectiveness

“Always include a way to assess your out­comes. Oth­er­wise you will have no idea if your learn­ers have attained them.” Joan Kaplowitz

In the world of UX Research, the best way to assess your out­comes is to keep track of which issues are on the plan to get­ting fixed, and which ones have not been fixed. You can keep track of these in an Excel spread­sheet on your own, but the best is to have your issues make it to the prod­uct roadmap.  You want to see your usabil­ity issues turn into “user sto­ries” if you are work­ing with Agile teams, or pos­si­bly the roadmap, bug fixes, or “change requests” if you are work­ing in a water­fall envi­ron­ment. How­ever your orga­ni­za­tion keeps track of changes that need to be made to the sys­tem, do what you can to make sure usabil­ity issues are rep­re­sented there.

7. Take a multi-format approach

“Use hand­outs, web pages, and pre and post activ­i­ties to extend your con­tact time with your learn­ers. Offer ways for learn­ers to keep the con­ver­sa­tion going after instruc­tion is com­pleted — through email, blogs etc.” Joan Kaplowitz

A Pow­er­Point pre­sen­ta­tion is a great way to walk through usabil­ity find­ings.  How­ever, you don’t want to only deliver a slide deck. In addi­tion to your pre­sen­ta­tion, you will want to give a “hand out” to your key stake­hold­ers, the peo­ple who need a “check list” of things to address.  Don’t rely on them tak­ing their own notes dur­ing the pre­sen­ta­tion, give them a list that they can eas­ily make notes to.  You don’t need to pass this out to every­one, just to your 1 or 2 key stake­hold­ers. Also, I’ve seen a fol­low up email work really well, with a link to your pre­sen­ta­tion, to the record­ing of the dis­cus­sion, and a cou­ple bul­lets that sum­ma­rize the key take­aways.  Invit­ing users to con­tinue the con­ver­sa­tion using wiki pages is a great way to keep them engaged, and con­tinue with next steps.

So there you have it. This is my very best advice on how to present usabil­ity find­ings so that it inspires your stake­hold­ers to take action. Many of the infor­ma­tion lit­er­acy prin­ci­ples listed here apply to bas­ing design deci­sions on evi­dence.   How do you com­mu­ni­cate your usabil­ity issues?