Advanced Search
Username
Password
Forgot password?
 

Top Forum Posts
Welcome to the Catalyze Forums

The Forums on Catalyze give members an opportunity to network with other members and ask/answer questions on current topics.
Want to post?

You must be a registered member of the Catalyze community to post;

Click here to JOIN TODAY  If you are already a member, SIGN IN HERE
 
Forums > All Catalyze > Usability Testing
Subject: What do the observers typically watch/see while testing?
 Add Tag
You are not authorized to post a reply.  
Author Messages
Rating:
jdnutter
Posts:4
12/13/2007 11:16 AM Alert 

I am having a company set up a fixed usability lab in a small conference area 22 by 12; we do not have room for an observation area.  My question is: do clients what to see a screen view of the testing sessions in addition to seeing the participant?

 How do other people handle the client observation part of usability testing?

 

Thanks for your support.

 

James
John S
Posts:10
12/15/2007 2:27 PM Alert 
If I understand what you're saying, yes. Many times the user will have a difficulty they can't fully articulate. Simultaneously showing what's happening on the screen in front of the user can provide valuable extra hints about what they might mean when they say things like "I didn't expect that to happen."

BobGoodman
Posts:1
12/18/2007 11:13 PM Alert 
Hi, James. I have run hundreds of usability lab sessions, and I can tell you that you definitely want someone from the client to be able to see the screen, or watch it projected. You want the client representative to be able to watch the lab participants' "behavior", ie, where they are clicking, as well as their comments, and be able to compare the two. A user's actions often speak louder than their words, and can be revealing of gaps between the system's design intent and the user's reality. A simple solution is just to have one observer present, sitting back from the table near the user, or to have an extra monitor setup across the table that's "projecting" the user's desktop. Best regards, Bob
mmdeaton
Posts:7
03/27/2008 7:20 PM Alert 

James,

I realize this may be too late to be of much use, but as a professional usability engineer, I want clients to observe user testing whenever feasible, but I do not want them in the same room with the test participant nor do I want them to interact with the participant. I have clients sit in another room or broadcast the test video remotely to the client's location.

It is very difficult to create a test environment that does not impact the outcome of a test. It is critical to avoid any disruptionsm, distractions, or conditions that could seriously impact how the test participants interacts with the product.

I do not even sit in the same room with the participant when I facilitate a test so that my presence does not influence the outcome.

I also avoid taping the user's face unless it is critical for gathering the data. I will record the face, but not broadcast it to observers and not include it in any highlights tape. Participants are entitled to privacy and in this day-and-age, to feel confident their session video will not show up on YouTube!

That said, if you feel you have no other option, you must make sure that any observers are quiet and say nothing at all during the testing.

Mary Deaton
montgomeryam
Posts:2
04/03/2008 12:12 PM Alert 
Yes, client participantion is a must! You definitely want some client participation, and developer participation.

I do sit in the same room becuase I learned quickly that testers will talk more when they feel like they are talking to someone. It didn't work so well for us to stick them in a room by themselves.

We use the Morae software so the observers are watching the testers screen and headshot while hearing the testers comments.
mmdeaton
Posts:7
04/03/2008 3:11 PM Alert 

I never have problems getting participants to talk when they are alone in the room. Gentle prompting is all it takes. Of course, I spend time with them before the testing begins and they know who I am and what I look like. We rehearse the think-aloud protocol and they understand why I want them to talk as they work.

I think the risk of contaminating the results far outweighs the risk the user will not talk enough while working.

Mary Deaton
montgomeryam
Posts:2
04/03/2008 4:02 PM Alert 
But what do you do when they have a total road block and don't finish the task? What we do is note it as a faluire but then help them get over the hurdle so they can finish the task some then by doing that we learn what they feel was the road block.

Do you feel that's contaminating the results? I get the same result as you which would be 'faliure to complete task' but then maybe get more information as to what the road blocks might have been.

I'd be interested in hearing your thoughts.
Thanks - Amy
LMarine
Posts:28
04/03/2008 11:03 PM Alert 
It’s more than contaminating the results, it can invalidate the test. You no longer are testing your interface as much as you are observing your users ability to follow your lead. Once the user has been guided towards a solution, any reaction, be it verbal or nonverbal, is now biased by their new information. The whole point of bringing in users is to gain from their objectivity and any interaction you have with them begins to bias that objectivity.

We've run into roadblocks before and, with just a few non-leading questions, were often able to identify the root cause of the problem. Sometimes the road block is the simple result of the wrong word for a button, and we can usually help the user understand what we meant by the verbiage on the page without leading them by the nose. (Not to say that’s what you were suggesting. I just like that phrase.)

We’ve also found that by letting users struggle for an appropriate while, we got more insight into how they were thinking, which usually led to a better solution. It’s not usually necessary for the user to complete the task. We also try to write our test scenarios so they don’t rely on the successful result of a previous task.

My two cents…