YASB – Yet Another Symfony Blog

July 14, 2007

RobotReplay, a usability testing tool

Filed under: Ajax, Browsers, General, Tips and tricks — Krof Drakula @ 4:54 pm

If you’ve ever asked yourself just why your web site or web app doesn’t perform as well as you’ve thought it would, since you’ve given a lot thought into the design of the page and user interface, then you might ask yourself – are you the proper person to be judging the design?

In comes usability testing. It’s giving the end user a chance to work with your product (in our case, a web page or application) and having a system measure the user interaction – how long it takes them to locate a certain function, where they have the most difficulty identifying a certain feature, finding information, etc. There’s already great software that does exactly this (Morae, for example), but the problem here is that you still need a lab to conduct such experiments.

Now when I say lab, I don’t mean the white coat and bottle-bottom glasses in a sterile room wearing gloves, an ordinary room is just fine. The problem is the software – you have to have a recorder (the computer where the user sits and uses your application) and a player (the computer where you observe and store the user recordings). Here’s where it gets a bit hairy – the user needs to come to you for you to have any recordings at all. Now, there are remoting options (Techsmith provides this with Morae), where you can have user on separate locations and viewing all sessions on a single master computer, but you still need software installed on the client, wherever in the world they are.

The costs involved in such a study grossly overflow the budget on most projects, so most just give up. Paying for a test computer, the software (Morae just ain’t that cheap, mind you), the people testing the software and, probably most importantly, researching enough of the target audience to know which people to enroll into your test program. And how many it takes to run a reliable test.

Wouldn’t it be great to have a tool that could run anywhere with any browser on any hardware (maybe even within the browser)? Well, Google Analytics is usually the weapon of choice for most – it provides you with marketing-oriented tools that visualize the way your users search for information on your website, all geared up towards maximizing your visibility on the web, making a profit from AdWords (and such) and selling your product more effectively. All fine and dandy, but having only the path your users take and the time it usually takes them to reach there is usually not enough.

Next step? CrazyEgg. It takes user clicks and turns them into a sort of a blended heat map of your web pages to show you where your users tend to click most. An indispensable tool to help you visualize the numbers you’ve already had at your fingertips with Analytics. But still, this just isn’t enough to address key questions like “How long does it take a user to locate the login button?” or “How long does it take for them to fill out this form, field by field?”

Now there’s a way for you to answer those kind of questions, at least inside a browser – with RobotReplay. It does the same function as user screen recorders do – it takes recordings of user interactions (including mouse movements) and sends Ajax requests to a recorder server, where the data is raassembled to create a complete recording of the user session. And did I mention it’s just plain old Javascript?

The purpose? Get data from REAL users, not the ones you expect to use your page. This way, you get REAL metrics from users that you haven’t handpicked, which is good, since you can rarely predict who will actually be using your product, or use it in a manner that you can foresee (unless you have tons of money at your disposal to rent out psychologists, ergonomists and interface design experts).

There is a problem that I can see here, and it’s not the technology. It’s the level of privacy that the user gives up when browsing your site. This is the equivalent of having someone peek at your computer screen from behind your back without you knowing. Granted, he/she only gets to see stuff on your web site, but still. Granted, the user is not identifiable by any trivial means. But the user isn’t aware of the level of scrutiny that is involved when they browse the website. Analytics and CrazyEgg only aggregate information, usually no single user’s path can be determined (unless you have a single user generating the statistics), but RobotReplay handles each session as a single recording, not just an aggregation of. Here’s where it might be prudent to notify the user on how their data is being collected and whether they do want their session to be saved, with a promise of using this data to enhance the quality of service the application provides.

Haven’t tried RobotReplay yet, but I imagine it could, given a thorough examination of user’s paths, give significant insight into the websites you develop. But it is a project that’s inovative in the way it solves the problem of porting an entire platform’s functionality (okay, really a subset) inside a limited environment, in this case, a browser. Well done, guys. I’ll be keeping an eye on this one.

1 Comment

  1. I agree, privacy is definitely an issue with RobotReplay. However, I see RobotReplay as a great usability tool while a web project is in development. It provides the ability to include remote users.

    While viewing users firsthand and hearing their comments as they interact with a website is ideal, RobotReplay is an excellent option for learning more about the user experience of a developing website or application.

    Comment by Deborah — November 10, 2007 @ 11:27 pm

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress