Thoughts on listening to users

« Older posts
Newer posts »

Impression Testing

By admin | Published: April 18, 2011

The five-second test (clueapp, fivesecondtest) is a very simple method for capturing a user’s first impressions about the website.  We did a few experiments with this test on various websites (some ours, some not).  In our experiments, five seconds was really not much time to register a value proposition – it was just enough for the brain to register images and perhaps a headline or two.  It seems that if you want to know if the user gets the gist of the value proposition, he needs a bit more time.

Last week we rolled out a similar feature on TryMyUi.  When commissioning a use test, you can specify if you want to include an impression test.  If you check yes, then we precede the actual test with a short test, in which the user is shown the website for about 10 seconds (it’s rough because it may take the website a second or two to load), and then we ask them three basic questions:

  • What is the website about?
  • What services are offered and for whom?
  • What is the feel of this site? (e.g. professional, fun, small-company, corporate?)

Basically, with the impression test you get a quick review of the user’s impressions of the landing page.  It doesn’t cost any more – try it out and give us your feedback. What would make this more useful?

Posted in TryMyUi | 3 Comments

Create High-Fidelity Wireframes for UX Testing with Justinmind

By Farhad Farzaneh | Published: April 14, 2011

Testing Prototype with TryMyUI JustinmindLast month, we announced a new feature in TryMyUI that optimizes user feedback for prototypes and wireframes.  Wireframes traditionally lack many of the design and visual elements that real people outside your project team rely on to make sense of a website or application.  This poses a special challenge when it comes to usability testing on wireframes.  If users have the wrong expectations during usability testing, they may be confused and unsure about how to respond to what they are seeing.  That’s why TryMyUI allows you to specify if the test will be done on a prototype, wireframe or live site.  TryMyUI then provides extra guidance behind the scene to testers to ensure that you receive quality feedback.

But did you know there’s a wireframing tool designed to build fully functional dynamic prototypes without any coding?  Meet Justinmind Prototyper.  Using Justinmind Prototyper and TryMyUI, you have at your fingertips a powerful set of DIY testing tools that enable you to take full advantage of the benefits of first design iterations using a high-fidelity wireframing software ready for user experience testing within TryMyUI.com’s remote usability testing tool.

Wireframes and mockups have been used to think and define websites or web applications before their implementation. But with the coming of the Web 2.0, dynamic websites or RIA, you need to think everything from UI to conditional navigation or the type of interactions you want to use.  Mockups or even clickable wireframes, however, may not be enough.  Justinmind Prototyper lets you create Rich Interactive Application prototypes with simple drag and drop plus have access to a range of reusable templates.  You can also integrate real data and simulate their behaviors through datamasters and datagrids. Ultimately, it simulates real conditional navigation, like functional login pages.

Here’s an example of a TryMyUI’s tester giving “think aloud” feedback on a  prototype created using Justinmind.

Justinmind has got you covered on the front end by enabling you to quickly create high-fidelity prototypes and wireframes.  And TryMyUI’s got your back covered by setting expectations and communicating with users the purpose of prototypes and wireframes.  Since we’ve got you fully covered, go out there and get the insights from putting your earliest designs in front of users for testing.

Posted in Partners, Practice | Leave a comment

5 Tips on Designing a Better Event Website

By Sani El-Fishawy | Published: April 13, 2011

Create Better Event Websites with TryMyUICass Phillipps is a Bay Area event organizer extraordinaire.  If you’re a start-up or entrepreneur, you’ve likely attended an event where she was the behind-the-scene force to make it all come together:  FailCon, SF MusicTech Summit, Inside Social Apps to name a few.    One of the reasons why Cass is so good is because she is constantly testing her own assumptions about how things work and figuring out if there’s a better approach.   She recently used TryMyUI to see if she can design better event websites, since this is her bread-and-butter.  For example, should you or should you not post the event ticket price on the home page? Check out her guest post on Eventbrite Blog and see how she used TryMyUI to glean 5 tips on designing effective event websites.

Posted in Entrepreneurship, Practice, TryMyUi | Leave a comment

Prototypes, Wireframes and Live Site — Oh My!

By Sani El-Fishawy | Published: March 30, 2011

That’s right!  We rolled out some cool new features.  Check them out, and let us know what you think.

Prototypes & Wireframes
When creating a new test in TryMyUI, you can now specify if the test is for a live site, prototype or wireframe.  We will instruct the users accordingly to set their expectations.   This is important to guide testers for quality feedback, since not everything will be functional on prototypes and wireframes.

Here’s what you’ll see when you create your test:

Step-by-Step Task Presentation to Users
To help ensure testers don’t miss any tasks that you have requested, TryMyUI now presents tasks to users one by one.  Completeness and efficiency make your job even easier.  We want you to be happy with the video results from TryMyUI.

Stay tuned, as we have more new features to announce in the coming weeks.

Posted in Practice, TryMyUi | Leave a comment

Using TryMyUI and Loop11 in Tandem to Obtain Quantitative and Qualitative Measures

By Sani El-Fishawy | Published: March 27, 2011

Improving website usability is consistently a challenge for companies – whether a startup or a large enterprise.  There’s a plethora of information and healthy debate on what user experience research methods should be used and when.   When it comes to quantitative versus qualitative research, best practices suggest a mix of qualitative and quantitative approaches to effectively capture a more holistic perspective.  A challenge, however, is how to combine qualitative and quantitative testing in a cost-effective, productive way to rapidly obtain data to drive user-centered design.   That’s where TryMyUI and Loop11 partnership comes in.  When TryMyUI and Loop11 are used in tandem, designers, researchers and marketers have a powerful set of tools to optimize site and user experience. 

What’s the difference?

The difference between quantitative and qualitative approach is often explained using contrasting terminology, like “hard vs. soft”, “numeric vs. non-numeric”, “statistics vs. insights”, “measure vs. explore”, “what vs. why”.  The quick contrasts are useful to highlight both the strengths and limitations of each approach if used solely without consideration of the other.

Broadly, quantitative research can provide path and performance analyses by capturing the “what” of user behavior.  Loop11 is a remote usability testing tool that provides quantitative measures of real users’ behavior through clickstream analysis.  Loop11 delivers quantitative metrics such as task completion rates, number of clicks, time on tasks, and detailed path analysis.   To get started, researchers use Loop11 to create tasks or questions that are then presented to real users.  Loop11 supports unlimited tasks and questions, and as many as 1,000 users can participate.  Data generated from users’ clickstream are automatically presented in real-time reports.

The quantitative measures from Loop11 is especially useful if researchers want to

  • identify any usability problems
  • measure task efficiency and success
  • compare against competitor usability metrics

In contrast, a goal of qualitative research is to gain valuable insight into the thought processes behind the user’s actions or clickstream.   TryMyUI is a remote usability testing tool that captures the “voice of the customer” via a video recording and a written summary documented by the user.   As users navigate a website, they “think aloud”, verbalizing their thoughts as they complete the tasks or questions posed by the researcher.   For example, a user may express surprise to find the “register” button in a particular location on the site.  Quantitative research tracks that the user clicked “register” and what the user clicked on before and after.  For researchers, the additional insight that the user was “surprised” augments the analysis of the user’s behavior, and the researcher observes this first-hand by viewing the narrated video delivered by TryMyUI.

What you can learn from the qualitative measures from TryMyUI:

  • how does the user experience match the user’s natural expectations about how things work
  • why is the user experiencing this with in-context feedback

What’s the linkage?

Loop11 provides the quantitative measures through metrics and reports, and TryMyUI delivers the qualitative insights via narrated videos and written answers to survey questions.   With these two easy-to-use, on-demand testing tools, any researcher or marketer can simultaneously gain insight into both the “what” and the “why” of user behavior.

Although there is no single usability tool that provides this powerful combination, the combined functionality can be replicated by initiating a project in Loop11 and then augmenting the testing process in TryMyUI.  To get started, open an account on TryMyUI.com and Loop11.com.  Both sites offer a free trial for the first project to help get started and become familiar with the process and results.   Once the accounts are created, the process is straightforward:

1)       At Loop11, create the test scenarios, invite user participants, and analyze metrics and reports

2)       At TryMyUI, copy the project URL generated upon launching your Loop11 project.  In “Create a New Test”, paste the Loop11 project URL in “Web Site Address”.   There’s no need to re-enter the test scenario and tasks in “Scenario” and “Task” sections, as that information is already captured in the Loop11 project URL.

3)       Use the default “Survey Questions” or customize as needed.  Select the demographics.  Select number of testers.

4)       TryMyUI handles the rest, and will deliver videos of each test user navigating the website with the “think aloud” narration.   Users will also provide written answers to “Survey Questions”, for example, “what did you like about the website.”

That’s it!  Let us know how this works for you.

Posted in Practice | Leave a comment

Customer Spotlight: Andy Shield – Director of User Experience, Red Bullet UK

By Sani El-Fishawy | Published: February 17, 2011

We learn so much from our customers, and wanted to pass on the goodness. We recently checked in with Andy Shield, founder of Red Bullet, a research-led web development agency based in the UK.   Andy lives and breathes usability research, and uses OpenHallway (TryMyUI’s sister company).  Thanks for your perspective, Andy!

What’s the best part about your job? Insight.  I’ve always found usability tests fascinating and the insight we’ve gained over the last few years has helped to drive quality throughout our entire design and development process.  It’s reassuring to know that we can design for the masses rather than an elite group of expert users.

What prompted you to use OpenHallway and how has OpenHallway helped? I became aware of OpenHallway through the industry. There are plenty of competing products but I found most of them to be unreliable and buggy. It’s also slightly ironic that quite a few of them have usability issues…  OpenHallway is simple to use and quick to administer which means I can direct more of my grey matter towards interpreting the results.

What tips would you give to new users of OpenHallway? Start small by running some dummy tests with people you know. The most important element of a usability test is interpreting the results in a meaningful way. Hence, by exploring how to create tasks which produce insightful results, you can then approach a full scale test with a greater understanding of what you are trying to achieve.

In terms of the application itself, it’s self-explanatory, and it’ll only take a couple of dry runs to be fully up to speed.

What’s one of the key challenges of managing usability testing and what advice would you give? The challenges depend on the type of test you’re undertaking. For example, the greatest challenge for a moderated usability test is the logistics. Expect people to be late and in some cases not turn up at all. Always recruit more people than you need and have a short-notice reserve list. It pays to plan properly and think through possible issues well in advance. For example, we were commissioned to undertake a study for a client during the 2010 World Cup; hence we made sure that we didn’t arrange any of the tests on a match day.

For remote usability tests, remember to send out a very clear set of instructions as the users are taking the test without any handholding.

Whether the tests are moderated or remote, always offer a decent incentive and keep them short as people lose interest (no more than 45 minutes).

Also, remember to tell the participants (face to face or by email) that you’re not testing them, you’re testing the website. This encourages them to provide useful feedback as often people feel that they are to blame when an issue occurs.

Any suggested good read (blog, books, etc.) on usability testing you’d recommend? There are plenty of industry blogs on the subject, but I would highly recommend Steve Krug’s book “Rocket Surgery Made Easy” which is an essential read for anyone involved in usability testing.  It may also help to gain an understanding of human factors by reading “What Makes Them Click” by Susan M. Weinschenk.

Find Andy on Twitter @andyshield or www.redbullet.co.uk

Posted in Customers, Practice | 1 Comment

TryMyUI and OpenHallway give away $140,000 of usability testing in sponsorship if UXLX conference

By Sani El-Fishawy | Published: January 31, 2011

The remote usertesting services TryMyUI and OpenHallway are sponsoring the UX-LX conference and are giving away $140,000 worth of remote usability testing to approximately 500 conference attendees. OpenHallway is donating 1 year of service valued at $228 and TryMyUI is giving away two remote user tests valued at $50 for each attendee. Attendees can claim their free usertesting services here.

Posted in Uncategorized | 1 Comment

TryMyUI and OpenHallway are proud to sponsor UX-LX in Lisbon this summer

By Sani El-Fishawy | Published: January 31, 2011

If you interested in usability and can make it to Lisbon this summer we highly recommend you attend the phenomenal UX-LX three day usability conference. This conference has something for everyone in the user experience field. The event will feature workshops in three tracks: Research, Design, and Strategy; as well as talks by such experts as Don Norman, Louis Rosenfeld and Jeffrey Veen. You can check out the full line up of speakers and workshops and purchase tickets at the UX-LX website. The remote usertesting services, OpenHallway and TryMyUI, are proud to be Silver Sponsors of the event this year. We are offering all attendees free testing on TryMyUI and OpenHallway as well as a chance to win an iPod Nano.  If you are attending the conference you can claim the OpenHallway-TryMyUI-UX-LX offer here.

Posted in Uncategorized | Leave a comment

Compelled Shopping Test

By Sani El-Fishawy | Published: November 19, 2010

At Bolt Peter’s User Research Friday conference today Jared Spool talked about a test I had never heard of before. He called it the Compelled Shopping Test.

When I first heard the premise I must admit it made absolutely no sense to me!

Here is how it works…

  1. You recruit test users for testing a shopping site who are carefully screened to really and truly be in need of buying particular item at that moment (for example they are in the process of replacing their laptop computer).
  2. You give them 2 weeks to do all the research they need to figure out what they want to buy.
  3. On the day of the test all they have to do is buy the product on the site in front of you.
  4. You pay for their purchase as part of the study!

What? You buy the product for them? How could you possibly get further from reality!

But later on in the talk Jared gave the rationale for the Compelled Shopping Test in the form of a story that made a lot of sense.

I will call it the 7-11 parable.

What if you had a device that would tell you in real time who had just run out of milk and was just heading out the door to go to 7-11 to pick some up?  You pull up in a limo in real time and offer her a ride to 7-11 and also offer to pay for the milk. What would the conversion be in this scenario. 100% right? Only a drive-by shooting at the convenience store would reduce that conversion.

The purpose of the Compelled Shopping Test is to eliminate the myriad of reasons why people don’t purchase (not ready to buy, not sure what they want, planning on buying it elsewhere, etc) and leave nothing but a brilliantly distilled scenario which should have 100% conversion. Then the gap between actual conversion and the 100% you would expect with the CST scenario is really due to broken stuff on the shopping site in question.

What blew me away is that Jared said that the best conversion rate they have seen in a compelled shopping test is only 66%. Kind of unbelievable if you think about it: 1 out of 3 people who are given the money to buy something they want to buy don’t do it in the end because of something broken on the site. And that is for the best sites!

Here is UIE’s description of the 7-11 experiment.

Posted in Uncategorized | Leave a comment

Dana Chisnell at B|P User Research Friday – Research in a Social Context

By Sani El-Fishawy | Published: November 19, 2010

Usability Testing in a Social Context

Dana Chisnell gave a great talk today at Bolt Peters User Research Friday. Her observation was that almost all activities on the web occur in a social context that spills out far beyond the particular product being tested.

Studies that don’t account for the social context are broken

Dana put together a study for a financial company that was providing retirement planning for users. When she put her panel together she realized that it was broken! Specifically she realized that the decisions users make in regard to their retirement planning are made with the help of family members and other people. Any testing that put users in an isolated environment would necessarily be a bad proxy for reality.

The right way in retrospect to conduct this study would have been to ask the recruits who they relied on for financial decision and to invite those members to the sessions.

The Nature of Online is Social.

Social means anything that someone does that affects the behavior of someone else. It involves relationships.

Scale is a game changer.

Task doesnt mean what you think it means.

Specific isolated tasks in scenarios are not reality. What we need to be thinking about is the high level generalized intent across a broad social network.

Satisfaction may now equal user control.

Users are continuously designing your UI

Social is there already

Social is not a sauce that you add to your design. It is already there. You need to find it. You need to properly accommodate when you do find it. And you can’t test social experiences with testers in isolation.

How do we do research on social products?

  • Spend a lot of time with users ahead of time. Giving them homework.
  • Get recruits to show you their relationships.
  • Get recruits to tell you how they met the people in their social networks.
  • Try to understand the social context that the proposition presented by the product will involve even if the product does not yet well accommodate that social context.

Usability research is more than finding bugs in flows

Usability testing is about determining design direction by getting perspective on the broad context in which their user acts. It is about forming cogent hypotheses about how products fit within real people’s lives.

Commodity usability testing is too controlled and too contained.

Posted in Uncategorized | Leave a comment
« Older posts
Newer posts »
  • Pages

    • About
  • Categories

    • Customers
    • Entrepreneurship
    • Gallery
    • Musings
    • Partners
    • Practice
    • TryMyUi
    • Uncategorized
  • Blogroll

    • TryMyUi
  • RSS Links

    • All posts
    • All comments
  • Meta

    • Log in