Methods Used


Technologies Used




PDF iconDownload the poster

Home > Portfolio > Inherent Value Test

Remote Inherent Value Testing of CouchSurfing.com

An ambitious effort to interview CouchSurfing users from around the world to find out the differences between new users' and experienced users' use of the site.

Date: May-August 2008.
Collaborators: Sebastian LeTuan, Wendy D., Saisuda Suebsuan
Client: CouchSurfing.com

My Role: I came up with the idea of doing an inherent value test, wrote the test protocol, created the recruitment screener survey, figured out the remote testing software setup, and conducted 7 of the 15 test sessions. (My teammates conducted the rest of the tests, and did most of the analysis & report writing.)

The Challenge


Collaborate with the other 3 volunteer members of the CouchSurfing UX Research Team (located remotely in Thailand, the Netherlands, and San Francisco) to decide which method of user testing is of greatest benefit to CouchSurfing, then plan and conduct the test in 3 months.

Process


At the beginning of the summer, the four of us used Skype to have conference call planning sessions across 4 time zones. We agreed that because CouchSurfing knew so little about its users, the best research strategy would be to start broadly and learn what we could about the behaviors and attitudes of users, rather than just test for usability problems. I came up with the idea of doing an inherent value test after reading about it on Jared Spool's blog. Inherent value tests are a variant of usability testing that are used to find if new users understand the value or benefits of the site differently from experienced users. This type of test fit well with the needs of CouchSurfing, as they were having some issues with integrating new users after experiencing record growth.


World map showing locations of all UX team members and participants

We recruited participants through a news announcement on the site which linked to a screener survey. From the more than one hundred responses we received, we chose a representative sample of 15 users (5 newbies, 5 active members, and 5 very active members) to be our participants. We scheduled each participant for an hour-long session, which we conducted remotely over the span of three weeks using a setup consisting of TechSmith's UserVue, Skype, and a free conference call service that made international calls free for the participant. Using UserVue, we were able to have members of the team and developers from CouchSurfing watch the sessions remotely, which was greatly beneficial.

Our participants came from 15 countries, including Brazil, South Korea, Turkey, Serbia, France, Italy, Germany, Canada, and Mexico. One lesson we soon learned was how greatly Internet connection strength varies around the world!

Results


After finishing all the sessions, we analyzed the results and created a report for the CouchSurfing Leadership team, which my collaborator Saisuda presented in person (right). Our major finding was that many new users were not aware of the experienced users' favorite things about the site, such as the groups and events. So, we recommended ways to help new members discover these features. We also found out which parts of the site were most valuable to the active users, which allowed us to recommend development priorities. Finally, this sort of testing produced a wealth of information that helped to answer some of the burning questions that CouchSurfing had had about its users, and revealed some common usability problems that could be easily addressed.