Web Advisory Group
Usability Testing: Discussion Notes
The first demo
of the usability process
for MIT web authors was held Feb. 23, 1999, in the Dewey
Electronic Classroom. In addition to the members of the Web
Advisory Group, the attendees were: Diane d'Almeida, Michael
Leininger, Poping Lin, Margo Miller, Katy Poole, Nataly Reed, and
Ruth Seidman. Following are notes of the discussion which followed
the demo. (Another demo is scheduled for Feb. 24 in the same
place.)
Nicole gave the demo, which is based on her former work at Bose. In answer to questions, she said:
- Observers should be strictly observing and not commenting on the tester's methods or conversing with her/him.
- Most testers will know what a search tool is (as distinct from the website itself), but we should make sure that is the case.
- The Bose site is more or less the same size as the MIT Libraries site -- perhaps slightly larger because Bose has more departments and more subpages.
- In the second test of the improved Libraries site, there may be some of the same people who are tested, but there should likely be a whole new suite of questions.
1. Who should we test?
2. If we test 25-30 people, how many of each group should we test?
There was general agreement on the following rough numbers: equal numbers of graduate and undergraduate students; 1 or 2 library staff members (perhaps new staff); 1-3 faculty members; some of the assistants who help faculty members in web research; 1-2 administrators; 1-2 users with disabilities.
The general feeling was that the test should involve people in the MIT community: no outside users at all. This test will also be only for the public web: a test of the staff web will be done separately later.
3. How should we recruit the volunteer testers?
A couple of methods were suggested:
- Personal contacts (including students and student workers)
- Faculty Committee on the Library System
Stephanie agreed to receive email from people with suggestions of testers. Once she has this list of names, she and Nicole will compile a reasonably sized list and then contact the people to ask them if they would participate. Each test will last a half-hour, and tests will be conducted over a two-week period.
4. Where should we hold the tests?
It was generally agreed that a mix of venues would be the best way to go. The public workstations in the libraries are not a good venue because of the chaotic atmosphere and because there sometimes is a queue at workstations already. Some possibilities suggested were the Libraries Training Room or the tester's office. The responsibility for the selection of the venue should be left to the individual observer.
5. Web Advisory Group members will write the questions. We will try to include questions from many different sections of our site. We may write two sets of questions in order to cover a lot of pages. Are there any particular pages or departments to include? Or avoid?
The general goal is that questions should be phrased so that testers do in fact get to the pages we want to test. The purpose of the questions is help us gather information about the usability of the site as it is currently designed, not to be a means of retraining users in their web research. We should emphasize to testers that they should not be looking at sites outside of MIT (we want to test the MIT pages) and should not be using their personal bookmarks to find information (one of the ideas is to find out how long it takes people to get to those pages that they have bookmarked).
The following specific suggestions and questions (in italics) were offered:
- A question which gets at whether users are distracted by the different layouts and "feels" of the various libraries' pages. For example, does a user have difficulty finding the same type of information (e.g. hours) on different pages? Example: How late is Dewey open?
- A question which tests the Services page, e.g. ILL. Examples: How do I get a book that MIT does not hold? Where do I find out whether BU has this particular book? Also: Where can I find information on American authors? Where can I find company information? Where do I get access to this particular database?
- Test the Virtual Reference Collection. Examples: Find a page that links to a zipcode directory? Where can I find a style manual? Where can I find an online dictionary?
- Test the Data Resources page. Example: How do I find a list of links to housing statistics?
- Ask a question which gets testers to use the libraries' search tool.
6. Any other questions or comments?
No.
http://libstaff.mit.edu/webgroup/usability/questions.htm
Last updated Feb. 23, 1999, by Wayne Jones