Test 2
Usability test results, April - May 2001Test 2
April 25 - May 9, 2001
This test continues our plan of doing more frequent, less formal tests, focusing on finding and fixing specific usability problems.who we tested:
• 5 faculty
• 1 grad students
(we planned to test 4 faculty and 4 students, but not all of the volunteers were available)
what we focused on:
• some of the less obvious information on our site, subect pages, and individual library pages
what we looked for:
• whether our terminology and wording makes sense to people
• whether our individual library home pages are easy to use
• whether people find the subject pages useful
the test:
• each test was about 1/2 hour long
• volunteers were asked to find the answer to 10 questions
• they were asked to "think out loud"
• we asked a few post-test questions (for the subjective opinions)
• instructions for observersALL QUESTIONS START AT THE HOMEPAGE, EXCEPT FOR 4, 5, AND 6, WHICH START FROM THE SUBJECTS + COURSES PAGE
the results
Questions:
The first 3 questions focus on Barker's site, since we wanted to test an individual library page, and Barker was one of the few sites that was completed at the time of conducting the test.
1. What is the mailing address of the Barker Engineering Library?
3 people found it and 3 didn't find it.
Our interpretation: Here we were asking them to find something that we've purposely de-emphasized. We made the decision to put the mailing (i.e. postal) address on the "contact us" page for each library so that the other, more important information (like building number, hours, and phone numbers) would stand out more on the top of the page.
After we thought about it we realized that this question would have been more appropriate for a test of outside users, since they are the ones who would be more likely to come looking for a mailing address. Our own users are more interested in where we are on campus, and our email and phone information.
Since we not only have the mailing address on the "contact us" page for each library, but also in the footer of every single page on the site, we decided not to change this, since adding it to the top would make the information that we do want to emphasize less easy to find.2. Make an appointment with an engineering librarian for help with your research.
4 people found the Research Consultation form with no problem. 1 person found "ask us" instead and 1 person used the Barker staff list to find the email addresses of their librarians instead.Our interpretation: We decided that using the Barker staff list to find an email address was certainly a fine way to do this task and that it's good that we have more than one way to contact people.
Another person went to "ask us" instead. This was a graduate student whose first language was Japanese. He had difficulty with many of the questions and even after explanation, didn't seem to understand the difference between making an appointment with a librarian and just asking a question via "ask us."
We decided that since most people had no trouble finding this that there wasn't a specific problem we needed to fix.3. On what floor is the Barker Library group study space located?
Everyone found this one.
Our interpretation: No problem here.
ALL QUESTIONS START AT THE HOMEPAGE, EXCEPT FOR 4, 5, AND 6, WHICH START FROM THE SUBJECTS + COURSES PAGE
4. Find a list of new books about neuroscience.
Everyone found this one easily, though the grad student mentioned above took longer to find it. He tried Barton first.Our interpretation: No problem. We have links from almost every subject guide to new book lists and we have links in "About Us: new items in our collection" and we also have a link to "new items in our collection" on the "Search Our Collection" pages. One suggestion we have is to add a link from the Barton screens somewhere, if and when we can get Barton to autogenerate our new book lists for us. For now our manually created lists are easy to find.
5. (START FROM THE SUBJECTS + COURSES PAGE) Pick a page in a field with which you are familiar. What links on this page are most useful to you?
We tried something new this time with this question and the one following it. Since in our previous tests people told us that they had no idea we had Subject Guides, we wanted to show them to people and find out if they thought they were useful. So we had them begin on the "subjects + courses" page - (which is a list of all our subject guides) for this question and the next question. (also the previous question about new books in neuroscience).Strictly speaking this kind of open-ended question is not a "usability" question, but we did look to see if they had any problems navigating the subject pages themselves. No problems were noticed, these pages seemed easy to navigate.
Two faculty (since they were from Sloan) noted the lack of a specific "management" subject guide. And one looked for an "economics" subject guide which wasn't there. The grad student said he would never have thought of looking on our site for these - he would use Google. (we've heard that before!)
Our interpretation: People didn't really have time in a test like this to thoroughly examine the subject pages. However, we were happy to hear the most faculty were pleasantly surprised at the depth of information on the pages they looked at. At first, they were doubtful and wondered if the library really needs to be "recreating portals", but after looking around a bit, they made some positive comments and found the pages to be useful. Chemistry, Civil Engineering, Literature and Political Science all received postive comments. (these were the fields that were the specialities of people we tested)
Our main conclusion is that these pages are valuable and easy to use, so we need to do more publicity about the Subject Guides - perhaps next Fall.
6. Pick a page in a field with which you are _not_ familiar. What links on this page are most useful to you?
Same comments as above.Our interpretation: People looked at Architecture, Art, Physics, Math, and Women's Studies. Architecture and Women's Studies received some especially positive comments.
7. How would you get an item delivered from storage?
4 people found it quickly and 2 people found it after looking around a bit.Our interpretation: The 2 people who took a while went to Barton first. This is of course very reasonable and the link to RSC will be there when 3rd Barton is implemented. The others found it in "borrowing + ordering," or in the "quick links" menu, or from RSC's home page. So there is no problem with this one.
8. How do I find a German patent?
5 people found it (one by using "ask us! - live" for assistance!), and 1 person didn't find it.Our interpretation: The majority of our users found it, so this one seems OK for now. The name "publication types" is still a bit of a problem, but we are at a loss as to a better name for these guides. That is somewhat overcome by the fact that we've linked to many of these guides from several places. A link to this Patents guide is in "search our collections" and another link to "publication types" is under "Research Help".
The professor who used "ask us! - live" just loved it, by the way!
9. Where would you find general information about course reserves?
5 of 6 people found it.Our interpretation: The student who didn't find it looked in the Circulation FAQ. There is actually a link to the reserves page there (in addition to the questions and answers about reserves), but it's buried in the middle of a paragraph, so he didnt' see it). So we added a more prominent link to the general reserves page at the beginning of the Reserves section of the Circulation FAQ. We also plan to link to the "course reserves" section of Barton when 3rd Barton is ready. (from this page and the general reserves pages) This is where most of the real content is, i.e., what materials are on reserve. What we have right now is mostly policies about reserves.
10. Find a list of institutions which are members of the Boston Library Consortium.
5 of 6 people found it.Our interpretation: Everyone (including the one who didn't find it) started by looking at the Interlibrary Borrowing page. So we decided to add a link to the "access to other libraries" page near the top of the ILB page.
Post-test questions:
Our interpretation: All the usability studies say that it's not so important what people say about your site, it's what you see them doing when watching them find things. In spite of that, we were curious about people's likes and dislikes. Keep in mind that this is not a sample population (as in a formal survey), it's just a few comments by 8 of our users. We were quite happy to hear their comments.
1. Do you have any suggestions for improvements to the site?
- put link to professors' course pages above link to librarian-created course pages, since there are so few of these
- doesn't see why the course pages by librarians should be separate from those by professors
- would like that Jake were better integrated with Vera: for example, when he searches a title in Vera that is not there, then the "no results found" page should provide a link to Jake; would also like to see records for Jake titles in Vera -- at least the ones for which we also hold the print version
- mentioned in passing that he doesn't like Web Barton much -- that he normally uses telnet; said that the keyword search in the former does not work well, but it turned out that that problem is that Web Barton defaults to a browse search, when he had thought he was doing keyword -- did not notice the small "keyword" link on the left
- make home page bigger
- put everything in pull-down menus instead of having "more" link
- make Vera and Barton stand out more (Japanese student liked old home page since he didn't have to read as much text - just looked for Vera and Barton links at top)
- "nothing needs improving" - "very well organized"Our interpretation:
- we could do more to integrate our course pages with those by professors, perhaps ask them to link to our pages from their pages for the same course and vice-versa
- the suggestions about Jake may be solved by doing the "Serials Solution" pages that we've been talking about doing, (see recent NERD discussions), also we plan to make more links from Vera to ways to get journals if they aren't available in Vera, such as links to ILB, Web-Docs, & Serials Solution pages
- the Barton problem was also noted when we did the Barton usability tests - (these results haven't been put on the web yet), we are addressing this in the design of 3rd Barton screens2. What do you like best about the site?
- really likes the new 5-group classification - works much better than the old way where she couldn't find much
- Vera and Barton are prominently displayed and that's good
- likes the look of it
- the "quick links" menu
- "it's easier to find stuff"Our interpretation:
- we are happy to hear that faculty find the new site useful!
3. Which parts of the site are most useful to you?
- literature subject guide
- "ask us" and Barton
- Vera and Barton
- Civil Engineering subject guide
- the tables on "Research Help" and "Borrowing + Ordering"4. About how often have you used the libraries¹ web site in the past?
___ never used (0)
___ only a few times (0)
___ a few times a year (0)
___ about once or twice a month (4 users)
___ every week (0)
___ every day (2 users)5. Is English your first language? If not, what is your first language?
- English - 4 people
- (not English, but didn't say what language and anyway English was extremely fluent as her second language)
- 1 Japanese
Future tests will focus on different audience groups: users with disabilites, alumni, administrative staff, outside users.
We will also do more tests of Vera.