"Big Test" usability test
Problems & Possible Solutions

Test A | Test B | "Ask Us—Live" Scenario

Test A -	Observations	Possible solutions
Question 1 Barton catalog	people don't understand citations, they search for the article titles and authors of articles in Barton	We had planned the Info Navigator originally as a short page about how to find articles that we could link to from various places, but it grew into a much more comprehensive resource. We still need a brief page with maybe 3 steps that simply tells people how to find articles. This should be linked to from Barton and Vera and other appropriate places.
	[link to "finding articles" section of InfoNavigator on home page] [InfoNavigator screen shot] (will open in new window)	The InfoNavigator is too wordy - (users commented that they didn't want to read through it). Would be good to make it shorter.
		Ideas for InfoNav: have a table on first page with links to longer descriptions, make screen shot of a sample citation with big red arrows pointing to which parts to look up in the catalog (journal title), etc.
	people didn't tend to use the journal search, had trouble figuring out which ones were journals [Barton home screen shot] (will open in new window)	Would be great to direct more people to the journal search in Barton. Ideas: we suspect people will have banner blindness for the links at the top (we looked at mockup of no frames version). Could we put something in the body, like: Journal Search (as a hyperlink) right above the search box - it would take them to the journals-only search screen. But we then wonder if there should be links in the body to all the other specific searches: conferences, etc. and would that make it too cluttered? We didn't resolve anything - needs further discussion from the Barton OPAC design team.
	people don't understand the word "serial" [Barton results screen shot] (will open in new window)	In Barton records when scanning through brief display of keyword results, people see "Format: Serial" and they don't know what that is. We don't have an answer for this one because it's more than just journals. Part of it would be helped if we got them to the journal search up front, so they wouldn't have to try to distinguish between different types of records. But still -- is there a better word than "serial" that people understand? Do we need a link that says "what is a serial?" (hope not!)
	clicked on "more info" - but it didn't help [Barton "more info" screen shot] (will open in new window) [Barton "more info" results screen shot] (will open in new window)	This link leads to information that is useless to the vast majority of people (such as barcode, etc.) Could we get rid of the link? Do we really need it for our staff? If we do need it, let's change the name of it, perhaps to "staff info" or "barcode" - something that would be less enticing, so people know there's nothing useful there.
	people think that search box on bottom of barton screen of full record means they can search within journal [Barton search box on bottom screen shot] (will open in new window)	Would be good to add a bold text label just above this box that says something like: "New Search."
	find articles link to info navigator - they expect a search box [link to "finding articles" section of InfoNavigator on home page]	Change link to read: "How to find articles."
	trouble finding availability of a volume
	check availability, but not all vols are cataloged, so they think we don't have it [Barton availability screen shot] (will open in new window)	On the availability screen add a message that says: "Some volumes may not display here. Check summary holdings to see complete range of volumes owned by MIT." Summary holdings - could those words be a hyperlink back up to the summary holdings at the top of the page?
	doc services delivery link - don't understand [link to copy options screen shot] (will open in new window) [Copy Options page screen shot] (will open in new window)	There are two links, one on availability screen that says "copy options" and one on left sidebar that says "copy services." These should be the same and they could be renamed to "Get a Copy." They should point to the intermediate page that explains all the options (free and paid).
	links to order forms on left get lost [links to order forms screen shot] (will open in new window)	this will be fixed in the no frames Barton (check to make sure)
	some users don't like keyword default in Barton	already discussed - [in previous usability tests we decided keyword should be default - better to give too much than have them think we don't own something because they didn't search correctly, with too much, they tend to try another search to narrow it]
	keyword search gives too many results, people have trouble finding journals	already discussed
	they look in body of record instead of links on left	will be fixed in no frames: more body visible, navigation links will be on top of screen instead of left
	confusion between article being delivered electronically and being available online, i.e., PDF sent by email vs. using an online e-journal	(see copy options above)
question 2 Barton catalog	people clicked on "internet resource" instead of URL [Barton brief results screen shot] (will open in new window)	fix by only making "all items" clickable, not "internet resource" - should just be plain text, double check to make sure that URL will also be on the availability screen, consider changing lable to say "Location", and "all items" to say "check availability" - follow up on "get URLs" in Barton, can we do it?
	even a savvy user had no idea they could get electronic version from Barton [home page screen shot] (will open in new window)	above changes will help Bigger question - how do we tell users where to start? When to use Barton or Vera?
	user would like to put hold on an item from the availability screen [Barton availability screen shot] (will open in new window)	will be fixed in no frames (check to make sure)
question 3 ILB form	ilb form has link to webdocs, user confused between webdocs and ILB [ILB form screen shot] (will open in new window)	OK for now
question 4 Barton catalog	some people still use "the" at beginning of title. Barton search is not ignoring the "the" and it looks like it's OK to use "the" because some records have the "the" there. [Barton search screen shot] (will open in new window) [Barton results screen shot] (will open in new window)	recommend cleaning up titles in OPAC that begin with "the" and similar first articles (Stephen will bring to catsub)
BookPage	Barton link says only "BookPage." Our quicklinks menu says "Bookpage - delivery" - that's a better name, more descriptive [BookPage link screen shot] (will open in new window) [Quicklinks menu screen shot] (will open in new window)	call it "Bookpage Delivery" in Barton
	people don't know we have a bookpage service	there might be a problem in Barton where you can't restrict people from doing it within one library, so let's do nothing for now about publicizing it more
	there are other things that seem like they could be bookpage - your account, your bookshelf, etc. [Barton top nav-bar screen shot] (will open in new window)	suggest changing "your bookshelf" to "saved items", then also change button to "save items", also fix error message that appears if you don't mark anything
question 5 Barton catalog + RSC	RSC: "restrospective collection" - they thought it was a collection and asked "what library is it in", some people ask "where is the RSC section?" "retrospective collection" is a bad name [Barton full record screen shot] (will open in new window)	we should call it "storage facility", or "remote storage center"
question 6 Pub Types	no specific comments
question 7 Pub Types	easy to find because of question 6
	they don't think of guides as general	suggest to each selector to put their subject guide in Vera, if it makes sense to do it for that subject
question 8 Home Page	one person thought "Dspace - MIT research" means they are doing research at MIT, need to distinguish it from the libraries' collection [home page screen shot] (will open in new window)	on our home page, Dspace needs better name, for example: MIT's Research Repository: Dspace, make quicklinks match this
Research Help	most testers said they would come to the library in person, but these are people who are already using the library	later did test with 5 non-frequent library users: 3 of the 5 said they would use "help yourself", 1 said would use phone, 1 would come to reference desk in person
Test B	Observations	Possible solutions
question 1 Vera	people use databases they are familiar with whether or not they are appropriate	consider adding another field in Vera for type of material, such as articles, data, ebooks, statistics, data sets (if there is a way that's not too time-consuming, and if we can come up with a managable list of categories), and/or consider an advanced search screen in Vera with these options
	"what's a database?" - people know, but they don't call them databases	user education
	some people skipped the vera browse, went to keyword search	consider putting subject on different screen as hyperlinks instead of menu
		consider dividing Vera home page into 2 sections: 1)don't know where to start? (subjects) 2) if you know the name of the journal or database (keyword search)
		consider changing List by Subject to "choose a subject"
		database titles only, e-journal titles only, titles of what? Make it clear - titles of databases or e-journals
	some people didn't know to click the button after highlighting subject in vera	change nothing for now
	some people wanted to select more than one subject at once	consider this addition for an advanced search screen in Vera
	some people couldn't find the button	consider making more prominent graphical button
	people have trouble thinking of broader or narrower terms	add more descriptive terms under each subject on a different page, coordinate with program and department names at MIT, look also at LSCH headings in Barton records
	subjects in vera don't always relate to what people are looking for, esp. interdisciplinary subjects like "gene therapy" or "ethics"	would be nice to have a link that says "try these other biology databases" - keyword would bring up most appropriate databases for biology, add new search capability that lets you search subjects assigned to each database
		search box could search subjects and descriptions
	confusion about full text books (medical ebooks), didn't get that they aren't articles	add e-book designation to description or add it to display near or after title, consider adding e-books as a vera subject, consider making radio button say "databases and e-books"
	only 3 people read descriptions of databases, hard to tell if it goes with one above or below	consider layout improvements so you can tell which description goes with which item, encourage people to make descriptions shorter and more like bulleted lists
	people who use keyword search in vera never get the descriptions	we could use different template that includes descriptions if they select the databases radio button
question 2 databases within Vera	one user stuck - "current protocols" - when you get into the interface you have to select from a group of databases, people don't get that [database screen shot] (will open in new window)	give feedback to vendors
	people sometimes pick a different database than the one they chose (in groups of databases)	give feedback to vendors
	some people feel obligated to fill in every field in an advanced search with lots of boxes [database screen shot] (will open in new window)	user training issue
	some people click wrong search button when there are multiple ones [database screen shot] (will open in new window)	vendor design issue
	people are bad at changing their search strategy	user training issue
	sometimes hard to get back to vera once you're in a database	vendor design issue
question 3 Barton catalog	frames problem when scrolling down in Barton - can't see top	will go away in "no-frames" version
	sometimes link to add to bookshelf isn't there	?
	seeing availability makes then think it's in library - without clicking	limitation of ALEPH
	confusion between availabilty screen and full record screen - what's the difference - why can't it be one?	limitation of ALEPH
	availability screen is confusing - one library or multiple libraries, you want all items on one screen	Change "all items" link. It should say "check availability." We are still discussing what the label should say.
		would be nice if each library is a word that's not linked
question 4 SFX menu in FirstSearch	people don't click on SFX button, they click on text link	include text link wherever possible
	"libraries that own this item" - people clicked on that instead	ask vendors if we can put sfx link higher than competing links, especially in FirstSearch
	people clicked on publisher's web site instead	put our SFX link first if possible
	SFX window that pops up is small, they have to resize or scroll	Austin is working on controlling window size in SFX
	don't know what ILB is	spell out Interlibrary Borrowing in menu
	link to Barton from menu is too small	Austin is working on controlling window size
	hard to know if barton searched for the right thing, they have to wait	fix error screen when no results found in Barton coming from SFX to say do the search manually
	how reliable is the barton link when we don't have the item? [Barton no results screen shot] (will open in new window)	new error screen should say to try search again in different ways
question 5 Subject Guides	some subject guides are useful, some not	that's OK, it's a fact of life for now, revisit priority of subject guides in grand scheme of things
	people ignored sidebar, looked mainly in the body of page	come up with new template, remind CMG to send reps to work with us on new subject guide templates
	people assumed links in sidebar would leave the site	revise sidebar look, see dspace.org site for ideas, include headings, buttons, white space around it, subject name in sidebar, etc.
	new book lists are really old	put on future webgroup agenda to coordinate with Barton reports of new books, auto generate pages
		consider linking book to individual barton records when we get no-frames, really long lists of broad subject is not that useful, make it more specic or include abstracts
	long lists of call numbers and titles of books are confusing	?
	people don't like lists of books, they do like web sites	but they are useful to some people, especially librarians, doesn't hurt to have them, but should include at least some electronic stuff
question 6 Home Page + Research Help	no specific comments
Ask Us - Live Scenario	Observations	Possible solutions
	red arrow is not noticable	librarian could point to something, but it needs to be more obvious, maybe flash it
	frames problems	consider systems using multiple windows instead of frames, or at least test frames with linking to sites that also use frames
		can the system handle chat well separately from pushing pages
	window on left takes up too much space and is full of "useless info", at least have links to what's there, table of contents	would be nice to have full screen option when you're just chatting, but then turns into frames or multiple windows or some other interface when you're pushing pages
	people feel the chat session should be more prominent	OK
	people don't always see the answer when it came up - too small, not prominent enough	use different colors for each person talking
	people confused when window changes, want to have something stay the same	would be useful to have tech info in constant window
	need more cues to show it's not frozen, that something is happening	have a moving clock or a dotted line that moves, like AOL have a message that says other person is typing, some message that comes up every so often: "Librarian is still working on your question"
	should I type or should they? Confusion between who should go when	follow user interface guidelines used by other chat services like AOL and iChat
	who should be controlling the session?	either librarian or patron could click on a link and both browsers would go there, so confusion between who clicks first
		librarian can shut off co-browsing - this is a good feature to be able to toggle it off, also librarian needs good communication skills with chat
	time lag is a problem	some of it is how fast you type, user can go 3 steps past you while you're still typing, more librarian education and practice
	a lot of waiting
	people wait just long enough to start typing, then happens at same time	both user and librarian start to type in search box of Barton at same time, whoever hits tab first, theirs types over other one, this is confusing
		making it more clear by librarian communication, have the do it or say you will do it, make it clear, maybe system shouldn't allow you to both type at once
	people didn't like being led through each and every step, they just want the answer, they don't want a lot of instruction	librarian should ask do you want me to show you how to find it or just find the answer? Librarians are not yet comfortable doing a reference interview in this environment. Need to verbalize it more. We are used to reading body language, here we can't.
	it's info overload to be sent to page after page, after page	librarian training needed, when to send them results vs every page
	depends on the question - do they want help on how to use something or just get the info?
	barton title tags showing in transcript, "item sent - start0-0-mit01" - very confusing	will go away with no-frames
	user confused about pages being sent to them, thought it was an error	librarian should ask the person if they have used this system before, or librarian should say: "I'm sending you a page now."
	user pressed return and the message was sent even though she wasn't ready to send it	want to make a new paragraph, so hit return, but didn't mean to send it yet
	some people would hit enter key and it didn't work, so they had to cklick the send button	maybe enter shouldn't send? or maybe it should - they just have to get used to it, people are used to chat in other systems, they can learn not to type long paragraphs and that the "enter" key sends
	when session is closed, there is a list of links you've visted and other screen goes blank, user assumed her computer had crashed at first because of blank screen	screen should have a closing message
	people close browser without clicking "exit" button	can system log them out automatically when they close browser? Librarian can't quite tell when they have left.
	many people thought ask us live would be invasive, or didn't want to bother librarain with "stupid questions", some people afraid about what would happen, shyed away from it, sense of catastrophe that could happen	interesting
	pushing pages to people is confusing, they are used to chat, but not used to having pages pushed at them	Have the librarian ask them - are you new to this? Put up a stock answer that tells them pages will be pushed and other basic info - if they are not a repeat user.
	AIM Express is chat system someone likes, she's used to new message come up from bottom
	a few comments, wow this is cool, but only if they understand what's happening
	seems like you want your chat session in one place and push in another place, separate window
	when user is navigating in a separate window and librarian sends a message window pops to front and user doesn't realize that window popping to front is librarian sending message, they think it's an accident	Include in stock answer - when librarian sends a message, window will pop to front.