Professional Development

NC-LITe and WFU Summon Usability Study Findings Presented at Appalachian State University

Tuesday, May 29, 2012 4:13 pm

On Monday, May 21, Hu Womack, Kevin Gilbertson, Lauren Pressley, and I drove to the High Country of Boone to attend NC-LITe. It was doubly wonderful for me because I had the opportunity to travel with three amazing colleagues to one of my favorite places on earth.

It seems hard to take in that it has been seven years since I left my position at Appalachian as an Instruction Librarian. Our family moved from Boone three months before the Library moved into their new facility so it was fun just seeing the building! When we drove up in the parking deck and told the attendant that we were there for NC-LITe, she proclaimed in her authentic mountain brogue, “We finally got us one!” and for me, it felt like I was home.

When we entered the library, Scott Rice greeted us and escorted us to the fourth floor conference room. We were welcomed with a beautiful spread of pastries and coffee. Institutions represented were Salem College, UNC-G, High Point University, Appalachian, and WFU. The intimate group encouraged informal discussion.

The day began with around the table introductions and catch-up. Most of the shared information had to do with strategies for reaching the masses by offering consistent content for specific courses such as Expository Writing and Freshmen Seminars. (We are so fortunate at Wake to be able to tailor our Freshmen Seminars to the content taught.) It was interesting to note that each represented institution is currently using or has plans to use a discovery tool similar to Summon.

After the round table discussions, Kevin, Lauren and I presented our findings from our Summon usability study. Kevin began the discussion by offering some historical perspective regarding how our Library first used Multi-search and then implemented Summon last summer. Our tech team was forced to move quickly in order to get it up and going before Erik Mitchell left. In light of this, the tech team decided it would be particularly helpful to conduct a usability study to see how ZSR’s Summon (the “Everything” link) was being used.

After Kevin’s introduction, we used a clicker presentation to introduce our findings. (Note: having three techies on this trip was a true gift!!) Our usability study was based loosely on NC State’s usability study which was done three years ago and was sponsored by Summon (NC State was a Summon beta site). Their study was led by an outside consultant and their report can be found here: http://www.lib.ncsu.edu/userstudies/studies/2010_summon/

For our study, we used eight student workers who were available for the afternoons that we conducted the usability study (thank you, ZSR student supervisors for sharing your students with us!). Each individual interview lasted 30-45 minutes and we met in the vacant office of the Instruction Suite (302B). Kevin conducted the interviews while Lauren and I took notes.

We began with what NC State called “Standard Framing” which helps to determine the student’s proficiency at navigating the Library website. This ended up being a very interesting section of the study! Students were asked to go to the Library homepage and asked to find a book about Bill Gates but not by Bill Gates and to tell where it was located in our library. A couple of students thought they had a book about Bill Gates when it was about the Bill and Melinda Gates Foundation. A couple of students did not notice that they had selected an ebook, and at the end of the interview one student asked, “What is an ebook?” One student found a book in the Professional Center Library and did not realize that it was not in the ZSR Library.

Students were given a journal article citation and were asked to locate the article. There were two students who used the “Journals” link to search for the article title and those students could not locate the article (one of those eventually used the “Everything” link and was successful). Two students used the journal names in the journal link and they were successful. One used the “Article” link and she was successful, and one started with the “Everything” link and was successful. One student started with the catalog link and after a long and winding search, located the article.

We found similar results when we asked students to find a scholarly journal article about stress among college students. One student started with the Databases link and another started with the Journals link and each typed in their subjects within the boxes; neither was successful. Students who started with the “Everything” link or the “Articles” link were successful. One student (not the first Database selector) got frustrated with the Databases link and decided to switch to JSTOR because her professor showed it to her once and she knew it was really big.

We then switched to having the students search only using the Everything link (Summon). We asked them to find a book by an author they liked. Two of the students were not successful with their search because their authors were J.D. Salinger and H.G. Wells; many sources came up about these authors, but not by these authors (to be successful, they should have used the Books/eBooks facet). One student searched for “Harry Potter” and there were six entries before a “Harry Potter” book appeared on the list.

Students were asked to find the latest issue of the Journal of the American Chemical Society. None of the students were successful with this search. What happened was that they began with a searches for the [Journal of the American Chemical Society] which produced a list, but the first three entries on that page directed them to Carpenter Library. (To correctly answer this question, they needed to click the fourth entry to see our subscription to the latest articles.) When they were not successful with the first couple of links, they switched to the “journal articles” facet where they lost the ability to sort by relevance. Summon does an excellent job searching by title, but after that, relevance is lost. Several of the students were confident that they had found the answer, when in fact they were looking at articles where the Journal of the American Chemical Society was cited.

We asked the students find to an article from the New York Times from the month and year they were born. Only two students noticed that newspapers were excluded from the Summon search results (it could be turned on from the facets). Two students were simply not successful with this search, two other students thought they were successful but their articles were from New York Times Magazine, and two students were successful because they ended up with links to Proquest Historical Newspapers.

None of the students were successful finding an article on “Is genetically modified “golden rice” safe?” As the interviewers, we really wished the student would use quotation marks to look for strings of words or “Ctrl-F” to search for words in their results. We hope that our LIB100 instructors will start teaching these skills as part of their curriculum!

We asked the students if they would recommend the Everything link to their friends, and five out of eight said they would “if they don’t know where else to start.”

We also asked the students what they would change about the Everything link and they had a few suggestions. The number one suggestion was to make it so that it could do a better job searching by relevance; they worded this concept something like this: If you search for “Harry Potter,” a “Harry Potter” book should come up to the top. A couple of students said that they wished that you could limit the category options before you do your search (searching by author, title, etc.). Another student said that it would help if the results were sorted by format with maybe the books on top followed by the journals.

After our presentation, Margaret Gregor, Appalachian’s Instructional Materials Center Librarian and Scott Rice, Appalachian’s E-Learning Librarian, presented a tool they created to provide virtual tours of the IMC. The students were required to find 16 hidden puzzle pieces and a book in the collection (the students were instructed to write down the bar code of the book they found). Scott set up a temporary password for us so that we could try it out. They reported that over 500 students had completed the tour. Here is a link to this site and I can give you a password that you can use if you are interested: http://library.appstate.edu/elearn/imc

After Margaret and Scott’s discussion, we then had some group discussions about things we were doing at different institutions. That discussion was followed by lunch at “Our Daily Bread” in downtown Boone and then a tour of the Library.

Overall, it was a great way to spend the day! I’m already looking forward to NC-LITe in the fall!!

 

 

2 Responses to “NC-LITe and WFU Summon Usability Study Findings Presented at Appalachian State University”

  1. It was a great day! And it was nice to talk about our Summon findings, and even more so after the trip to Duke the week before! I was most struck that the usability study both gave ideas for how to improve the service (as usability studies are designed to do) but also to help us think about how and what we teach (a nice bonus benefit). I’m looking forward to seeing improvement to Summon and our teaching based on the findings!!

  2. Such great info on how students use our resources! Thanks for the summary – I’m already plotting how to improve how I teach Summon in my one-shots and LIB210 classes.


Pages
About
Categories
2007 ACRL Baltimore
2007 ALA Annual
2007 ALA Gaming Symposium
2007 ALA Midwinter
2007 ASERL New Age of Discovery
2007 Charleston Conference
2007 ECU Gaming Presentation
2007 ELUNA
2007 Evidence Based Librarianship
2007 Innovations in Instruction
2007 Kilgour Symposium
2007 LAUNC-CH Conference
2007 LITA National Forum
2007 NASIG Conference
2007 North Carolina Library Association
2007 North Carolina Serials Conference
2007 OCLC International ILLiad Conference
2007 Open Repositories
2007 SAA Chicago
2007 SAMM
2007 SOLINET NC User Group
2007 UNC TLT
2007_ASIST
2008
2008 Leadership Institute for Academic Librarians
2008 ACRL Immersion
2008 ACRL/LAMA JVI
2008 ALA Annual
2008 ALA Midwinter
2008 ASIS&T
2008 First-Year Experience Conference
2008 Lilly Conference
2008 LITA
2008 NASIG Conference
2008 NCAECT
2008 NCLA RTSS
2008 North Carolina Serials Conference
2008 ONIX for Serials Webinar
2008 Open Access Day
2008 SPARC Digital Repositories
2008 Tri-IT Meeting
2009
2009 ACRL Seattle
2009 ALA Annual
2009 ALA Annual Chicago
2009 ALA Midwinter
2009 ARLIS/NA
2009 Big Read
2009 code4lib
2009 Educause
2009 Handheld Librarian
2009 LAUNC-CH Conference
2009 LAUNCH-CH Research Forum
2009 Lilly Conference
2009 LITA National Forum
2009 NASIG Conference
2009 NCLA Biennial Conference
2009 NISOForum
2009 OCLC International ILLiad Conference
2009 RBMS Charlottesville
2009 SCLA
2009 UNC TLT
2010
2010 ALA Annual
2010 ALA Midwinter
2010 ATLA
2010 Code4Lib
2010 EDUCAUSE Southeast
2010 Handheld Librarian
2010 ILLiad Conference
2010 LAUNC-CH Research Forum
2010 LITA National Forum
2010 Metrolina
2010 NASIG Conference
2010 North Carolina Serials Conference
2010 RBMS
2010 Sakai Conference
2011 ACRL Philadelphia
2011 ALA Annual
2011 ALA Midwinter
2011 CurateCamp
2011 Illiad Conference
2012 SNCA Annual Conference
ACRL
ACRL 2013
ACRL New England Chapter
ACRL-ANSS
ACRL-STS
ALA Annual
ALA Annual 2013
ALA Editions
ALA Midwinter
ALA Midwinter 2012
ALA Midwinter 2014
ALCTS Webinars for Preservation Week
ALFMO
APALA
ARL Assessment Seminar 2014
ARLIS
ASERL
ASU
Audio streaming
authority control
Berkman Webinar
bibliographic control
Book Repair Workshops
Career Development for Women Leaders Program
CASE Conference
cataloging
Celebration: Entrepreneurial Conference
Charleston Conference
CIT Showcase
CITsymposium2008
Coalition for Networked Information
code4lib
commons
Conference Planning
Conferences
Copyright Conference
costs
COSWL
CurateGear 2013
CurateGear 2014
Designing Libraries II Conference
DigCCurr 2007
Digital Forsyth
Digital Humanities Symposium
Disaster Recovery
Discovery tools
E-books
EDUCAUSE
Educause SE
EDUCAUSE_SERC07
Electronic Resources and Libraries
Embedded Librarians
Entrepreneurial Conference
ERM Systems
evidence based librarianship
FDLP
FRBR
Future of Libraries
Gaming in Libraries
General
GODORT
Google Scholar
govdocs
Handheld Librarian Online Conference
Hurricane Preparedness/Solinet 3-part Workshop
ILS
information design
information ethics
Information Literacy
innovation
Innovation in Instruction
Innovative Library Classroom Conference
Inspiration
Institute for Research Design in Librarianship
instruction
IRB101
Journal reading group
Keynote
LAMS Customer Service Workshop
LAUNC-CH
Leadership
Learning spaces
LibQUAL
Library 2.0
Library Assessment Conference
Library of Congress
licensing
Lilly Conference
LITA
LITA National Forum
LOEX
LOEX2008
Lyrasis
Management
Marketing
Mentoring Committee
MERLOT
metadata
Metrolina 2008
MOUG 09
MOUG 2010
Music Library Assoc. 07
Music Library Assoc. 09
Music Library Assoc. 2010
NASIG
National Library of Medicine
NC-LITe
NCCU Conference on Digital Libraries
NCICU
NCLA
NCLA Biennial Conference 2013
NCPC
NCSLA
NEDCC/SAA
NHPRC-Electronic Records Research Fellowships Symposium
NISO
North Carolina Serial Conference 2014
Offsite Storage Project
OLE Project
online catalogs
online course
OPAC
open access
Peabody Library Leadership Institute
plagiarism
Podcasting
Preservation
Preservation Activities
Preserving Forsyth LSTA Grant
Professional Development Center
rare books
RDA/FRBR
Reserves
RITS
RTSS 08
RUSA-CODES
SAA Class New York
SAMM 2008
SAMM 2009
Scholarly Communication
ScienceOnline2010
Social Stratification in the Deep South
Social Stratification in the Deep South 2009
Society of American Archivists
Society of North Carolina Archivists
SOLINET
Southeast Music Library Association
Southeast Music Library Association 08
Southeast Music Library Association 09
SPARC webinar
subject headings
Sun Webinar Series
tagging
TALA Conference
Technical Services
technology
ThinkTank Conference
Training
ULG
Uncategorized
user studies
Vendors
video-assisted learning
visual literacy
WakeSpace
Web 2.0
Webinar
WebWise
WFU China Initiative
Wikis
Women's History Symposium 2007
workshops
WSS
ZSR Library Leadership Retreat
Tags
Archives
November 2014
October 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007

Powered by WordPress.org, protected by Akismet. Blog with WordPress.com.