Professional Development

NC-LITe and WFU Summon Usability Study Findings Presented at Appalachian State University

Tuesday, May 29, 2012 4:13 pm

On Monday, May 21, Hu Womack, Kevin Gilbertson, Lauren Pressley, and I drove to the High Country of Boone to attend NC-LITe. It was doubly wonderful for me because I had the opportunity to travel with three amazing colleagues to one of my favorite places on earth.

It seems hard to take in that it has been seven years since I left my position at Appalachian as an Instruction Librarian. Our family moved from Boone three months before the Library moved into their new facility so it was fun just seeing the building! When we drove up in the parking deck and told the attendant that we were there for NC-LITe, she proclaimed in her authentic mountain brogue, “We finally got us one!” and for me, it felt like I was home.

When we entered the library, Scott Rice greeted us and escorted us to the fourth floor conference room. We were welcomed with a beautiful spread of pastries and coffee. Institutions represented were Salem College, UNC-G, High Point University, Appalachian, and WFU. The intimate group encouraged informal discussion.

The day began with around the table introductions and catch-up. Most of the shared information had to do with strategies for reaching the masses by offering consistent content for specific courses such as Expository Writing and Freshmen Seminars. (We are so fortunate at Wake to be able to tailor our Freshmen Seminars to the content taught.) It was interesting to note that each represented institution is currently using or has plans to use a discovery tool similar to Summon.

After the round table discussions, Kevin, Lauren and I presented our findings from our Summon usability study. Kevin began the discussion by offering some historical perspective regarding how our Library first used Multi-search and then implemented Summon last summer. Our tech team was forced to move quickly in order to get it up and going before Erik Mitchell left. In light of this, the tech team decided it would be particularly helpful to conduct a usability study to see how ZSR’s Summon (the “Everything” link) was being used.

After Kevin’s introduction, we used a clicker presentation to introduce our findings. (Note: having three techies on this trip was a true gift!!) Our usability study was based loosely on NC State’s usability study which was done three years ago and was sponsored by Summon (NC State was a Summon beta site). Their study was led by an outside consultant and their report can be found here:

For our study, we used eight student workers who were available for the afternoons that we conducted the usability study (thank you, ZSR student supervisors for sharing your students with us!). Each individual interview lasted 30-45 minutes and we met in the vacant office of the Instruction Suite (302B). Kevin conducted the interviews while Lauren and I took notes.

We began with what NC State called “Standard Framing” which helps to determine the student’s proficiency at navigating the Library website. This ended up being a very interesting section of the study! Students were asked to go to the Library homepage and asked to find a book about Bill Gates but not by Bill Gates and to tell where it was located in our library. A couple of students thought they had a book about Bill Gates when it was about the Bill and Melinda Gates Foundation. A couple of students did not notice that they had selected an ebook, and at the end of the interview one student asked, “What is an ebook?” One student found a book in the Professional Center Library and did not realize that it was not in the ZSR Library.

Students were given a journal article citation and were asked to locate the article. There were two students who used the “Journals” link to search for the article title and those students could not locate the article (one of those eventually used the “Everything” link and was successful). Two students used the journal names in the journal link and they were successful. One used the “Article” link and she was successful, and one started with the “Everything” link and was successful. One student started with the catalog link and after a long and winding search, located the article.

We found similar results when we asked students to find a scholarly journal article about stress among college students. One student started with the Databases link and another started with the Journals link and each typed in their subjects within the boxes; neither was successful. Students who started with the “Everything” link or the “Articles” link were successful. One student (not the first Database selector) got frustrated with the Databases link and decided to switch to JSTOR because her professor showed it to her once and she knew it was really big.

We then switched to having the students search only using the Everything link (Summon). We asked them to find a book by an author they liked. Two of the students were not successful with their search because their authors were J.D. Salinger and H.G. Wells; many sources came up about these authors, but not by these authors (to be successful, they should have used the Books/eBooks facet). One student searched for “Harry Potter” and there were six entries before a “Harry Potter” book appeared on the list.

Students were asked to find the latest issue of the Journal of the American Chemical Society. None of the students were successful with this search. What happened was that they began with a searches for the [Journal of the American Chemical Society] which produced a list, but the first three entries on that page directed them to Carpenter Library. (To correctly answer this question, they needed to click the fourth entry to see our subscription to the latest articles.) When they were not successful with the first couple of links, they switched to the “journal articles” facet where they lost the ability to sort by relevance. Summon does an excellent job searching by title, but after that, relevance is lost. Several of the students were confident that they had found the answer, when in fact they were looking at articles where the Journal of the American Chemical Society was cited.

We asked the students find to an article from the New York Times from the month and year they were born. Only two students noticed that newspapers were excluded from the Summon search results (it could be turned on from the facets). Two students were simply not successful with this search, two other students thought they were successful but their articles were from New York Times Magazine, and two students were successful because they ended up with links to Proquest Historical Newspapers.

None of the students were successful finding an article on “Is genetically modified “golden rice” safe?” As the interviewers, we really wished the student would use quotation marks to look for strings of words or “Ctrl-F” to search for words in their results. We hope that our LIB100 instructors will start teaching these skills as part of their curriculum!

We asked the students if they would recommend the Everything link to their friends, and five out of eight said they would “if they don’t know where else to start.”

We also asked the students what they would change about the Everything link and they had a few suggestions. The number one suggestion was to make it so that it could do a better job searching by relevance; they worded this concept something like this: If you search for “Harry Potter,” a “Harry Potter” book should come up to the top. A couple of students said that they wished that you could limit the category options before you do your search (searching by author, title, etc.). Another student said that it would help if the results were sorted by format with maybe the books on top followed by the journals.

After our presentation, Margaret Gregor, Appalachian’s Instructional Materials Center Librarian and Scott Rice, Appalachian’s E-Learning Librarian, presented a tool they created to provide virtual tours of the IMC. The students were required to find 16 hidden puzzle pieces and a book in the collection (the students were instructed to write down the bar code of the book they found). Scott set up a temporary password for us so that we could try it out. They reported that over 500 students had completed the tour. Here is a link to this site and I can give you a password that you can use if you are interested:

After Margaret and Scott’s discussion, we then had some group discussions about things we were doing at different institutions. That discussion was followed by lunch at “Our Daily Bread” in downtown Boone and then a tour of the Library.

Overall, it was a great way to spend the day! I’m already looking forward to NC-LITe in the fall!!



2 Responses to “NC-LITe and WFU Summon Usability Study Findings Presented at Appalachian State University”

  1. It was a great day! And it was nice to talk about our Summon findings, and even more so after the trip to Duke the week before! I was most struck that the usability study both gave ideas for how to improve the service (as usability studies are designed to do) but also to help us think about how and what we teach (a nice bonus benefit). I’m looking forward to seeing improvement to Summon and our teaching based on the findings!!

  2. Such great info on how students use our resources! Thanks for the summary – I’m already plotting how to improve how I teach Summon in my one-shots and LIB210 classes.

ALA Annual
ALA Midwinter
Career Development for Women Leaders
Carolina Consortium
CASE Conference
Celebration: Entrepreneurial Conference
Charleston Conference
Coalition for Networked Information
Digital Forsyth
Electronic Resources and Libraries
Elon Teaching and Learning Conference
Entrepreneurial Conference
Evidence Based Library and Information Practice (EBLIP)
Ex Libris Users of North America (ELUNA)
First-Year Experience Conference
Handheld Librarian
ILLiad Conference
Innovative Library Classroom Conference
Journal reading group
Leadership Institute for Academic Librarians
Library Assessment Conference
Lilly Conference
LITA National Forum
Mentoring Committee
Music Library Association
NCCU Conference on Digital Libraries
North Carolina Serials Conference
online course
Online Learning Summit
Open Repositories
Professional Development Center
Site Visits and Tours
Society of American Archivists
Society of North Carolina Archivists
Southeast Music Library Association
Sun Webinar Series
TALA Conference
UNC Teaching and Learning with Technology Conference
University Libraries Group
ZSR Library Leadership Retreat
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007

Powered by, protected by Akismet. Blog with