Many libraries have responded to user demand for Google-style searching by implementing web scale discovery services such as EDS, Primo, Summon, and WorldCat Local. By exporting their local catalogs to the meta-aggregate article indexes provided by these vendors, libraries create “single search” across articles and books.
One frustration with this approach has been that local collections get “lost” in the overwhelming amount of subscription article content. For this post I wanted to measure discoverability of local collections in the four major systems. I conducted a test of known item print book searches in eight different implementations.
I chose eight libraries with large local collections that had implemented one of the four major discovery services. All eight combined local and subscription content in a single result list. For comparison I also tested Google.
I searched using the keyword default search box and did not facet or sort. In each system I ran the following types of searches:
- Single word titles (e.g. 1984)
- Titles with “stop words” (e.g To have and have not)
- Title/author keyword (e.g. Smith and On beauty)
- Book citation (copied from bibliographies)
All of my searches were for print items held by the libraries.
- WorldCat Local won, ranking highest in all search types
- Google came in a close second
- Summon scored well in all categories, but did not excel in citation searches
- WorldCat Local and Google were the only systems to rank highly in citation searches
- Primo scored low on author/title searches and failed citation searches
- EDS did the worst: stop words generated errors, and no results were returned for citations or ISBN searches
Title vs. keyword matching
Some systems ranked keyword relevance higher than an exact title match. This could make single-word titles hard to find. For example, a user looking for the novel “V” by Pynchon might see this first result in EDS:
This test only measured known item searching for print materials. In that category it is not surprising that WorldCat Local, a discovery system built around OCLC’s union book catalog, excelled. It is also not surprising that local print content tends to get “lost” in systems focused more on subscription article content.
EDS and Primo were the low performers, but it should be noted that these tools may perform significantly better in different implementations: I tested only two implementations of each system and did not take into account any customized “boosting” of local content that may be applied by libraries.
This test, while cursory, does suggest that library discovery systems continue to have problems with known item searching. Our users come to these systems with expectations formed by Google: they expect to be able to search by author/title keyword, ISBN, and to be able to copy and paste formatted citations. And we are failing to meet those expectations.
Library sites tested
University of Georgia (EDS)
Oxford University, UK (Primo)
University of Minnesota (Primo)
University of Texas Libraries (Summon)
Duke University (Summon)
University of California Berkeley (WorldCat Local)
University of Washington (WorldCat Local)