3. The Role of Software in Addressing Hidden Collections

Reducing archival backlogs fundamentally requires adopting more-efficient means of processing collections, but software can contribute to that efficiency and make it easier for archives to provide online access to archival descriptions. At many archives, information is scattered across several different digital and physical systems, resulting in duplication of effort and difficulty in locating needed information. For instance, one archive uses a hodgepodge of methods to manage its collections, including paper accession records; an Access database for collection-level status information; lists and databases for tracking statistics; hundreds of EAD finding aids; hundreds of paper control folders providing collection-level information, some of which is duplicated in Word files or in XML finding aids; and item-level descriptions of objects to be digitized in Excel spreadsheets. This miscellany means that there are problems with versioning, redundancy, finding information, and making that information publicly accessible. Likewise, Chris Prom found that many archives are using a variety of tools at various steps in their workflows, so much so that "their descriptive workflows would make good subjects for a Rube Goldberg cartoon." Examples include the Integrated Library System (ILS) for the creation of MARC records, NoteTab and XMetaL for authoring finding aids, Access for managing accessions, Word for creating container lists, and DynaWeb for serving up finding aids (Prom 2008, 27). (See Appendix 1 of this paper for a more detailed description of the archival workflow.)

In addition to the inefficiencies of using multiple systems to manage common data, Prom et al. (2007, 158-159) notes a correlation between using EAD and other descriptive standards with larger backlogs and slower processing speeds. (EAD is an XML-based standard for representing archival finding aids, which describe archival collections.) Some institutions simply lack the ability to produce EAD finding aids or MARC catalog records. As Prom et al. suggest, "Until creating an on-line finding aid and sharing it with appropriate content aggregators is as easy as using a word processor, the archival profession is unlikely to significantly improve access to the totality of records and papers stored in a repository" (2007, 159). One of the ARL Task Force on Special Collections' recommendations thus focuses on developing usable tools to describe and catalog archival collections: "Since not all institutions are currently employing applicable national standards, the development of easy-to-use tools for file encoding and cataloging emerge as a priority. These tools should be simple enough to be used by students or paraprofessionals working under the supervision of librarians or archivists" (Jones 2003, 11). Greene and Meissner (2005, 242) suggest that software can play a vital role in streamlining archival workflows by enabling archivists to describe the intellectual arrangement of a collection without investing the time to organize it physically. In 2003, Carol Mandel observed that "I also have been told again and again that we really don't have software for managing special collections. We don't have the equivalent of your core bibliographic system that helps you bring things in and move them around efficiently and know what you are doing with them" (Mandel 2004, 112).

Fortunately, powerful software for managing special collections and archives is emerging. This report is more a sampling of leading archival management systems that offer English-language user interfaces than a comprehensive examination of every potentially relevant application.¹ Of course, software itself cannot solve the problem of hidden collections; what matters is how software is used and incorporated into streamlined, effective workflows. Although archival management systems such as Archon and Archivists' Toolkit can play an important role in facilitating the production of EAD and MARC records and streamlining archival workflows, Prom, a developer of Archon, cautions that "archivists should not treat them as magic bullets. They will only prove to be effective in encouraging processing and descriptive efficiency if they are implemented as part of a strategic management effort to reformulate processing policies, processes, procedures" (Prom 2008, 32).²

In conversations with archivists, I asked what their dream software would be as a way of identifying what features would be most important to them and envisioning what may be possible. They often responded that they liked the software applications they were currently using, but would add a few features. The responses point out some of the strengths of existing software and future directions for software developers. Through conversations with archivists and a review of existing research, I've identified the following desired features for archival management systems.³

Integrated: Rather than having to enter data in multiple databases, an archivist could enter the data once and generate multiple outputs, such as an accession list, EAD finding aids, a MARC record, a shelf list, and an online exhibit. As one archivist remarked, "The ideal approach to minimal processing is that you touch everything only once. Every time you touch it is more staff time."
Supports importing legacy data: Many archives have already invested a great deal of time in creating EAD finding aids. Likewise, they want an easy way to import other data, such as accessions information. They want software that will seamlessly import existing data—which can be a challenge, given the variability of EAD documents and other forms of archival data.
Enables easy exporting of data: Given how quickly software becomes obsolete, archivists recognize the need for being able to export data cleanly and easily. One archivist commented, "Archival material is so specific that you don't want to get locked into anything... Ideally, I would want something that would also preserve that information in a format that is able to migrate if needed."
Provides Web-publishing capabilities: Many archives lack the ability to make their finding aids available on-line. By providing a Web-publishing component, an archival management system would enable archives to provide wider access to their collections. Through on-line access, archives have found that they become more visible. As Victoria Steele (2008) writes, "As new finding aids become viewable online, we have seen, over and over, that researchers are at our door to consult the collections they describe. But it must be said that a consequence of our success has been that staff whose primary focus was the processing of collections are now almost wholly engaged in handling reader requests, reference inquiries, and licensing agreements—leaving them almost no time for processing."
Simple yet powerful: Archivists want software that is "as easy to use as Word but transforms to the Web and generates EAD at the click of a button." Students and paraprofessionals without strong archival training need software that provides simple templates for entering data, so that they know what information goes where. (Clear user guides can also assist in ensuring the quality and consistency of data). If software is too complex or cumbersome to use, much time will be lost. The software should be flexible enough to adapt to the archive's existing workflow.
Rigorous, standards-based: The archival community has embraced standards such as EAD, Describing Archives: A Content Standard (DACS), and Encoded Archival Context (EAC), and archivists want software that ensures conformity to these standards. The potential for inconsistent, incorrect data increases as more people participate in describing archival collections. Archival management systems can reduce the likelihood of error by ensuring that data are entered according to standard archival practices (for instance, making sure that dates are in the proper format).
Provides collection management features. Archivists want software that helps them manage and track their operations more efficiently. Several interviewees wanted to be able to track reference statistics, while others would like to generate temporary records and track locations.
Portable: Archivists often work in environments where they do not have access to a desktop computer or even to the network, such as the home of a donor or a room in a small museum. As a result, they may begin collecting data using offline software such as spreadsheets. Once they return to their offices, they have to redo much of the work to make it fit into their existing systems. According to one archivist, "It would be useful if we could begin processing on-site, where we first encounter the material. We have to begin again each time we start a new stage." Archival software could thus support offline data entry, allowing archivists to enter data into a laptop and then upload it into an archival management system once they have network connectivity. Perhaps archival management system could also support data entry through mobile, wireless device such as iPhones).⁴
Aids in setting priorities for processing: Some archival management systems enable archives to record which collections are higher priorities, thus allowing archivists to plan processing more effectively. In defining approaches to hidden collections, the ARL Special Collections Task Force put forward several recommendations that involve using tools and measures to assess processing priorities. Two of these recommendations are "Develop qualitative and quantitative measures for the evaluation of special collections" and "Support collection mapping to reveal the existence of special collections strengths and gaps, as well as to identify hidden collections" (ARL 2006). Such tools are outside the scope of this report, but it is important to acknowledge the role of related technologies. Examples of tools and protocols that can be used to assess collections and prioritize processing include the Philadelphia Area Consortium of Special Collections Libraries (PACSCL) Consortial Survey Initiative,⁵ OCLC's WorldCat collection analysis tool,⁶ the University of California, Berkeley's survey tool,⁷ and Columbia University's Mellon Survey database.⁸ In some cases, such as with the PACSCL FileMaker database, the information collected through these survey tools can be used as the basis for accessions databases and for DACS-compliant EAD or MARC records (Di Bella 2007).

FOOTNOTES FOR SECTION 3

¹ Archival/collection management and description software that go beyond the scope of this report include Andornot Archives Online, ARGUS/Questor, Collections MOSAiC Plus, CollectionSpace, Embark, Filemaker Pro, HERA2, IDEA, KE EMu, Microsoft Access, Mimsy xg, Minaret, Re:discovery, and VernonSystems Collection. Integrated Digital Special Collections (INDI), currently under development at Brigham Young University, is geared toward large archives or consortia and aims to support a distributed workflow for archival description and management. The accessions and appraisal modules have already been released, but as of August 2008 the future direction of the project was still being determined.

² How to efficiently manage archives is beyond the scope of this report, but Greene and Meissner 2005 and Prom 2007 take up the issue in detail.

³ Many of these desired features jibe with Archivists' Toolkit's (AT) recent survey of 171 users investigating what new features they most desire. The most popular options included "Search improvements" (average of 4.04 out of 5, with 5 being "very important"); "Enable batch editing/ global updating," (4.31); "Web publishing of AT data" (4.2); "Digital objects record revision," which would include support for technical metadata, visual metadata, and independent digital objects (3.97); and a "Use tracking module," which would provide "Support for tracking and reporting the use of a repository's collection" (3.86). See AT User Group Survey Results: Proposed New Features and Functionality at http://www.archiviststoolkit.org/AT%20User%20Group%20SurveyResultsFD.pdf.

⁴ Some tools already provide support for offline editing or data creation through a handheld device. For example, PastPerfect's Scatter/Gather module allows archives to enter information offline through a desktop client, then create a transfer file that is merged with the main data. MINISIS also supports data entry through mobile devices.

⁵ http://www.pacsclsurvey.org/

⁶ http://www.oclc.org/collectionanalysis/default.htm

⁷http://blogs.lib.berkeley.edu/bancsurvey.php/2008/06/02/bancroft_survey_project

⁸http://www.columbia.edu/cu/lweb/services/preservation/surveyTools.html

Role of Tools

3. The Role of Software in Addressing Hidden Collections