How can digital collections continue to evolve without compromising the integrity of the original archival material? We explore how Veridian supports text corrections, image display adjustments, and other enhancements while keeping the underlying source files intact.
When libraries and archives make historical collections available online, they need to balance two important responsibilities: preserving the integrity of the original material and making it as accessible and useful as possible for researchers.
During a recent Veridian webinar, a participant raised an important question about whether adjustments to historical materials, such as changing image contrast, could compromise the integrity of the original archival source.
It's a valid concern and one that extends beyond image adjustments. Digital collections can evolve through text corrections, user-contributed tags and comments, and other enhancements that improve how people discover and use historical material. So, what happens to the underlying archival files when these changes are made?
For us, this distinction is important. The original archival source data is retained separately from the changes and enhancements made to the online collection. This allows collections to evolve without compromising the integrity of the preserved source.
From archival source to online collection
A typical newspaper digitization project, for example, involves several stages, from scanning historical material to creating the digital files required for online access.
Scanning generally produces high-resolution TIFF images. These are processed alongside optical character recognition (OCR) text, metadata, and structural information to create digital objects, commonly using the METS/ALTO standards. The resulting files are then ingested into Veridian, where they can be searched, browsed, and viewed online.
An important distinction is that the high-resolution TIFF files are retained for preservation purposes but are not typically used to deliver images to website visitors. This is because their large file sizes (40-80 MB per page) make them impractical for everyday online viewing. Instead, more efficient image formats, such as JPEG 2000, are used to provide online access, while PDFs may also be available for viewing, printing, or downloading material. This approach applies to a wide range of other digitized historical materials, including books, journals, and personal manuscripts, maps and photographs.
This separation means the preservation files can be maintained independently of the formats used to display material online.
Related reading: Understanding file structure behind a newspaper digitisation project
Supporting corrections and display adjustments
Making a collection available online is not the end of its development. As people use the material, opportunities arise to improve its accuracy, discoverability, and accessibility. A key principle of Veridian's approach is that these improvements can be managed separately from the original source data.
Text corrections
OCR technology converts text within scanned images into searchable digital text, but historical materials present particular challenges. Faded print, unusual typefaces, damaged pages, and inconsistent printing quality can all contribute to transcription errors.
Veridian's User Text Correction functionality allows these errors to be corrected without modifying the original OCR source files. Corrections are stored separately in the database, preserving the original OCR text and allowing it to be restored if necessary. These corrections can improve the accuracy of full-text searching while keeping the source data intact.
Image display adjustments
Historical documents are not always easy to read. Faded ink, uneven contrast, the condition of the original material, and limitations in the scanning technology used at the time of digitization can all make details difficult to distinguish.
Veridian provides viewing controls that allow users to adjust brightness and contrast, as well as invert image colors, to improve readability.
These adjustments affect how an image is displayed in the viewer. They do not change, overwrite, or damage the underlying source files.
Redactions and future enhancements
Some collections also require controls over what information is publicly accessible, including the redaction of sensitive or restricted content. The same principle applies: where redactions are implemented as a separate layer, the preserved source remains unchanged while the public-facing version reflects the required restrictions.
Looking ahead, this approach can also support future image-processing enhancements without requiring permanent changes to the archival originals. Separating the source from subsequent changes means collections can continue to develop while preserving the material from which they were created.
Protecting source data through verification and backups
Preserving archival integrity also means protecting digital files against accidental corruption, loss, and other risks over time. For Veridian-hosted collections, our backup processes maintain the original source data batches, typically containing images, METS/ALTO files, and PDFs, alongside separate backups of user-generated information such as text corrections, comments, and tags.
An important part of this process is BagIt, a standardized format for packaging and validating digital files. BagIt uses checksums, which are digital values calculated from a file's contents, to help detect whether files have changed or become corrupted. Our systems automatically validate underlying source data batches using BagIt checksums each day. We also compare live and backup data to help identify inconsistencies.
These measures support the integrity of the files we manage and help ensure the original source data remains available independently of subsequent improvements to the online collection.
Related reading: Veridian's multi-layered backup system
Learn more about Veridian
Want to know more about how Veridian protects archival source files while supporting improvements to your digital collections?
