--- title: When Scraping Distorts Your Collection’s Usage Figures description: Learn why scraping inflates digital collection usage figures, how to estimate affected periods, and how access options can support clearer reporting. image: https://veridiansoftware.com/hubfs/Blog/analytics-reporting.jpg --- - [![facebook](https://veridiansoftware.com/hubfs/VeridianSoftware_July2025/images/facebook.svg)](https://www.facebook.com/VeridianSoftware) - [![linkedin](https://veridiansoftware.com/hubfs/VeridianSoftware_July2025/images/linkedin.svg)](https://www.linkedin.com/company/dlconsulting-veridian-software) --- [![Veridian](https://veridiansoftware.com/hubfs/VeridianSoftware_July2025/images/veridian-logo.svg "Veridian")](https://veridiansoftware.com) - [About](https://veridiansoftware.com/about) - [Accessibility](https://veridiansoftware.com/accessibility-statement-conformance-report) - [Services](https://veridiansoftware.com/services) - [Scanning](https://veridiansoftware.com/services/scanning) - [Data Conversion](https://veridiansoftware.com/services/data-conversion) - [Presentation Software](https://veridiansoftware.com/services/presentation-software) - [Projects](https://veridiansoftware.com/collections) - Knowledge Base - [Articles & Case Studies](https://veridiansoftware.com/knowledge-base) - [Archival Metadata Standards | Guide](https://veridiansoftware.com/archival-metadata-standards-guide) - [WCAG and Accessibility | Guide](https://veridiansoftware.com/wcag-accessibility-compliance-guide-acr-vpat-for-libraries-archives) - [Contact](https://veridiansoftware.com/contact) - - - - # When Scraping Distorts Your Collection’s Usage Figures # When Scraping Distorts Your Collection’s Usage Figures October 19, 2026 [Collection Health](https://veridiansoftware.com/knowledge-base/tag/collection-health) [Analytics & Impact](https://veridiansoftware.com/knowledge-base/tag/analytics-impact) [Collection Management](https://veridiansoftware.com/knowledge-base/tag/collection-management) Scraping can make a surge in recorded visitors look like audience growth. We explain why this happens, how to report affected figures, and which access options could help. Libraries and cultural heritage organizations rely on usage statistics, such as visitor numbers, browsing sessions, and pageviews, to understand how their digital collections are being used. More importantly, these figures help demonstrate the value of their collections, justify funding, and guide decisions about future investment. Unfortunately, automated tools can now browse websites in ways that closely resemble human visits, inflating the figures and making them harder to interpret. We’ve been seeing this problem across the historical newspaper and archive collections we work with. These spikes are often temporary, with traffic returning toward normal as protection services such as [Cloudflare](https://veridiansoftware.com/knowledge-base/how-veridian-uses-cloudflare-to-protect-digital-collections) identify and respond to unwanted activity. However, the inflated numbers remain in Google Analytics and other reporting systems, making it difficult to accurately measure, track, and report on engagement by real users. So, how can collection owners make sense of these spikes, report usage more reliably, and manage access in ways that make genuine visits easier to distinguish from scraping activity? ![analytics-reporting](https://veridiansoftware.com/hs-fs/hubfs/Blog/analytics-reporting.jpg?width=4078&height=3059&name=analytics-reporting.jpg) ## How has scraping changed and why it affects analytics Traditionally, many scrapers retrieved content without opening and using each page as a person would, meaning platforms like Google Analytics generally did not record their activity. This is because Google Analytics records visits using a small JavaScript program that runs when a page opens in a visitor’s browser. More sophisticated scrapers now use “headless browsers.” These are browsers such as Chrome or Firefox that are controlled by software to mimic human visitors in order to get past protections that would block simpler scraping methods. Unfortunately, this type of scraping can trigger the JavaScript that records website visits, causing scraping activity to be mixed in with genuine human visits in usage statistics and making the two difficult to separate. Tools like Cloudflare continue to develop ways to help tackle this problem. Earlier this year, [Cloudflare introduced Precursor](https://www.cloudflare.com/en-gb/press/press-releases/2026/cloudflare-introduces-precursor-one-click-behavioral-defense-against-modern-bots), which checks activity throughout a visit rather than only when someone first arrives. These developments aim to improve detection, but scraping methods also continue to evolve. ## How to report through an unreliable period When scraping activity cannot be reliably separated from genuine visits, a practical workaround is to replace the affected figures with estimates based on unaffected activity. This gives you an estimated total for the full reporting period, whether you report monthly, quarterly, or yearly. In the example below, September traffic rose sharply during a period identified as scraping activity. The biggest peaks were around September 24–25, but traffic stayed elevated for several days afterward. We therefore identified September 22–28 as the period to exclude and replace with estimates. *![analytics-reporting-scenario](https://veridiansoftware.com/hs-fs/hubfs/Blog/analytics-reporting-scenario.png?width=904&height=352&name=analytics-reporting-scenario.png)* A useful starting point is to match each affected day with the same weekday in unaffected weeks. For a Tuesday, for example, use the average sessions recorded on the other unaffected Tuesdays that month. Repeat this for each day you need to estimate. This allows for differences between weekday and weekend use while reflecting the collection’s current activity. ### Include other measures of engagement Organizations should also consider adjusting what they report on, adding evidence of how people use and benefit from the collection alongside visitor numbers, sessions, and pageviews. Activity from registered users, along with citations, community activities, user feedback, and participation in [text correction](https://veridiansoftware.com/knowledge-base/user-text-correction-guidelines), can help demonstrate the collection’s value to funders and other stakeholders. Registered-user activity will not capture people who use the collection without logging in, so it should complement other usage measures rather than replace them. These reporting workarounds can help in the short term, but relying on estimates leaves gaps in our understanding of collection use. This makes it worth considering changes to how people access a collection. ## Could using different access methods support clearer reporting? Changing how people access a collection may help organizations manage scraping activity and distinguish everyday browsing from bulk downloads. The approach needs to balance public access, protection, operating costs, and how the organization wants its material to be used. ### Keeping access free while requiring registration One suggestion raised by clients is to require visitors to create a free account and log in before viewing the collection. This may seem like a straightforward way to keep scrapers out, but bots can create accounts and log in too, and we have seen this happen. Even so, requiring an account gives collection owners more ways to manage and protect access. Registration also adds an extra step for genuine visitors, so it’s worth explaining why an account is now required and making clear that access is still free. ### Make scraping easier instead of harder It may seem counterintuitive, but one alternative we have been discussing as a team is to make bulk access easier. Rather than focusing solely on preventing scraping, collection owners could offer a supported way to download large quantities of content. This would give people who want the content an alternative to working through individual pages using scraping methods that can be complex and costly, both for them and for the organization hosting the collection. In return for easier access, those requesting bulk downloads could be asked to identify themselves, explain what they want the content for, and request permission. Approved users could then access a bulk download service or receive an API key that allows their software to request content directly. Offering supported bulk or programmatic access to content is not a new idea. Large public resources such as Chronicling America and Trove already offer this option. Depending on the collection, this could allow owners to: - Set terms and conditions for bulk access and reuse - Limit downloads to selected material, such as public-domain newspapers - Record bulk downloads separately from everyday browsing - Potentially charge a fee for access to the bulk download service. Some users may prefer to pay for a straightforward download service rather than spend time and money scraping the collection. Making bulk access easier could help collection owners understand who wants their content and why, while keeping that activity separate from everyday browsing. Whether this is the right approach will depend on each collection’s access goals and how its material can be used. --- **If scraping is affecting your usage reports, please talk to our team. We can help review the affected figures and discuss options for managing access and reporting on genuine use of your collection.** **[![CONTACT US](https://hubspot-no-cache-ap1-prod.s3.amazonaws.com/cta/default/441834891/interactive-232441011680.png)](https://veridiansoftware.com/hs/cta/wi/redirect?encryptedPayload=AVxigLISLuiK1%2F9VBpKvLpdExxbqWub7zZcG8gkz6eMK6QI%2FuaBJu%2BBmQEYU%2FoRXX5kK5a%2FV4sNXWQxp3hk1rOBDlJeIg9pqb8WTAo1GrF8sTqCW7VLQQfx%2B5eMsg1iMZC1A5%2BsLNikB0IIWlozCanaA%2BdPPFme2Q9hk4egGpXKmOr69NYptJGTMcMlVEWcwlHokLW2cPqvH%2BZD7WbCZtA%3D%3D&webInteractiveContentId=232441011680&portalId=441834891)** ### Related reading - [Searching and Browsing Digital Newspaper Collections with Veridian](https://veridiansoftware.com/knowledge-base/searching-and-browsing-through-a-veridian-collection) — Learn more about Veridian’s search and browse features for digital newspaper collections, designed... - [How Does Veridian Use Cloudflare to Protect Digital Collections?](https://veridiansoftware.com/knowledge-base/how-veridian-uses-cloudflare-to-protect-digital-collections) — Veridian uses Cloudflare to help protect digital collections from scraping, cyberattacks, and other... - [Bot Wars 4.0: Protecting Digital Collections from AI](https://veridiansoftware.com/knowledge-base/bot-wars-4.0-protecting-digital-collections-from-ai) — Explore the evolution of the Bot Wars and how Veridian is tackling AI-powered mass scraping to... ## Our newsletter Keep up with digitization best practices, case studies, Veridian feature updates and more. Each month, straight to your inbox. Subscribe [![Facebook](https://veridiansoftware.com/hubfs/VeridianSoftware_July2025/images/facebook-white.svg "Facebook")](https://www.facebook.com/VeridianSoftware) [![LinkedIn](https://veridiansoftware.com/hubfs/VeridianSoftware_July2025/images/linkedin-white.svg "LinkedIn")](https://www.linkedin.com/company/dlconsulting-veridian-software) [![Veridian](https://veridiansoftware.com/hubfs/VeridianSoftware_July2025/images/veridian-logo-white.svg "Veridian")](https://veridiansoftware.com) - [About Us](https://veridiansoftware.com/about) - [Our Services](https://veridiansoftware.com/services) - [Projects](https://veridiansoftware.com/collections) - [Knowledge Base](https://veridiansoftware.com/knowledge-base) - [Contact Us](https://veridiansoftware.com/contact) © 2026 DL Consulting Ltd. All Rights Reserved. [Privacy Policy](https://veridiansoftware.com/privacy-policy/) ```json { "@context" : "https://schema.org", "@id" : "https://veridiansoftware.com/#organization", "@type" : "Organization", "legalName" : "DL Consulting Ltd", "name" : "Veridian Software", "url" : "https://veridiansoftware.com/" } ``` ```json { "@context" : "https://schema.org", "@id" : "https://veridiansoftware.com/#website", "@type" : "WebSite", "inLanguage" : "en-US", "name" : "Veridian Software", "publisher" : { "@id" : "https://veridiansoftware.com/#organization" }, "url" : "https://veridiansoftware.com/" } ``` ```json { "@context" : "https://schema.org", "@id" : "https://veridiansoftware.com/knowledge-base/scraping-digital-collection-usage-statistics#webpage", "@type" : "WebPage", "description" : "Learn why scraping inflates digital collection usage figures, how to estimate affected periods, and how access options can support clearer reporting.", "inLanguage" : "en-US", "isPartOf" : { "@id" : "https://veridiansoftware.com/#website" }, "mainEntity" : { "@id" : "https://veridiansoftware.com/knowledge-base/scraping-digital-collection-usage-statistics#article" }, "name" : "When Scraping Distorts Your Collection’s Usage Figures", "url" : "https://veridiansoftware.com/knowledge-base/scraping-digital-collection-usage-statistics" } ``` ```json { "@context" : "https://schema.org", "@id" : "https://veridiansoftware.com/knowledge-base/scraping-digital-collection-usage-statistics#article", "@type" : [ "BlogPosting", "TechArticle" ], "about" : [ { "@type" : "Thing", "name" : "Digital collection usage statistics" }, { "@type" : "Thing", "name" : "Web scraping" }, { "@type" : "Thing", "name" : "Usage estimation and reporting" }, { "@type" : "Thing", "name" : "Digital collection access management" } ], "audience" : { "@type" : "Audience", "audienceType" : [ "Librarians", "Archivists", "Digital Collection Managers", "Cultural Heritage Professionals" ] }, "author" : { "@id" : "https://veridiansoftware.com/#organization" }, "citation" : "https://www.cloudflare.com/en-gb/press/press-releases/2026/cloudflare-introduces-precursor-one-click-behavioral-defense-against-modern-bots", "datePublished" : "REPLACE_WITH_PUBLICATION_DATE_YYYY-MM-DD", "description" : "Learn why scraping inflates digital collection usage figures, how to estimate affected periods, and how access options can support clearer reporting.", "headline" : "When Scraping Distorts Your Collection’s Usage Figures", "image" : "REPLACE_WITH_ABSOLUTE_FEATURED_IMAGE_URL", "inLanguage" : "en-US", "isAccessibleForFree" : true, "keywords" : [ "digital collection usage statistics", "web scraping", "bot traffic", "Google Analytics", "usage reporting", "bulk downloads", "collection access" ], "mainEntityOfPage" : { "@id" : "https://veridiansoftware.com/knowledge-base/scraping-digital-collection-usage-statistics#webpage" }, "publisher" : { "@id" : "https://veridiansoftware.com/#organization" }, "url" : "https://veridiansoftware.com/knowledge-base/scraping-digital-collection-usage-statistics" } ```