Semantic search and RAG are creating new ways to explore digital collections — but they also introduce different infrastructure requirements. We look at what organizations should consider when planning for what comes next. 

For years, investment in digital collections has centered on digitization, metadata quality, and keyword search — helping users locate relevant material within growing collections. Now, approaches such as semantic search and retrieval-augmented generation (RAG) are creating new ways for users to explore and interact with digital content.

Supporting these capabilities relies not only on software, but also on the infrastructure behind it.

How digital collection search is changing

Traditional keyword search remains fundamental to digital collections, but newer discovery approaches can provide additional ways to explore large volumes of material.

Semantic search can identify content based on meaning rather than relying solely on matching words or phrases. This can help users discover relevant material even when their search terminology differs from the language used in the source material.

RAG takes this a step further by retrieving relevant information from a collection and using it as context for a generative AI model. This can support new ways of asking questions, exploring connections, and making sense of material across a collection.

These capabilities introduce different processing requirements from those traditionally associated with hosting and searching digital collections.

Why semantic and RAG search change infrastructure requirements

Semantic and RAG-based search can place significantly different demands on infrastructure.

Some of these capabilities rely on GPU-based processing for tasks such as generating embeddings, running models, and processing retrieval requests. Depending on the implementation and scale, this can require more specialized and powerful hardware than organizations have traditionally needed to host digital collection platforms.

Infrastructure planning may therefore need to consider:

• The processing requirements of semantic search, RAG, and other emerging discovery tools
• Whether GPU resources are required and at what scale
• Collection growth and the resources required for indexing
• Expected search and processing workloads
• Performance, reliability, and uptime requirements
• How easily computing resources can be expanded or upgraded

The challenge isn't simply providing more processing power. It is deciding what infrastructure to invest in while the technologies — and their requirements — are still developing.

Planning for requirements that are still evolving

Infrastructure is typically a long-term investment. An organization purchasing new hardware today may reasonably expect it to remain in service for five years or more.

Semantic search, RAG, AI models, and the hardware used to support them are evolving on a much shorter timescale. It can therefore be difficult to predict exactly what processing requirements a digital collection will have three to five years from now.

Investing heavily in hardware today could mean specifying infrastructure for requirements that subsequently change. Equally, planning only around current search and hosting requirements could make it more difficult to introduce new discovery capabilities later.

The aim isn't to anticipate every future requirement. Instead, organizations can consider how adaptable their infrastructure needs to be and how emerging discovery capabilities might affect decisions being made during current infrastructure planning and hardware refresh cycles.

Choosing an infrastructure approach

There is no single infrastructure model that will suit every digital collection.

For some organizations, owning and managing dedicated hardware may provide the most practical and cost-effective approach. Where workloads are reasonably predictable and the organization has the technical resources to manage its infrastructure, dedicated hardware can provide substantial computing capacity without ongoing cloud processing costs.

For others, cloud-based infrastructure can provide greater flexibility. Computing resources can be introduced, tested, or changed as requirements develop, without committing to particular hardware several years in advance.

Cloud infrastructure can also provide access to GPU resources when required, making it possible to support semantic and RAG workloads without necessarily owning the underlying hardware.

Neither approach is inherently the better option. The appropriate infrastructure depends on factors including the size and nature of the collection, expected workloads, discovery capabilities being introduced, available technical resources, performance requirements, and cost.

In some cases, a combination of approaches may also make sense, with different workloads supported by different infrastructure.

Planning for what comes next

Semantic and RAG search are already demonstrating new ways to explore digital collections — you only need to explore Elephind.com to see these discovery approaches in action.

For organizations planning infrastructure upgrades or hardware refreshes, emerging discovery capabilities are another factor to consider alongside existing hosting, performance, and budget requirements.

That doesn't necessarily mean moving to the cloud or investing immediately in specialized hardware. The appropriate approach will depend on the collection, expected workloads, available resources, and how an organization expects its discovery services to develop.

If you're beginning to think about what this means for your own collections, our team is happy to talk through the options and share what we're learning as these approaches continue to evolve.