Forage
In-memory Search made Easy
Forage is a Java library that helps you build an in-memory search index from data residing in your database or persistence layer. Powered by Apache Lucene, it provides full-text search capabilities without the overhead of a dedicated search infrastructure.
Why Forage?¶
Consider a scenario where your primary datastore holds a modest amount of data, yet you require sophisticated search capabilities. Would you really want to provision an entire search engine — such as a dedicated Elasticsearch or Solr cluster for this purpose? Alternatively, would you resort to cluttering your database with secondary indexes, only to find that true free-text search remains out of reach?
Whichever path you choose, a few problems are almost inevitable:
- Overkill: Deploying a full-scale search infrastructure is often disproportionate for many use cases, particularly when your dataset consists of only a few tens of thousands of rows.
- Expensive: Beyond the hardware and cloud costs of hosting a search engine, you must also architect a pipeline to synchronize data from your primary datastore. Maintaining this consistency adds significant overhead.
- Latencies: Modern search engines are fast—especially on well-provisioned hardware — but you still pay the price of network hops, no matter how optimized the setup is.
Forage aims to address these issues by building a lightweight search index directly in the memory of each application
node.
As long as you can pull all the required data from your primary datastore, Forage makes it easy to spin up a quick,
in-memory search engine that continuously stays in sync with the latest state of your database.
Zero Infrastructure¶
No need for dedicated Elasticsearch or Solr clusters. Search runs in your application's memory.
Ultra-Low Latency¶
Eliminate network hops entirely. Search queries execute in microseconds, not milliseconds.
Cost Effective¶
No additional hardware or cloud resources required. Perfect for small to medium datasets.
Full Lucene Power¶
Access the full power of Lucene - fuzzy matching, phrase queries, boolean logic, and more.
Features¶
| Feature | Description |
|---|---|
| Full-Text Search | Powerful text analysis with tokenization, stemming, and fuzzy matching |
| Boolean Queries | Combine queries with AND, OR, NOT, and FILTER logic |
| Range Queries | Filter by numeric or date ranges |
| Function Scoring | Customize ranking with field values, scripts, decay functions |
| Phrase Matching | Match exact phrases in text fields |
| Prefix Matching | Autocomplete-style prefix searches |
| Pagination | Efficient cursor-based pagination for large result sets |
| Dropwizard Integration | First-class support for Dropwizard applications |
When to Use Forage¶
Forage is ideal when:
Your dataset is small to medium (up to ~1 million documents)- You need low-latency search responses
- You want to avoid operational overhead of search infrastructure
- Your data fits comfortably in application memory (at least the fields you want indexed)
Consider alternatives when:
Your dataset exceeds available heap memory- You need distributed search across multiple nodes
- You require real-time indexing of high-velocity data streams
- You don't have means to periodically pull all data from your primary datastore
Getting Started¶
Ready to add powerful search to your application?
Quick Start Guide Read the Overview
Tech Stack¶
- Java 17+
- Apache Lucene 9.12.3
- Dropwizard 2.1.0 (optional)
License¶
Forage is released under the Apache License 2.0.