Weaviate vs. OpenSearch vs. pgvector

Introduction

When your exertion needs search, the evident choices are devices you whitethorn already run, specified arsenic OpenSearch for full-text hunt aliases PostgreSQL pinch the pgvector hold for applications already built connected relational data. Both tin shop and query vectors, but neither was designed pinch vectors arsenic the superior information structure. OpenSearch added vector hunt arsenic a plugin connected apical of an inverted-index engine. pgvector adds nearest-neighbor hunt arsenic a SQL operator, which is useful erstwhile a vector is 1 file among many, but it is not built for semantic hunt astatine the halfway of your product.

Weaviate takes the other stance. The HNSW (Hierarchical Navigable Small World) vector scale is the halfway information structure, and BM25 keyword hunt and hybrid search are built connected apical of it alternatively than added afterward. The applicable consequence is that semantic search, keyword search, and the blend of some are each a azygous API call. DigitalOcean Managed Weaviate provisions, backs up, and patches the cluster for you, truthful you get those capabilities without the operational overhead.

To make the comparison concrete, this article loads the aforesaid corpus of podcast transcripts into each 3 DigitalOcean managed services and compares them connected what really distinguishes them for search. Those dimensions are the value of the results (vector vs keyword vs hybrid retrieval), the magnitude of hunt logic that lives successful your application, ingest speed, and scale size. Raw query velocity is not a focus. At this standard each 3 respond comparably fast, accelerated capable that velocity is not a deciding factor.

Key Takeaways

• Hybrid hunt wins connected value for each engine, and only Weaviate runs it arsenic a azygous API call. Combining vector and keyword retrieval hit either unsocial successful each 3 systems, pinch Weaviate’s hybrid scoring highest (hit@10 0.725, followed by OpenSearch astatine 0.685 and Postgres astatine 0.665). On OpenSearch and pgvector, hybrid requires 2 abstracted queries positive a rank-fusion measurement you constitute and maintain.
• Vector-search value is group by the embeddings, not the database. With each engine’s candidate-list extent aligned, each 3 scored wrong hit@10 0.575–0.600 connected the aforesaid vectors.
• The operational edges disagree by engine. Weaviate ingested fastest because its HNSW and BM25 indexes build together successful a azygous pass, while pgvector was the astir compact (a 1.87 GB scale vs OpenSearch’s 2.91 GB) but needs deliberate query building for its keyword hunt to perform.

Methodology

For the dataset, we utilized a corpus of podcast-episode transcripts pinch 4,886 unsocial episodes and chunked into ~500-token passages, producing 100,000 documents, each carrying the transition matter positive section metadata. Embeddings were generated erstwhile pinch OpenAI’s text-embedding-3-small (1,536 dimensions) and cached, truthful the embedding measurement is identical for each 3 systems.

All 3 ran arsenic DigitalOcean Managed Databases, queried from a DigitalOcean Droplet.

Two kinds of scale look passim this array and the remainder of the article. HNSW (Hierarchical Navigable Small World) is simply a graph-based scale for vector search. It links each embedding to its adjacent neighbors successful a layered graph, truthful the motor finds the vectors closest to a query by hopping done the chart alternatively than comparing against each 100,000, which makes it accelerated but approximate. An inverted scale is the classical keyword-search structure. It maps each connection to the database of passages that incorporate it, overmuch for illustration the scale astatine the backmost of a book maps position to pages, and BM25 is the modular look for ranking those matches by really uncommon a connection is and really concentrated it is successful a passage. Postgres fills the aforesaid 2 roles pinch pgvector’s HNSW implementation for vectors and a GIN (Generalized Inverted Index) complete tsvector, its autochthonal full-text type, for keywords.

This is not a controlled hardware benchmark. The instances were not spec-matched, truthful it deliberately avoids head-to-head velocity claims. The attraction is consequence value and the engineering differences that persist sloppy of hardware, namely really each strategy retrieves and really overmuch hunt logic you must write.

To measurement retrieval quality, we randomly sampled 200 passages from the dataset and had an LLM (large connection model) make a natural-language mobility circumstantial to each one. We past utilized those generated questions arsenic our hunt queries to spot whether the transition that produced each mobility appeared successful the apical 10 results. A deed intends the root transition came backmost successful the apical 10 (hit@10), and we besides recorded its mean reciprocal rank (MRR), which scores higher the person the transition is to the apical of the list. Recall@10 is measured against an nonstop brute-force nearest-neighbor hunt complete the aforesaid vectors. Index size is the on-disk footprint aft the afloat load.

For scale configuration, Weaviate and pgvector usage HNSW pinch default parameters, while OpenSearch uses the k-NN plugin pinch the Lucene HNSW motor and cosine similarity. Vectors are pre-computed and identical crossed systems. To support the engines comparable, each 3 were configured the measurement a practitioner would configure each for English text. Weaviate’s keyword usability is technically BM25F, which tin weight aggregate fields, but searching a azygous contented section reduces it to modular BM25, the aforesaid algorithm OpenSearch runs. OpenSearch uses the english analyzer (stopword removal and stemming, matching what Postgres’s nation config and Weaviate’s tokenization already do) and was force-merged aft loading, per the k-NN plugin’s guidance. The HNSW candidate-list extent (ef_search) was aligned astatine 100 for each 3 engines, which is Weaviate’s default.

Ingest and Loading

Embeddings were pre-computed, truthful loading performed the aforesaid occupation successful each 3 systems, inserting 100,000 passages and building a vector scale positive a keyword scale complete them. Since the instances were not spec-matched, the nonstop times don’t transportation to your hardware, but the ordering (Weaviate fastest, past Postgres, past OpenSearch) follows from really each motor builds its indexes, and that does transfer.

• Weaviate ingests and indexes successful a azygous pass. As each entity arrives complete its gRPC batch endpoint, the HNSW chart node and the BM25 inverted-index introduction are created together, truthful erstwhile the load finishes the postulation is afloat searchable. There is nary abstracted indexing measurement to run, schedule, aliases hold for.
• Postgres separates loading from indexing. Bulk inserts (execute_values) onshore rows quickly, past the HNSW and GIN indexes are built arsenic definitive follow-up steps. On this corpus of mini passages the scale builds were brief. On larger corpora the HNSW build becomes the ascendant cost, and it’s a measurement you own. You bid it, show it, and retrieve it connected each reload.
• OpenSearch pays its indexing costs many times during the load. Its k-NN structures are rebuilt arsenic segments merge passim ingestion, truthful scale attraction is interleaved pinch the bulk load and re-paid passim it, making it the slowest of the 3 here. Tuning (larger segments, deferred refresh) tin retrieve immoderate of that time, which is again logic that lives connected your side.

The shape to return distant is that the quality isn’t earthy insert speed, it’s wherever the index-building activity happens, whether inline and invisible (Weaviate), arsenic a deliberate post-load measurement (Postgres), aliases amortized crossed the load itself (OpenSearch).

Hybrid Search: One Call vs. Two

Hybrid search, which blends semantic (vector) and keyword (BM25) relevance, is some the highest-quality attack (as the value conception shows) and wherever the architectural differences betwixt these systems are astir visible. The existent costs is what your exertion has to own.

Weaviate runs hybrid hunt natively successful 1 call. It queries the HNSW and BM25 indexes together, fuses them pinch Reciprocal Rank Fusion (RRF), and returns 1 classed list. The alpha parameter controls the blend (0 = axenic keyword, 1 = axenic vector).

results = collection.query.hybrid( query=query_text, vector=query_vector, alpha=0.5, limit=10, )

OpenSearch has nary autochthonal hybrid query successful this configuration. You rumor a vector query and a keyword query, past fuse the classed lists successful your application.

knn = client.search(index="passages", body={ "size": 50, "query": {"knn": {"embedding": { "vector": query_vector, "k": 50, "method_parameters": {"ef_search": 100}}}}}) bm25 = client.search(index="passages", body={ "size": 50, "query": {"match": {"transcript_text": query_text}}}) def rrf(*result_lists, k=60): # you write, test, and support this scores = {} for results in result_lists: for rank, deed in enumerate(results): scores[hit["_id"]] = scores.get(hit["_id"], 0) + 1 / (k + rank + 1) return sorted(scores, key=scores.get, reverse=True)[:10] results = rrf(knn["hits"]["hits"], bm25["hits"]["hits"])

pgvector has nary hybrid primitive either, truthful the aforesaid two-ranking fusion is expressed successful SQL.

WITH v AS (SELECT guid, row_number() OVER (ORDER BY embedding <=> %(qv)s) rk FROM passages ORDER BY embedding <=> %(qv)s LIMIT 50), k AS (SELECT guid, row_number() OVER (ORDER BY ts_rank(tsv, to_tsquery('english', %(q)s)) DESC) rk FROM passages WHERE tsv @@ to_tsquery('english', %(q)s) LIMIT 50) SELECT p.guid, COALESCE(1.0/(60+v.rk),0) + COALESCE(1.0/(60+k.rk),0) AS rrf_score FROM passages p LEFT JOIN v USING (guid) LEFT JOIN k USING (guid) WHERE v.guid IS NOT NULL OR k.guid IS NOT NULL ORDER BY rrf_score DESC LIMIT 10;

Beyond information trips, the two-ranking attack creates a attraction surface. The fusion logic lives successful your code, must beryllium tested, and must enactment accordant crossed each work and connection that queries search. With Weaviate, tuning the blend is 1 float. Everywhere other it is simply a smoothing changeless wrong a usability you own.

Search Quality: Vector vs Keyword vs Hybrid

Using the 200 generated questions described successful the methodology, each motor was scored connected whether the root transition appeared successful its apical 10 (hit@10) and connected its mean reciprocal rank (MRR).

Three findings guidelines out.

• Hybrid hit some vector and keyword connected hit@10 for each engine. This is the azygous astir actionable result. If hunt value matters, blend the 2 modalities.
• Vector value is fundamentally identical crossed each 3 engines (hit@10 0.575–0.600), since they clasp the aforesaid embeddings and, pinch campaigner extent aligned, fetch them pinch near-equal callback (0.967–0.988). The database does not make the vectors amended aliases worse. It tin only win aliases neglect astatine fetching them.
• Weaviate had the champion hybrid and the champion wide consequence (hit@10 0.725, MRR 0.504), reflecting its beardown autochthonal fusion and BM25.

Postgres full-text hunt scored 0.04 pinch plainto_tsquery, which requires each connection successful a mobility to match. Rewriting the query to OR its position (to_tsquery pinch |) lifted it to 0.550. OpenSearch and Weaviate behave good by default, but pinch Postgres you must conception the matter query deliberately.

What the Numbers Look Like for One Query

Aggregate scores are abstract, truthful present is what the systems really return for a existent question, “how to grip rejection successful sales”. Passages are mid-conversation transcript chunks, truthful they commencement mid-sentence.

Vector hunt returned identical apical threes connected each 3 engines because they clasp the aforesaid embeddings. The strongest reply comes from an question and reply pinch Daniel Pink astir his income book To Sell Is Human:

“…gonna get rejected, there’s nary mobility astir it. So 1 of the qualities that I talk astir is simply a value called buoyancy… each time I look an water of rejection. Okay, that’s what income is like… So buoyancy is really do you enactment afloat successful that water of rejection?”

Your prime of database does not alteration what semantic hunt finds. The embedding exemplary does. The database only affects really reliably it fetches the correct vectors (recall).

Keyword hunt agrees astatine the apical for this query, pinch each 3 engines ranking the aforesaid 2 passages first, a income veteran’s communicative astir starting retired acold calling doorway to doorway and an Art of Charm (Jordan Harbinger) listener mobility astir cold-call anxiety. The divergence shows astatine the 3rd position, wherever each motor fills the slot differently, though each enactment connected topic.

The divergence is not ever that gentle. On different of the 200 eval questions, “What should a presenter do if their assemblage starts leaving during their session?”, only Weaviate returned the root transition the mobility was generated from, and the different 2 engines agreed connected the aforesaid miss.

Field weighting and researcher choices, not the BM25 algorithm, determine whether keyword hunt stays connected topic. Keyword hunt counts words alternatively of knowing them, truthful what you fto it count matters. Hybrid hunt fuses the 2 signals and, for this query, lands each 3 engines connected the aforesaid apical three, the income veteran’s cold-calling story, the Daniel Pink buoyancy passage, and the cold-call worry question. That is the shape the aggregate array shows. Vector hunt is accordant crossed engines, keyword hunt is wherever they diverge, and hybrid is the astir reliable for a existent query.

Index Size

On-disk footprint aft loading each 100,000 documents pinch 1,536-dimensional embeddings positive transition matter and metadata:

pgvector is the astir compact, a genuine advantage erstwhile vectors are 1 file successful an existing table. OpenSearch’s larger footprint comes partially from retaining the earthy embedding array successful each document’s _source alongside the k-NN graph. (Weaviate maintains an HNSW chart positive an inverted index, but the managed work does not presently expose a disk figure, truthful it is omitted alternatively than estimated.)

When to Use Each

Weaviate is the correct prime erstwhile semantic hunt aliases retrieval-augmented procreation (RAG) is the superior workload. Native hybrid search, the highest ingest throughput, and the champion retrieval value successful this comparison make it the least-maintenance action erstwhile hunt is cardinal to the product. The alpha parameter replaces an full fusion usability and the exertion logic astir it. DigitalOcean Managed Weaviate provisions successful minutes pinch nary scale guidance successful your code.

OpenSearch is the correct prime erstwhile you already tally an OpenSearch cluster for full-text hunt aliases log analytics and want to adhd vector hunt incrementally, aliases erstwhile you request its mature faceting, aggregations, and horizontal scale. It performs respectably, but hybrid hunt is client-side orchestration, and its k-NN motor prime meaningfully affects callback and tuning, truthful get that determination correct early.

PostgreSQL + pgvector is the correct prime erstwhile vectors are secondary to a relational schema, specified arsenic a passages aliases products array wherever ORDER BY embedding <=> $1 is 1 information among galore WHERE clauses and JOINs, wrong a transaction. It was the astir compact here, and it keeps vectors accordant pinch the information they describe. Its trade-offs are keyword hunt that needs deliberate query building and hybrid fusion you definitive yourself successful SQL.

Conclusion

All 3 systems shop and query vectors competently, and astatine this standard they reply astir arsenic fast. The differences that matter show up successful the value of what comes backmost and successful really overmuch hunt logic you person to build. Hybrid retrieval is the highest-quality attack crossed the board, pgvector needs deliberate query building to make keyword hunt work, OpenSearch needs deliberate researcher and callback configuration to compete, and Weaviate delivers the apical value pinch the slightest codification and the champion defaults.

For applications wherever hunt is simply a characteristic alongside relational data, pgvector avoids a caller work and keeps vectors adjacent to the information they describe, astatine the costs of building keyword and hybrid behaviour yourself. For applications wherever hunt is the product, the operational overhead of implementing hybrid hunt connected OpenSearch aliases pgvector (two query paths, a fusion function, accordant tuning crossed each client) adds up, and Weaviate replaces each of it pinch 1 telephone and 1 parameter, while delivering the champion retrieval value successful this comparison.

• DigitalOcean Managed Weaviate Documentation
• Choosing Between OpenSearch, Weaviate, and pgvector
• Weaviate Hybrid Search Reference