Skip to main content

Search

This page covers every way to read and prune data in a VectorDB index: similarity queries, payload filters, score thresholds, paging through raw points, and deleting points. Each pattern comes with a runnable request and its expected output. If the words vector and embedding are new, start with the overview.

Every operation is shown three ways where all three surfaces have it — platformctl, curl, and the console. The curl tabs need the setup from the quickstart; the platformctl tabs need none of it, because the CLI resolves the address and your project itself.

export VDB="https://api.codyhill.dev"                      # VectorDB is served on the shared public API
export CAI_PROJECT="00000000-0000-0000-0000-000000000000"  # your project UUID
export CAI_TOKEN="<your session token or API key>"

Every example assumes an index named docs with 4 dimensions containing:

{"points":[
  {"id":1,"vector":[0.1,0.2,0.3,0.4],"payload":{"tag":"alpha"}},
  {"id":2,"vector":[0.9,0.8,0.7,0.6],"payload":{"tag":"beta"}}]}

Search for nearest neighbors​

A query takes a vector and returns the points closest to it, best first.

The technical name for this is ANN search, short for approximate nearest neighbor. It is "approximate" because the engine does not compare your query against every stored vector. It follows a shortcut structure built over the data instead. You give up a tiny bit of accuracy and get back a lot of speed.

platformctl vectordb query docs --vector '[0.1,0.2,0.3,0.4]' --top-k 5

You should see:

ID  SCORE   PAYLOAD
1   1.0000  {"tag":"alpha"}
2   0.8427  {"tag":"beta"}

--vector accepts a literal JSON array, @file to read a file, or - to read standard input, so any body written for the curl tab pipes straight in. Add -o json for the full response, including vectors when you asked for them.

Request fields​

FieldTypeDefaultNotes
vectorarray of numbersrequiredMust match the index's dimensions exactly
top_kinteger10How many nearest points to return; max 1000
filterJSON objectnonePayload filter (below); max 64 KiB
with_payloadbooleantrueInclude each hit's payload
with_vectorbooleanfalseInclude each hit's stored vector
score_thresholdnumbernoneDrop hits scoring below this value
offsetinteger0Skip the first N hits (must not be negative)

Scores​

Every hit carries a score, computed with the index's distance metric. Results come back best first. With cosine, a vector identical to a stored one scores a perfect 1.0. That is why the quickstart's exact-match query returns "score":1.0.

Let the engine drop weak matches for you with score_threshold, rather than fetching them and discarding them in your own code:

platformctl vectordb query docs --vector '[0.1,0.2,0.3,0.4]' \
  --top-k 10 --score-threshold 0.9

You should see only the strong match:

ID  SCORE   PAYLOAD
1   1.0000  {"tag":"alpha"}

There is no Console tab here: the query panel offers a vector, Top K, and a payload filter, and no score threshold. Filter in the browser, threshold from the CLI or the API.

Second stage: rerank the candidates​

Vector search is fast and blunt, and its scores are often too close together to threshold on. Measured on a real corpus: the right passage scored 0.5468 against a wrong one at 0.4262. Running the same two candidates through the platform's cross-encoder reranker moved them to 0.9489 and 0.0619 — a gap you can act on.

The pattern is: retrieve wide, rerank, keep few.

# 1. retrieve about 4x what you need, at least 20 candidates
platformctl vectordb query docs --vector @q.json --top-k 20 -o json

# 2. score the candidates' text against the same query - one argument per candidate
platformctl inference rerank "how do I reset my password" \
  "<candidate 1 text>" "<candidate 2 text>"

A reranker reads the query together with each document rather than comparing two vectors that were embedded separately, which is why it is sharper — and why it costs one model pass per candidate. That is affordable over 20 candidates and not over the whole index, which is exactly why it is a second stage.

Step 2 needs the candidates' text, and a query returns ids, scores, and payloads — never the source text unless you stored it. Put the text in the payload when you upsert, or the second stage has nothing to read.

Results come back with an index — the position of the document in the array you sent — and a relevance_score. Reorder by score and map each result back to your own record through index; do not use the position in results. The CLI already sorts for you and prints SCORE, INDEX, DOCUMENT.

The platform's ADK agent memory bank runs this two-stage retrieval internally. Over an index you built yourself, nothing reranks for you — call /v1/rerank from your own code. See Inference for the endpoint and its response shape.

Narrow results with a payload filter​

A filter answers questions like "nearest neighbors, but only among documents tagged alpha". It narrows the search to points whose payload matches a condition.

A filter is a JSON object holding one or more clauses. Each clause holds a list of conditions, and a condition names a payload key plus the test to apply to it:

platformctl vectordb query docs --vector '[0.5,0.5,0.5,0.5]' --top-k 5 \
  --filter '{"must":[{"key":"tag","match":{"value":"beta"}}]}'

You should see:

ID  SCORE   PAYLOAD
2   0.9891  {"tag":"beta"}

--filter takes the same three forms --vector does: a literal, @file, or - for standard input.

Filter syntax​

Clauses — combine them in one filter object, and a point must satisfy every clause you list:

ClauseMeaning
mustEvery condition in the list has to hold (logical AND).
shouldAt least one condition in the list has to hold (logical OR).
must_notNone of the conditions may hold (logical NOT).

Conditions — each entry in a clause list is an object with a key naming a payload field, plus one test:

TestShapeMatches
Exact value{"key":"tag","match":{"value":"beta"}}Payload field equals that string, number, or boolean.
One of several{"key":"tag","match":{"any":["beta","gamma"]}}Field equals any value in the list.
None of several{"key":"tag","match":{"except":["draft"]}}Field equals none of the listed values.
Numeric range{"key":"score","range":{"gte":0.5,"lt":1.0}}Field falls in the range. Any of gt, gte, lt, lte, alone or combined.

Nested payload fields are addressed with dots: {"key":"meta.author","match":{"value":"ada"}}. A clause list may also contain a nested filter object, so you can express "A and (B or C)".

A worked example — published documents tagged beta or gamma, scoring at least 0.5, excluding anything by ada:

{"must":[{"key":"published","match":{"value":true}},
         {"key":"score","range":{"gte":0.5}},
         {"should":[{"key":"tag","match":{"value":"beta"}},
                    {"key":"tag","match":{"value":"gamma"}}]}],
 "must_not":[{"key":"meta.author","match":{"value":"ada"}}]}

Filter rules:

  • The filter must be a JSON object (400 filter must be a JSON object).
  • It is capped at 64 KiB (400 filter is too large).
  • A payload key that no point carries is not an error — it simply matches nothing.
  • Some filter mistakes are caught by the search layer rather than by the platform's own validation. Those come back as 400 vector database rejected the request: <message>, with the underlying explanation passed through word for word, so you see the real problem instead of a generic message.

Browse without a query: scroll​

Scroll lists points in id order and needs no vector at all. It is the "just show me what's in here" operation, and it is what the console's Data browser uses.

platformctl vectordb scroll docs --limit 50

You should see:

ID  PAYLOAD
1   {"tag":"alpha"}
2   {"tag":"beta"}

When another page remains, the CLI prints the cursor as the flag you hand back:

more points: --offset 2
  • limit defaults to 50, max 1000 (400 limit must be at most 1000).
  • Scrolled points carry payloads but never vectors.
  • Paging works by bookmark, not by page number. When more points remain, the response includes next_offset, which is a point id. Send it back as offset on your next request to pick up where you stopped. When next_offset is absent, you have seen everything. A malformed offset gets 400 offset must be an unsigned integer or a UUID string.

Delete points​

Delete points by listing their ids, or by writing a filter that matches them. Send exactly one of the two, never both:

platformctl vectordb delete-points docs --ids 1

You should see:

ids_requested  1
index          docs
status         ok

Or by filter:

platformctl vectordb delete-points docs \
  --filter '{"must":[{"key":"tag","match":{"value":"beta"}}]}'

--ids is repeatable and also accepts a comma-separated list. Passing both --ids and --filter, or neither, is refused before any request is made.

Rules:

  • Provide exactly one of ids or filter (400 provide exactly one of ids or filter).
  • At most 1000 ids per request (400 at most 1000 ids per request).
  • An empty filter is deliberately refused, because it would match everything:
{"error":"an empty filter would delete every point; delete the index instead if that is the intent","request_id":"..."}

Common errors at a glance​

Symptom (verbatim error)CauseFix
index is not ready yet; its collection has not been createdQuerying right after createPoll GET .../indexes/{name} until ready: true
query vector has N dimensions; index "docs" expects MQuery vector from the wrong embedding modelEmbed queries with the same model you indexed with
top_k must be at most 1000Asking for too many resultsPage with offset, or rethink why you need 1000+ neighbors
filter is too largeFilter over 64 KiBSimplify the filter; move logic into payload design
offset must not be negativeNegative offset in a queryUse 0 or a positive integer

Next steps​