Skip to main content

Indexes and points

This page is the complete data model for VectorDB. It covers what an index is, what a point is, and every setting you choose when you create one. It also covers which of those settings can never change afterward, which matters just as much.

The curl tabs use the same three variables as the quickstart. The platformctl tabs need none of them.

export VDB="https://api.codyhill.net"                       # VectorDB is served on the shared public API
export CAI_PROJECT="00000000-0000-0000-0000-000000000000"   # your project UUID
export CAI_TOKEN="<your session token or API key>"

Index​

An index is a named container for vectors. Everything in one index shares two things:

  • one fixed width (the number of dimensions in every vector), and
  • one distance metric (the rule for measuring how close two vectors are).

Other vector databases call this same thing a "collection" (Pinecone-style products) or an "index" (Vertex AI, Azure AI Search). Here, the resource you create and name is the index. The word collection means only the internal storage name, explained below.

Name your index​

An index name uses lowercase letters, digits, and hyphens. It has to start and end with a letter or digit, and it can be at most 48 characters long. Uppercase letters and underscores are refused. Written as a pattern, that is [a-z0-9]([a-z0-9-]*[a-z0-9])?. Real rejection messages:

{"error":"index name \"My_Index\" must match [a-z0-9]([a-z0-9-]*[a-z0-9])? - lowercase letters, digits and hyphens, starting and ending with an alphanumeric","request_id":"..."}
{"error":"index name \"...\" is longer than 48 characters","request_id":"..."}

The internal collection name​

Every read of an index reports a collection field, such as p_k3xq9m_docs. That is the storage name the platform built from your project plus your index name. Because your project is baked into it, two projects can both own an index named docs without colliding.

The platform shows you this name only so you can recognize your own data — for example, through direct-access credentials. No endpoint ever accepts it as input.

Dimensions​

dimensions is how many numbers each vector holds. It can be anywhere from 1 to 65536. Omit it, or send 0, and you get the width of the embedding model this platform hosts — so an index you create without thinking about it fits the vectors our /embeddings endpoint produces. Every vector you ever write must have exactly this many numbers.

Confirm the width for yourself with platformctl inference embed "hello", which prints a DIMENSIONS column. If you bring your own embeddings from somewhere else, set dimensions to match that model instead.

Choose it by reading your embedding model's documentation. The model's output size is your index's dimensions. Get it wrong and every write fails with a dimension-mismatch error (see the quickstart's failure example).

:::caution Dimensions are permanent Width is fixed when the index is created. Existing vectors cannot be re-embedded into a different width, so the only way to change it is to delete the index and build it again. Get it right the first time — or leave it blank and let the platform match its own model. :::

Out-of-range values are rejected at creation:

{"error":"dimensions must be between 1 and 65536, or omit it for the platform default","request_id":"..."}

Distance metrics​

The distance metric defines what "close" means. Case does not matter: Cosine and cosine are both accepted.

MetricAliasesWhat it measuresWhen to pick it
cosine (default)—The angle between vectors, ignoring their lengthThe usual choice for text embeddings; most embedding models are trained for it
dotdotproduct, dot_productThe inner product (direction and length)When your model's docs say to use dot product / inner product
euclideuclidean, l2Straight-line distance between the pointsWhen your model's docs say to use Euclidean or L2 distance

Anything else is rejected:

{"error":"distance must be one of \"cosine\", \"dot\" or \"euclid\"","request_id":"..."}

Advanced creation options​

These settings trade memory against speed and durability. The defaults are fine for most workloads, which is why every interface tucks them out of the way.

FieldDefaultWhat it does
on_diskfalseStore vectors on disk instead of RAM. Cheaper for big indexes, slower to search.
payload_on_diskfalseStore payloads on disk instead of RAM.
quantization.kindnoneCompress vectors to save memory at a small accuracy cost. scalar (alias int8) or binary.
quantization.quantile—Scalar-quantization tuning value, e.g. "0.99".
quantization.always_ramfalseKeep the compressed copy in RAM even when vectors are on disk.
shards1How many pieces the index is split into.
replicas1Replication factor — how many copies of each shard you are asking the engine to keep. The field accepts 1–8, but a value the engine cannot place is refused: see What replicas means, and what it does not.

Setting them looks like this — a large index whose vectors live on disk, compressed to int8:

platformctl vectordb create big-docs --dimensions 3072 \
  --on-disk --quantization scalar --quantile 0.99 --always-ram

--quantile and --always-ram mean something only alongside --quantization. Pass them on their own and the CLI refuses, rather than sending a request that would quietly build an unquantized index you cannot change afterward.

Invalid quantization kinds are rejected with:

{"error":"quantization.kind must be one of \"none\", \"scalar\" or \"binary\"","request_id":"..."}

Index lifecycle and status​

Creating an index returns HTTP 201 right away, with state: "pending". The API does not wait for the storage to be built. Keep re-reading GET .../indexes/{name} — every second or so is plenty — until it reports ready: true. Reads and writes before that point return 409 index is not ready yet; its collection has not been created.

FieldMeaning
stateCoarse phase: pending, creating, ready, degraded, conflict, or blocked
readyThe authoritative "you can use it now" boolean
collection_statusThe engine's own health word — green, yellow, red, or grey — present once storage exists
achieved_replicasHow many copies the engine has actually placed, read back from the index's real shard placement — not the number you asked for. Absent until the platform has read the placement, which is not the same as zero; compare it with replicas
points_count, vectors_count, segments_countLive size counters
messageHuman-readable explanation when something is wrong (for example, why a conflict happened)

What can change, and what can't​

Only two fields are editable after creation — the replica count and whether payloads live on disk — and changing either needs the project admin role.

platformctl vectordb update docs --replicas 1 --payload-on-disk

Only the flags you pass are sent, so setting the replica count cannot silently revert a payload placement someone else just set. Passing neither flag is an error rather than a request that changes nothing.

1 is not a typo. It is the only replica count this deployment can place today — see What replicas means, and what it does not.

replicas must be between 1 and 8, or you get replicas must be between 1 and 8 — and a value inside that range is refused as well when the deployment has too few places to put the copies, which today means anything above 1. What replicas means, and what it does not has the whole of it. Every other field is immutable — fixed for the life of the index. The API tells you why rather than just refusing:

Field you tried to changeExact 400 error
dimensionsdimensions is immutable: existing points cannot be re-embedded, so a different width is a new index, not an edit
distancedistance is immutable: the metric is baked into the graph built over the existing points
shardsshards is immutable: resharding moves data and is not performed on a spec edit
on_diskon_disk is immutable: vectors are not restaged between memory and disk in place
quantizationquantization is immutable: it is fixed when the collection is built
(empty patch)no mutable fields to change; the editable fields are replicas and payload_on_disk
There is no "resize"

To change dimensions, distance, or any other locked setting, create a new index and load your data into it yourself. The platform does not move points between indexes. Deleting an index (HTTP 204) permanently removes all points contained in it.

What replicas means, and what it does not​

This is a correction to what this page used to say. It described replicas as "how many copies exist". It was not that, and nothing checked it: the platform accepted any number from 1 to 8, stored it, reported it back to you unchanged, and left a single copy of your points as all there was. A replicas of 3 read back as 3 and bought you nothing. Two things have changed.

The number you ask for is now checked against what the deployment can place. A second copy of a shard is only a second copy if it lives somewhere else. This deployment has room for one, so one copy is all there is room for, and 1 is the only value a create or an update accepts today. Anything higher is refused at request time, on both routes, with a 409 naming the real number and how many copies would have had nowhere to live — rather than being accepted and quietly not done. Asking for 1 is never gated on any of this; it needs no second place to put it, so there is nothing to check. Asking for more when that number cannot be read is refused as well: an unread number is not a big number, and accepting a replication factor nobody can show to be satisfiable is the failure this replaces.

The index now reports what was actually achieved. achieved_replicas, on the GET and PATCH responses, is read from the index's real shard placement. replicas is your request, read back out of the index's stored configuration — which is why it could never disagree with you. Read achieved_replicas for how many copies of your data exist; read replicas for what was asked for. platformctl vectordb list and get now print the pair in one column, as 1 requested, 1 achieved, so the number you asked for never appears on its own.

The field is staying. It is bounded rather than removed, because the bound belongs to the deployment and not to the API: when the platform can hold more copies, the ceiling lifts by itself and a replicas above 1 becomes both accepted and real. Nothing on this page has to change for that to happen — and nothing here should be read as saying it already has.

shards is untouched by any of this and stays immutable, for the reason in the table above: resharding moves every point.

The console's index dialogs are unchanged and still offer a 1–8 replica setting. The check lives in the API, so a value the deployment cannot place is refused there too; there is no separate console control for any of this yet.

Points​

A point is one entry in an index. It has up to three parts:

{"id": 42, "vector": [0.1, 0.2, 0.3, 0.4], "payload": {"tag": "alpha", "source": "manual"}}

Vector (required)​

The list of numbers. Its length must equal the index's dimensions exactly.

Each point carries exactly one dense vector, meaning one plain list of numbers with a value in every slot. VectorDB does not support named vectors (several vectors attached to one point) or sparse vectors (the mostly-zero kind used for keyword-style scoring).

Id (optional)​

Either an unsigned integer — a whole number, zero or greater — or a UUID string, the 36-character random kind like 7f3c1e02-9b5d-4a11-8c6f-2d0e5a91b4cc. Nothing else is accepted. Omit the id and the platform generates a UUID for you.

Writing a point with an id that already exists overwrites that point instead of adding a second one. That update-or-insert behavior is why the write operation is called upsert.

Invalid ids are rejected:

{"error":"point 0: id must be an unsigned integer or a UUID string","request_id":"..."}
{"error":"a string id must be a UUID (or use an unsigned integer)","request_id":"..."}

Payload (optional)​

Any JSON metadata you want to keep alongside the point. Payloads are what make search useful in practice: at query time you can narrow results to points whose payload matches a condition (see Search).

Store whatever you will want to filter on or show to a user. Tags, source URLs, timestamps, and the original text the vector was made from are all common choices.

Write limits​

  • At most 1000 points per upsert request (at most 1000 points per request (got N)), and at least one (points must contain at least one point).
  • Batches are all-or-nothing: one bad point rejects the whole request; nothing is written.
  • Any request body is capped at 32 MiB.

Summary​

DecisionRule of thumb
NameLowercase, digits, hyphens; ≤ 48 chars; pick something you can grep for
DimensionsCopy your embedding model's output size; you cannot change it later
Distancecosine unless your model's docs say otherwise; you cannot change it later
IdsLet the platform generate UUIDs unless you need to overwrite points deterministically
PayloadStore everything you'll filter on or need to show in results
Advanced optionsLeave at defaults until memory cost forces the question

Next steps​

  • Search — how to query all of this.