Topics and subscriptions

This guide details the Pub/Sub resource model: topic configurations, storage quota mechanics, retention rules, subscription types, and resource modification options.

Environment setup

Set the required environment variables for API requests:

export CAI_PUBSUB_API="https://api.codyhill.dev"
export CAI_PROJECT="<your-project-id>"
export CAI_TOKEN="<your-api-key-or-session-token>"

---

Topics: Named messaging channels

A topic is a named messaging channel where producers publish messages.

Topic settings

Property	Required	Default	Description
name	Yes	—	Immutable. Lowercase letters, digits, and hyphens (1–63 characters). Must start with a letter.
display_name	No	Empty	Editable human-readable label.
max_bytes	No	16Mi	Reclaimed storage budget reserved for this topic's unacknowledged backlog.
max_age	No	None	Maximum retention period for messages (e.g. 24h, 30m).
discard	No	old	Capacity overflow behavior: old drops the oldest unacknowledged messages; new rejects new publishes with HTTP 409.

Creating a topic

platformctl

platformctl pubsub topics create orders --display-name "Order Events" \
  --max-bytes 64Mi --max-age 24h --discard new

curl

curl -sX POST "$CAI_PUBSUB_API/v1/projects/$CAI_PROJECT/topics" \
  -H "Authorization: Bearer $CAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"orders","display_name":"Order Events","max_bytes":"64Mi","max_age":"24h","discard":"new"}'

Console UI

1. In the Crusoe Console, navigate to Messaging → Pub/Sub.
2. Click Create Topic.
3. Fill in the display name, storage limit (64Mi), max message age (24h), and overflow strategy (new).
4. Click Save.

---

Storage budget allocation

Each project has a Pub/Sub message storage budget (default 1 GiB).

• Claimed Budget: A topic claims its max_bytes allocation immediately upon creation, even when empty.
• Capacity Management: Deleting unused topics or lowering max_bytes frees storage budget for other topics.
• Monitoring Capacity: Check quota usage anytime via CLI, API, or Console.

platformctl

platformctl pubsub quota

curl

curl -s "$CAI_PUBSUB_API/v1/projects/$CAI_PROJECT/pubsub/quota" \
  -H "Authorization: Bearer $CAI_TOKEN"

Console UI

View the Quota Usage bar above the topic table in Messaging → Pub/Sub.

---

Subscriptions: Durable readers

Subscriptions track reader progress and retain unacknowledged messages for processing.

Subscription configuration

Property	Required	Default	Description
name	Yes	—	Unique identifier within the topic.
type	No	shared	Ordering and consumer model: shared, key-shared, exclusive, failover. Immutable after creation.
ack_deadline_seconds	No	30	Time window (1–600 seconds) for consumers to acknowledge pulled messages before redelivery.
max_deliver	No	5	Maximum delivery attempts before forwarding to a dead-letter topic (push subscriptions).
start_from	No	all	Initial read position: all (earliest available) or new (from subscription creation onwards). Immutable.
dead_letter	No	None	Target topic configuration for unprocessable messages: {"topic": "dead-letter-topic", "after_attempts": 5}.

Subscription ordering models

• shared: Competing consumer model. Messages are distributed across active workers for parallel processing.
• key-shared: Partitioned queue. Messages sharing the same key attribute are delivered in strict order to the same consumer instance.
• exclusive: Single active consumer stream with strict total ordering.
• failover: Primary single consumer stream with automatic failover to standby consumers.

Creating a subscription

platformctl

platformctl pubsub subscriptions create workers --topic orders \
  --type key-shared --ack-deadline-seconds 60 --max-ack-pending 500 --start-from new

curl

curl -sX POST "$CAI_PUBSUB_API/v1/projects/$CAI_PROJECT/topics/orders/subscriptions" \
  -H "Authorization: Bearer $CAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"workers","type":"key-shared","ack_deadline_seconds":60,"max_ack_pending":500,"start_from":"new"}'

Console UI

1. Open the orders topic page.
2. Click Create Subscription.
3. Select key-shared type, enter deadline 60 seconds, set start point to new, and click Create.

---

Updating resource settings

Dynamic topic and subscription settings can be modified in place. Immutable properties (name, type, start_from) require deleting and recreating the resource.

Updating a topic limit

platformctl

platformctl pubsub topics update orders --max-bytes 128Mi

curl

curl -sX PATCH "$CAI_PUBSUB_API/v1/projects/$CAI_PROJECT/topics/orders" \
  -H "Authorization: Bearer $CAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"max_bytes":"128Mi"}'

Console UI

1. Select Edit on the topic row.
2. Adjust the max bytes value and save changes.

---

Summary best practices

1. Pre-create Subscriptions: Ensure subscriptions exist prior to publishing messages so no events are missed.
2. Configure Dead-Letter Topics: Set up dead-letter targets on push subscriptions to capture and isolate unprocessable messages.
3. Select Appropriate Subscription Types: Use shared for parallel task queues and key-shared for entity-ordered stream processing.