Skip to main content

Pub/Sub quickstart

This quickstart guides you through creating a topic, attaching a subscription, publishing a message, pulling the message, acknowledging it, and cleaning up resources.

Prerequisites​

Before starting:

  1. Log in to your Crusoe AI Platform account using platformctl login.
  2. Ensure you have administrator access to your target project to create topics and subscriptions.
  3. For curl commands, set your environment variables:
export CAI_PUBSUB_API="https://api.codyhill.dev"
export CAI_PROJECT="<your-project-id>"
export CAI_TOKEN="<your-api-key-or-session-token>"

Step 1: Create a topic​

A topic is a named channel for your messages. This command creates a topic named orders with a 16 MiB storage allocation and an old discard policy when capacity is reached.

platformctl pubsub topics create orders --max-bytes 16Mi --discard old

Step 2: Create a subscription​

Subscriptions receive and retain messages published to a topic. You must create a subscription before publishing messages to ensure they are retained.

platformctl pubsub subscriptions create workers --topic orders \
  --type shared --ack-deadline-seconds 30 --start-from all

Step 3: Publish a message​

Publish a test message to the orders topic.

platformctl pubsub topics publish orders --message "hello world" --attribute region=eu

Output:

1234:0

Step 4: Pull and acknowledge the message​

Pull waiting messages from the subscription and acknowledge processing.

platformctl pubsub subscriptions pull workers --topic orders --max 10 --ack

Output:

ACK_ID    ID      KEY  DATA
a1b2c3d4  1234:0  -    hello world
acknowledged 1 message(s)

Step 5: Check storage quota​

View your project's storage usage and budget limits.

platformctl pubsub quota

Publishing from a deployed workload​

The commands above authenticate as you, against the public API host. A function or agent running on the platform does neither, and copying them into a handler fails in a way that points at the wrong thing.

  • Address: use the injected CAI_PUBSUB_URL with CAI_PROJECT_ID. Read the variables rather than hard-coding what they hold — that private address is the platform's to change. The public host https://api.codyhill.dev does not route from inside a project — a handler that reaches for it hangs until its own timeout.
  • Credential: use a service-account key with the member role, stored as a project secret and bound to the workload. member is enough to publish; creating topics and subscriptions needs admin.
  • Not the injected CAI_PROJECT_KEY. It holds no project authority — its one power is minting a token to read this project's secrets — and it is refused on every project route with 404 not found, not 403. So a wrong credential looks exactly like a misspelled topic name. Check the credential before you go hunting for the topic.
import json, os, urllib.request

PUBSUB = os.environ['CAI_PUBSUB_URL'].rstrip('/')
PROJECT = os.environ['CAI_PROJECT_ID']
KEY = os.environ['PIPELINE_KEY']      # the bound service-account key

req = urllib.request.Request(
    '%s/v1/projects/%s/topics/orders:publish' % (PUBSUB, PROJECT),
    data=json.dumps({'messages': [{'text': 'order-created'}]}).encode('utf-8'),
    headers={'Content-Type': 'application/json', 'Authorization': 'Bearer ' + KEY},
    method='POST')
urllib.request.urlopen(req, timeout=30)

Bind the secret, then Apply. A handler that reads the key at import time fails its first revision every time, because a bound secret only reaches the workload on the revision created after the Apply.

Cleanup​

Delete the topic when finished testing. Deleting a topic automatically removes associated subscriptions.

platformctl pubsub topics delete orders

Next steps​