Skip to main content

Functions + Pub/Sub integration

This integration turns a message queue into a worker pipeline. Pub/Sub transports the event and the Function processes it. The platform handles delivery and retries, and dead-letters a message that keeps failing — once you have configured a dead-letter topic. Without one, a message that exhausts its delivery attempts is dropped.

Use it when​

  • work arrives asynchronously,
  • each message is independent,
  • you want retries and a dead-letter topic without writing your own scheduler.

The pieces​

  • Pub/Sub topic — the durable channel for events.
  • Push subscription — calls your function with each message.
  • Function — the worker.
  • Dead-letter topic — where repeated failures go. It has to exist before the subscription names it, and it must not be the subscription's own topic.

Step 1: Deploy the worker​

Write a function that accepts the event payload and returns quickly. Keep it deterministic and idempotent when you can.

def handle(event):
    order_id = event.get("order_id", "unknown")
    total = event.get("total", 0)
    print(f"processing order {order_id} for {total}")
    return {"ok": True}

Deploy it like any other function.

A published JSON object arrives as the event itself: the shim unwraps the CloudEvent envelope — including the base64 data_base64 field every Pub/Sub delivery uses — before calling handle(). A message that is not JSON arrives as event["data"].

Two consequences worth knowing before the first failure:

  • The return value is discarded. A push delivery is a CloudEvent, so the shim ACKs with an empty 204 and throws the result away. Do the work inside handle().
  • Any non-2xx is a failed attempt. A handler exception becomes a 400, which the push worker counts as an attempt and retries with exponential backoff (1 s, 2 s, 4 s, capped at 60 s) until the message is dead-lettered or dropped.

Step 2: Create the topics​

Create a work topic and a dead-letter topic.

platformctl pubsub topics create orders --max-bytes 16Mi --discard old
platformctl pubsub topics create orders-dead --max-bytes 16Mi --discard old

Step 3: Wire the subscription​

Create a push subscription that targets the function and dead-letters repeated failures.

$FN_URL is the function's internal address, not its public endpoint. Push delivery is made from inside the platform and is validated against your own project, so the target must be the private http:// address the platform assigns the function. Paste the public HTTPS endpoint instead and it is refused twice over — first for the scheme, with a message saying https has no trust story for internal delivery yet, and then for the host:

deliver.push.url is outside this project. A push target must be a workload in this project, and the address to use is the one the platform already gives you: the "url" field on that workload, which 'platformctl status <workload>' prints.

Neither refusal points at the real mistake, which is using the public endpoint. Copy that address rather than assembling it by hand, then append the port and path your handler listens on if they are not the defaults:

export FN_URL=$(platformctl status order-worker -o json | jq -r .url)
# an internal http:// address, reachable only from workloads in this project

If you do not need to own the subscription's settings, a Pub/Sub trigger does this wiring for you: it resolves the function's address itself and creates a shared push subscription that starts from new messages. Build the subscription by hand, as below, when you want to choose the type, the start point, or the ack deadline.

platformctl pubsub subscriptions create to-worker \
  --topic orders \
  --type shared \
  --push-url "$FN_URL" \
  --max-deliver 3 \
  --dead-letter-topic orders-dead \
  --dead-letter-after-attempts 3

Step 4: Publish a message​

Publish one test message and watch the function log the delivery.

Successful delivery looks like a normal function invocation. Repeated failure lands in orders-dead after three attempts, carrying the original payload plus forensic attributes — cai-dead-letter-reason (target responded 400, say), cai-dead-letter-attempts, cai-dead-letter-subscription and the source topic. That is the point of the dead-letter topic: a poison message stops costing attempts and becomes something you can read.

Operational notes​

  • The defaults, when you leave a field out: max_deliver 5, ack_deadline_seconds 30, max_ack_pending 1000, and — only if you name a dead-letter topic — after_attempts 5. There is no dead-letter topic unless you configure one, and without one a message that exhausts max_deliver is dropped, not parked.
  • ack_deadline_seconds is also the push request timeout. At the default, a handler that takes longer than 30 seconds is cut off and the message is redelivered — and that budget has to cover a scale-to-zero cold start, not just the work.
  • A function should be idempotent because redelivery is possible: a slow handler is retried while the first attempt may still be running.
  • Fail fast if the payload is malformed; do not turn a bad payload into a long retry loop. A permanent failure is worth dead-lettering on the first or second attempt.
  • Keep the payload small enough that retries stay cheap. The function shim refuses a body over 8 MiB with 413 body exceeds 8388608 bytes.

Next steps​