Skip to main content

ObjectStore triggers (bucket events)

An ObjectStore trigger watches a bucket in any object store that speaks Amazon S3's API, and fires your function when new objects appear. The trigger works by polling: it lists the bucket on an interval and acts on what it sees. Nothing has to be configured on the storage side — no bucket notifications, no event bus.

A bucket trigger is never instant. poll_seconds defaults to 60, so an object can sit for up to a minute before your function runs. The usual first experience is a drop, twenty seconds of nothing, and the conclusion that the trigger is broken — it is not, it has not polled yet. The floor is 1 second. Leaving --poll-seconds off the CLI gives you 60, not 0: the flag's own default of 0 means "say nothing", and the server then applies 60.

This page is the function-side guide. Field-by-field reference is on triggers.

How it decides what to fire on​

The poller does not remember where it got to. It tracks progress by retiring each object it has handled — and how it retires is your choice, via after_read:

ValueBehavior
moveCopy the object to move_to, then remove it from the source. Keeps the data and delivers once.
deleteRemove it once delivered. Destructive, and says so.
noneLeave it — and fire again on every poll, forever. Fine only for a function that is idempotent (a repeat call with the same input changes nothing). A catastrophe for one that sends email.

There is deliberately no default — every possible default would be wrong for somebody, and leaving the field out is refused:

source.objectstore.after_read must be "move", "delete" or "none" - there is no default, because a bucket trigger that never retires an object fires on it forever

Choose none and the platform keeps reminding you: the trigger's No repeat delivery readiness check stays waiting and names the bucket, so the decision shows up on the trigger itself instead of on a bill weeks later. Choose move without a destination and that is refused too — an object moved onto itself re-delivers on every poll:

after_read: move needs move_to.bucket or move_to.prefix - moving an object onto itself re-delivers it on every poll

What your handler receives​

The body is the object itself, not a description of it. Each poll turns found objects into deliveries — one POST per object, up to max_messages_per_poll (default 10) — and the poller streams the object's bytes as the request body. A markdown file arrives as markdown, a CSV as CSV. There is no bucket field and no key field, and your function never talks to the bucket: it needs no S3 client and no credentials of its own.

Every delivery carries a ce-id header, so the shim treats it as a CloudEvent: the return value is discarded, success ACKs 204, and a handler exception becomes a terminal 400.

An object that is not JSON reaches a Python function only

Read this before you pick a runtime, because it decides whether this trigger works for you at all.

The Node.js, Go and Ruby shims run the body through a JSON parser and nothing else. A markdown file, a CSV, a log line or an image is refused before your handler runs:

{"error": "invalid JSON body: ..."}

The Go shim is stricter still: it parses into a map, so a JSON scalar or array is refused the same way.

That 400 is terminal on a CloudEvent delivery — no redelivery, and after_read: move or delete has already retired the object. So the drop is lost, your handler never ran, and the bucket looks like the work was done.

Only the Python shim handles a non-JSON body, and only Python exposes the CloudEvent attributes. Everything in the rest of this section — the four rules, event["data"], event["_cloudevent"] — is the Python shim's behavior. A JSON-object drop works from any runtime; anything else needs Python today.

The Python shim turns those bytes into the event dict by four rules:

What the object holdsWhat handle(event) gets
Text that is not JSON — markdown, CSV, a log line{"data": "<the text>"}
A JSON objectthe parsed object, at the top level: {"order_id": "o-123", ...}
A JSON scalar or array{"data": <the parsed value>}
Bytes that are not UTF-8 — an image, a zip{"data_base64": "<base64 of the bytes>"}

So a handler fed text drops reads event["data"]:

def handle(event: dict) -> dict:
    text = event.get("data")
    if text is None:                      # not UTF-8 text: an image or an archive
        print("skipped: object is not text")
        return {"statusCode": 200}
    print(f"ingesting {len(text)} characters")
    # ... process(text) ...
    return {"statusCode": 200}

None of those shapes is an error. A body that is not JSON reaches handle() like any other; the Python shim never fails a delivery for the object's format. That tolerance is exactly what the other three runtimes lack.

The CloudEvent attributes arrive under event["_cloudevent"]​

Every ce-* header on the delivery is handed to the handler as a dict under _cloudevent, with the prefix stripped. Matching is case-insensitive, so Ce-Id and ce-id both land as event["_cloudevent"]["id"]. A structured delivery (Content-Type: application/cloudevents+json) gets the same dict from the envelope's non-data keys instead. Every attribute is passed through rather than a chosen few, because which attribute carries an object's identity depends on the source.

Two things it deliberately does not do. If your payload already has a _cloudevent key, yours wins and no attributes are attached — losing a caller's data to platform metadata is the worse failure. And a plain HTTP request carries no ce-* headers, so an HTTP-invoked function sees no _cloudevent at all.

This is Python's alone. The Node.js, Go and Ruby shims read the body and discard the headers, so a handler in one of those runtimes has no route to the attributes and cannot see even the id.

The object's key is not delivered​

This is the trap that costs an afternoon. This source's subject is the literal string aws-s3-source — the name of the source, not the name of your object. It has no prefix and no extension, so a suffix check against it (if not name.endswith(".md")) rejects every file in the bucket. Nothing errors: the poller keeps retiring objects, the function keeps returning 200, and the work silently never happens.

One delivery from this source was measured carrying id, source, specversion, subject, time and type, and nothing that names the object. Treat that as a measurement, not a contract — print event["_cloudevent"] on your first delivery and read what your store actually sends:

def handle(event: dict) -> dict:
    print("attrs:", event.get("_cloudevent"))
    return {"statusCode": 200}

Believe an attribute is a key only when it looks like one: an object key contains a / or a .. When none does — the normal case here — name the object yourself:

  • Derive a name from the content. A markdown # H1, a CSV header, an id field. A heading names a document better than a generated id would, and it is what a citation ends up showing. Fall back to a hash of the content so two unnamed objects stay distinct.
  • Or write the identity into the object when you produce it, so the delivery carries it in the body.

Objects over 8 MiB never reach your handler​

Because the body is the object, the shim's 8 MiB request cap is a cap on object size. This one is not Python's alone — all four runtimes cap at the same 8 MiB. A larger object is refused with 413 before handle() runs:

{"error": "body exceeds 8388608 bytes"}

The Go shim words the same refusal {"error": "body too large or unreadable"}, so do not read that message as a different fault.

The delivery fails, and if after_read said move or delete the object has already been retired — so the drop is lost and nothing in the bucket says so. Keep large payloads out of a bucket trigger: drop a small manifest object that names the large one, and have the handler fetch it with an S3 client, using credentials you bind to the function yourself. Egress from a project to Crusoe object storage is open on 443, so that fetch connects.

A complete worked example​

Ingest CSV drops from an incoming/ prefix, then move them out of the way:

platformctl functions deploy ./my-function --name csv-ingest

platformctl serverless triggers create csv-drop \
  --target csv-ingest \
  --type objectstore \
  --bucket uploads \
  --endpoint https://object.us-southcentral1-a.crusoecloudcompute.com \
  --region us-southcentral1-a \
  --credentials-secret crusoe-object-store \
  --prefix incoming/ \
  --after-read move \
  --move-to-bucket uploads --move-to-prefix processed/

The CLI flag is --credentials-secret; the API field it fills is credentials_secret_name. The two spellings on this page are not a typo.

Fields, briefly​

FieldRequiredDefaultWhat it is
bucketYes—The bucket to watch.
endpointYes—The S3 endpoint. No default: the platform does not host the object store and cannot guess where it is.
credentials_secret_nameYes—Names a Secret in your project's own namespace. If the project is connected to Crusoe Cloud, the platform has already minted a real S3 key for it and stored it as crusoe-object-store (keys aws.accessKey and aws.secretKey) — use that name. It is not a Secrets Manager secret, and it is not your Crusoe Cloud API key: object storage answers that key with 403 The AWS access key Id you provided does not exist in our records. If the Secret is missing, press Check again on Project settings → Crusoe Cloud with a working connection and it is derived.
after_readYesnone — you must choosemove, delete, or none (above).
prefixNowhole bucketWatch only object names starting with this text — incoming/, say.
regionNous-east-1Must match the bucket's own region. The value goes straight to the poller's S3 client, so a mismatch fails at connect time, inside the poller, with nothing on the trigger to say why. Crusoe Cloud regions look like us-southcentral1-a, so the default is almost certainly wrong for a Crusoe bucket. The console's bucket picker fills this from the bucket you choose and overwrites what is already there; from the CLI, pass --region explicitly.
move_toWhen after_read=move—Destination bucket and/or prefix.
poll_secondsNo60How often the bucket is listed, minimum 1. This is your delivery latency: at the default, a drop can wait a minute.
max_messages_per_pollNo10Cap per poll, so 10,000 objects at once does not become 10,000 calls at once.
force_path_styleNotrueAddress the bucket as endpoint/bucket rather than bucket.endpoint.
eventsNo["created"]A poll can only observe objects that exist, so created is the meaningful value. A poll cannot observe a deletion.

Function-side notes​

  • Delivery is internal. The trigger uses the function's internal address, never its public URL — so an unpublished function is fireable, and traffic never leaves the platform's network.
  • Same-project only. A trigger fires a target in its own project; there is nowhere in the API to express a cross-project target.
  • Retry settings do not apply here. retry.max_attempts and retry.backoff reach schedule triggers (as the job's env) and Pub/Sub subscriptions (as the subscription's MaxDeliver). The bucket poller reads neither: it delivers once. The API accepts a retry block on an object-store trigger without complaint and hands it back on the next GET, so seeing it on the trigger is not evidence that anything reads it.
  • The 30-second attempt limit is not this path's. That number is the scheduled-trigger agent's own HTTP client timeout. A function's request budget is the platform's default 300 s, because a function's version is deployed with no request timeout of its own. Polling every 60 s while each object takes 90 s still backs deliveries up, so keep the handler short or make it enqueue and return.
  • suspend does not stop it. Suspension is wired to the scheduled job and the Pub/Sub subscription; the bucket poller is re-converged on every reconcile and keeps listing and firing while the trigger reports state: suspended. after_read stays in force too, so a bucket you believe is paused is still being drained by move or delete. Deleting the trigger is what stops it — recreate it from the same fields to resume.
  • A handler exception is terminal. CloudEvent rules: exception → 400 → never retried, and the object has already been retired if after_read said move or delete. A failed delivery is therefore a lost delivery. If the work can fail transiently, catch it inside the handler and write the object somewhere you can retry from, or run with after_read: none and your own idempotency check.

Next steps​

  • Triggers — the full field reference, the credentials secret mechanics, and readiness checks.
  • Language guides — per-runtime handler patterns.
  • Pub/Sub triggers — the push model, for comparison: ObjectStore polls; Pub/Sub pushes.