Skip to main content

Platform considerations and guidance

This page details key operational behaviors, default limits, and architectural characteristics across the Crusoe AI Platform. Each section explains how a feature behaves, why it works that way, and the recommended approach for your applications.

If you encounter unexpected errors not covered here, consult the troubleshooting guide or follow the steps in get help.

Identity and access management​

Session tokens expire after 12 hours​

Behavior. User sign-in session tokens use cryptographic signatures and remain valid for 12 hours after issue. Signing out clears the token from your web browser. Account suspension or user removal updates permission checks immediately for all subsequent API calls.

Best practice. For background scripts and automated CI/CD pipelines, use API keys assigned to service accounts. API keys can be revoked instantly at any time. For details, see service accounts and API keys.

Password authentication defaults​

Behavior. Account authentication uses email addresses and complex passwords (minimum 12 characters combining letters, numbers, and symbols).

Best practice. Generate unique, long passwords using a password manager. Create dedicated service accounts for machine workloads rather than sharing human user accounts.

Network and endpoints​

Workloads are private until you publish them​

Behavior. Every workload receives a canonical platform domain name in the format https://<name>-<project-short>.apps.codyhill.dev, but that address serves nothing to the internet until the workload is published: workloads are private by default, reachable only from inside the platform. platformctl deploy says so rather than implying the address answers — it prints hello-http is ready (private). Publish it to serve on https://....

Best practice. Publish through the gateway, naming how the endpoint is protected. There is deliberately no default: an open endpoint and a protected one are both consequential, and the API answers 400 naming auth.mode if you leave the choice out.

platformctl gateway publish function/hello-http --auth none

Use --auth apikey to have a key minted for you, or --auth jwt with your own issuer. For details, see publish an endpoint.

Custom domains are claimed in the project, not proxied​

Behavior. The platform registers custom domain names itself, verifies each with a single DNS record, and provisions TLS. A reverse proxy or CDN in front of the platform URL is not required — and a proxied record actively breaks it, because proxying blocks the automated TLS validation.

Best practice. Claim the address, create the one CNAME record the command prints, then publish onto it:

platformctl gateway domain claim api.yourdomain.com
platformctl gateway publish function/hello-http \
  --domain api.yourdomain.com --auth none

The record is checked about once a minute. Until it appears, the address is claimed but not usable and an endpoint published on it exists and serves nothing; both start working on their own once the record shows up. Use a subdomain (api.example.com) — the top level of a domain cannot be pointed here with a CNAME, and wildcards are not supported. On Cloudflare, leave the record DNS Only (grey cloud). For details, see custom domains.

Control plane resiliency​

Behavior. The control plane API runs across multiple replicas for high availability. In-flight build tasks maintain state in active build workers.

Best practice. Management operations are safe to retry. If a deployment operation pauses during a control plane restart, re-issuing the deploy command initializes a fresh build claim. Active, deployed workloads serve traffic independently of control plane management operations.

Compute and autoscaling​

Cold starts on idle workloads​

Behavior. Workloads automatically scale to zero when idle to conserve compute resources. The first request after an idle period boots a new instance, which takes slightly longer to respond. If a heavy initialization step exceeds 60 seconds, the request may time out. Retrying the request succeeds immediately because the instance is now warm.

Best practice. For production workloads requiring instant response times, set minimum instances to 1. You can configure this in the console (Configure modal) or via the API (PATCH /v1/agents/{name}/config). For details, see autoscaling and scale to zero.

Serverless revision quotas​

Behavior. Each revision of a serverless service reserves two slots from your project's Services quota budget. Deploying multiple updates to the same service consumes quota slots for each revision.

Best practice. Monitor service quota usage under Quotas in the console. Clean up unused services or request a quota increase prior to large production deployments. For details, see quotas and audit.

Explicit max_scale configuration​

Behavior. Setting max_scale: 0 in an API call signifies unlimited autoscaling scaling. Unset fields default to a safe maximum of 10 instances.

Best practice. Always set your desired maximum instance ceiling explicitly (such as max_scale: 5 or max_scale: 10).

Internal network scoping​

Behavior. Private internal services are accessible only from within the platform's private network space, keeping them off the public internet.

Best practice. If your internal service requires user-level authorization, validate request headers or API tokens inside your application handler.

Large archive uploads require CLI deployment​

Behavior. Web browser file uploads cap source archives at 32 MiB. The core platform API accepts source archives up to 100 MiB.

Best practice. Deploy applications larger than 32 MiB using the platformctl CLI tool:

platformctl deploy ./my-agent --name my-agent

For details, see install the CLI.

Registry mapping for Crusoe Container Registry (CCR)​

Behavior. Linking a project to Crusoe Cloud provisions a registry pull secret (ccr-pull) based on the container repositories present at mapping time. If repositories are created after the initial mapping, or if credential tokens expire after extended periods, workload image pulls may fail.

Best practice. Re-save your Crusoe Cloud connection credentials in Project Settings or execute a mapping update API call whenever new repositories are added:

curl -sS -X PUT "$CAI_API/v1/projects/$CAI_PROJECT/crusoe-cloud" \
  -H "Authorization: Bearer $CAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"access_key_id":"'"$CC_ACCESS_KEY_ID"'","secret_key":"'"$CC_SECRET_KEY"'"}'

Re-mapping updates credentials cleanly without affecting running workloads. For details, see Crusoe Cloud integration.

Function execution models​

Behavior. Functions handle incoming HTTP requests and CloudEvents automatically. To execute functions on a cron schedule or Pub/Sub event, attach a trigger.

Best practice. Use scheduled triggers or Pub/Sub triggers to execute functions automatically. For details, see HTTP and events.

Object store triggers: the credential, the region, and the first revision​

Behavior. A bucket trigger polls an S3-compatible store using a credential you name in credentials_secret_name. Four behaviors account for most first-run failures:

  • A Crusoe Cloud API key is not an S3 key. Object storage answers one with 403 The AWS access key Id you provided does not exist in our records. The platform mints a real S3 key from the project's Crusoe Cloud connection and stores it as the project secret crusoe-object-store, with the fields aws.accessKey and aws.secretKey that the poller reads. That secret name is what belongs in credentials_secret_name.
  • The credential is minted once. A project that already holds a complete credential is left alone, because every mint is a durable key in your Crusoe Cloud account. If the project has none, Check again on Project Settings re-derives it.
  • The trigger's region must match the bucket's. The region is passed straight through to the poller, so a mismatch surfaces as a connection failure rather than a readable validation error.
  • It polls, and poll_seconds defaults to 60. A dropped object can take a minute to fire. Nothing here is instant.

Best practice. If your handler reads a bound secret at import time, expect the workload's first revision to fail — the binding cannot exist before the workload does, so that revision is built without the value. That is the normal sequence, not a fault: deploy, bind the secret, then run platformctl secrets bindings apply <workload>, and the revision the apply rolls has the value. Choose after_read (move, delete, or none) deliberately; it has no default and a blank is refused, because a trigger that never retires a consumed object re-reads it on every poll forever. For details, see object store triggers.

Agent sessions and memory​

Isolation models for sessions and memory​

Behavior. Agent sessions and long-term memory use dedicated security boundaries:

  • Sessions: Conversation histories are isolated in per-project MemoryStore environments (ms-sys-sessions.<project>).
  • Memory bank: Agent memory stores vectors in VectorDB using per-agent scoped authentication credentials (jwt_rbac). Each agent accesses only its assigned collections.

Best practice. Use agent memory for structured facts and facts relevant to your AI agent workflows. For specialized data isolation requirements, provision a dedicated VectorDB index or MemoryStore instance within your project. For details, see use MemoryStore with agents and use VectorDB with agents.

Agent memory scope​

Behavior. An agent's memory bank persists facts across separate user sessions for that agent.

Best practice. Store general facts and knowledge in memory. Avoid storing private user credentials in public memory banks. For details, see memory.

Endpoint authentication​

Behavior. Managing workloads (deploying, inspecting secrets, viewing logs, or deleting resources) requires authentication. Enforce caller authentication on agent endpoints by enabling invoke authentication in your project settings.

Best practice. Use platformctl login or API tokens for authenticated access. When exposing sensitive endpoints to specific external consumers, publish them through the gateway with --auth apikey or --auth jwt. For details, see invoke and gateway publish.

Data services​

Persistence and data protection​

Behavior. Data services store data continuously to persistent disk volumes. Deleting a VectorDB index or MemoryStore instance permanently removes its storage volume.

Best practice. Keep original source documents and dataset files stored safely in object storage so you can re-ingest data whenever needed.

MemoryStore server setup​

Behavior. A MemoryStore instance runs as one high-performance instance with Redis-compatible persistence — there is no replica to fail over to. Disk snapshots ensure data survives a restart.

Best practice. Choose an appropriate eviction policy when creating an instance. For caching workloads, set "maxmemory_policy": "allkeys-lru" so old keys clear automatically when memory fills up. For session storage, keep the default noeviction policy. For details, see MemoryStore overview.

Immutable vector index settings​

Behavior. Five index settings are fixed at creation time: dimensions, distance, shards, on_disk, and quantization. replicas and metadata payload configurations remain editable.

Best practice. Select your vector embedding model before creating an index. To change dimension counts, create a secondary index, migrate your vectors, and delete the original index. For details, see Indexes and points.

External database endpoints​

Behavior. MemoryStore, VectorDB, and Pub/Sub support external access endpoints. Access is protected by strong authentication tokens, TLS encryption, and configurable IP allowlists.

Best practice. For internal application components, connect using high-speed internal network URLs. Enable external endpoints only when remote access is explicitly required, and configure IP allowlists for added security. For details, see IP allowlists.

Messaging (Pub/Sub)​

Pub/Sub storage reservation​

Behavior. Creating a topic reserves its configured max_bytes from your project's Pub/Sub storage budget immediately upon creation.

Best practice. Size topics according to expected backlog needs. Deleting unused topics or reducing max_bytes returns reserved budget to your project. For details, see publish and consume.

Ordered subscription types​

Behavior. exclusive and failover subscriptions enforce single-consumer message ordering.

Best practice. Use push delivery for ordered subscriptions. For multi-consumer load balancing, use shared or key-shared subscriptions. For details, see topics and subscriptions.

Dead-letter topic configuration​

Behavior. On push subscriptions, failed deliveries retry automatically according to backoff rules.

Best practice. Configure a dead-letter topic on push subscriptions to store messages that fail repeatedly, ensuring no data is lost. For details, see topics and subscriptions.

Real-time messaging responses​

Behavior. Message publishing returns immediate message IDs upon confirmation. Topic background metrics refresh periodically.

Best practice. Rely on publish API response IDs for immediate transaction confirmation.

CAI_PROJECT_KEY cannot publish to Pub/Sub​

Behavior. Every deployed workload is injected with CAI_PROJECT_ID and CAI_PROJECT_KEY. That credential is refused on project APIs by design — the refusal lives in the tenancy resolver every service shares, Pub/Sub included, not in one service's handler. Its single capability is minting a short-lived token to read its own project's secrets.

Best practice. To publish to Pub/Sub from a function or agent, create a service account, store its API key as a project secret, bind it to the workload, and apply the binding:

platformctl secrets bindings set my-function CAI_TOKEN --secret publisher-key
platformctl secrets bindings apply my-function

For details, see service accounts and API keys and publish and consume.

Direct broker token renewal​

Behavior. Direct broker connection tokens use configurable expiration periods (30 days by default).

Best practice. Refresh direct broker tokens prior to expiration. Token renewal is seamless and does not interrupt active connections. For details, see publish and consume.

Secrets management​

Workload secret delivery​

Behavior. Applying secret bindings reads each bound secret at its bound version, writes the values into the workload's own secret, and rolls a new revision so the running workload picks them up.

Best practice. Deliver values with platformctl secrets bindings apply <workload> — the command lives under the bindings group, not directly on secrets. For ultra-sensitive workloads, fetch secrets dynamically at runtime using short-lived workload tokens; that is the one thing a workload's injected CAI_PROJECT_KEY is allowed to do. For details, see use secrets in workloads.

Binding and applying secrets​

Behavior. Binding a secret records intent and changes nothing that is running. platformctl secrets bindings apply <workload> is what delivers the value and rolls a new revision. Rotating a secret is the same story: a new version exists, and no running workload sees it until an apply. Removing a binding is the exception — an unbind is a revocation, so it takes the variable off the workload and rolls a revision immediately.

Best practice. Always run platformctl secrets bindings apply <workload> after creating a binding or rotating a secret a binding tracks. For details, see manage secrets.

Project model API keys​

Behavior. Projects configure a shared model API key (console → Project Settings, or PUT /v1/projects/{id}/inference). Individual workloads can override this key by setting MODEL_API_KEY in their secrets.

Best practice. Set a project-level model API key for general use, or assign per-agent secrets for custom billing tracking. For details, see secrets and env.

Observability and monitoring​

Live metrics and status tracking​

Behavior. Real-time status reporting covers active instances, revision states, MemoryStore hit rates, and sandbox availability.

Best practice. Stream live logs using platformctl logs <name> -f or query historical logs with platformctl logs <name> --history.

Audit log tracking​

Behavior. Every configuration change, deployment, and security action is recorded in your project's append-only audit log.

Best practice. Review audit history in the console under Audit Log or retrieve entries via platformctl audit (one page per run; --limit accepts up to 200, and the printed token goes back as --page-token for older entries). For details, see break-glass and audit.

Source code control​

Behavior. Deployed code archives represent active release artifacts.

Best practice. Maintain primary source code in a Git repository (such as GitHub or GitLab) and deploy builds directly from your repository. For details, see files and editor.