--- title: Code Apps operator runbook description: Operate Cuitty Code Apps, the App Market, SpiceDB, and Airflow in self-hosted deployments. section: Code order: 41 updatedAt: 2026-06-09 slug: code/operator-runbook --- This runbook covers local and self-hosted operation for Code Apps, the App Market, SpiceDB authorization, and Airflow execution. Cuitty Registry now has a separate [Registry operator runbook](/docs/registry/operator-runbook) for package protocols, indexes, upload sessions, and artifact storage. ## Required services | Service | Purpose | Local default | | --- | --- | --- | | Cuitty Code API | REST API, Git HTTP, app, and callback endpoints | `http://localhost:4351` | | Cuitty Code frontend | Browser UI | `http://localhost:4350` | | Cuitty Auth | OAuth/OIDC sign-in | `http://localhost:7705` | | Database | Persistent metadata and run state | Deployment-specific | | SpiceDB | Relationship authorization | `http://localhost:50051` | | Airflow | App run orchestration | `http://localhost:8080` | | Cuitty Registry | Optional standalone package registry for app artifact workflows | Deployment-specific | | Object storage | Optional backing storage for large artifacts and logs | Deployment-specific | ## Startup order 1. Start the database. 2. Start SpiceDB. 3. Load the SpiceDB schema. 4. Start Cuitty Auth. 5. Start Airflow. 6. Run Cuitty Code API migrations, then start the API. 7. Start the Cuitty Code frontend. 8. Start Cuitty Registry separately if Code App workflows need package publish or install targets. ## Environment variables For self-hosted production, commit Safe references instead of raw secret values. The reference syntax is `cuitty-safe:account/safe/secret`; only the resolver sees the decrypted value. ```dotenv PUBLIC_API_URL=https://code.example.com PUBLIC_CUITTY_AUTH_URL=https://auth.example.com AUTH_ISSUER=https://auth.example.com AUTH_CLIENT_ID=cuitty-code CUITTY_PUBLIC_URL=https://code.example.com CUITTY_CODE_SECRET_KEY=cuitty-safe:acme/prod/cuitty-code-secret-key SPICEDB_ENDPOINT=https://spicedb.example.com:50051 SPICEDB_PRESHARED_KEY=cuitty-safe:acme/prod/spicedb-preshared-key AIRFLOW_URL=https://airflow.example.com AIRFLOW_USERNAME=airflow-service AIRFLOW_PASSWORD=cuitty-safe:acme/prod/airflow-password CUITTY_APP_EXECUTION_MODE=airflow CUITTY_APP_RUN_CALLBACK_SECRET=cuitty-safe:acme/prod/app-run-callback-secret REGISTRY_STORAGE_URL=s3://cuitty-registry/prod ``` For an isolated local development stack only, use disposable placeholder values and local endpoints: ```dotenv PUBLIC_API_URL=http://localhost:4351 PUBLIC_CUITTY_AUTH_URL=http://localhost:7705 AUTH_ISSUER=http://localhost:7705 AUTH_CLIENT_ID=cuitty-code CUITTY_PUBLIC_URL=http://localhost:4350 CUITTY_CODE_SECRET_KEY= SPICEDB_ENDPOINT=http://localhost:50051 SPICEDB_PRESHARED_KEY= AIRFLOW_URL=http://localhost:8080 AIRFLOW_USERNAME=airflow AIRFLOW_PASSWORD= CUITTY_APP_EXECUTION_MODE=airflow CUITTY_APP_RUN_CALLBACK_SECRET= PUBLIC_REGISTRY_URL=http://localhost:4371 REGISTRY_STORAGE_URL=file:///var/lib/cuitty/registry ``` ## Migrations Run migrations before traffic reaches a new API build: ```bash cargo run -p code-server -- migrate ``` Verify the API starts with the expected database URL and no pending migration errors. ## SpiceDB checks ```bash zed schema write spicedb/schema.zed zed permission check user:alice read repository:acme/demo zed permission check user:alice install app_listing:acme/review-bot ``` Inspect failed authorization relationship writes: ```sql SELECT id, operation_kind, resource_kind, resource_id, status, attempts, last_error FROM authz_outbox WHERE status IN ('pending', 'processing', 'failed_retryable', 'failed_terminal') ORDER BY created_at ASC LIMIT 50; ``` ## Registry integration checks If Code Apps publish or install packages, run Cuitty Registry separately and verify Code discovers it through `PUBLIC_REGISTRY_URL` or product endpoint discovery. Use the [Registry operator runbook](/docs/registry/operator-runbook) for npm, Cargo, PyPI, and OCI smoke checks. ## Airflow checks Submit or trigger an app run, then confirm the run has an Airflow DAG ID and DAG run ID. Reconcile submitted runs when callbacks are delayed: ```bash curl -X POST http://localhost:4351/api/v1/apps/runs/reconcile -H "Authorization: Bearer $CUITTY_TOKEN" ``` Callback failures usually mean the timestamp is stale, the `x-cuitty-signature` HMAC is wrong, or Airflow is signing with a value that does not match the resolved `CUITTY_APP_RUN_CALLBACK_SECRET` reference. ## Playwright E2E From the Cuitty Code `tests` directory, run the mocked app-market and integration specs: ```bash bun run test:e2e:apps-registry -- --project=chromium bun run test:e2e:apps-registry:headed ``` The specs set browser auth state and mock API responses by path, so they cover both direct API calls on `http://localhost:4351/api/v1/...` and frontend-proxied calls on `http://localhost:4350/api/v1/...`. ## Failure modes and recovery - SpiceDB unavailable: pause security-expanding app, package-target, and transfer writes; restore SpiceDB; drain the authz outbox. - Authz outbox backlog: inspect `last_error`, reset stale `processing` rows to `pending` only after confirming no dispatcher is active, and restart the dispatcher. - Airflow DAG submission failure: check `AIRFLOW_URL`, credentials, DAG ID derivation, API logs, and Airflow scheduler/webserver logs. - App run stuck in `submitted` or `running`: poll Airflow for the DAG run ID, check task logs, verify callback delivery, and run reconciliation. - Registry integration unavailable: keep Code App package workflows queued or failed retryable, verify `PUBLIC_REGISTRY_URL`, then run Registry smoke checks before replaying work. ## Related pages - [Registry operator runbook](/docs/registry/operator-runbook) - [Cuitty Code Apps](/docs/code/apps) - [Code App Market](/docs/code/app-market) - [Code integration](/docs/registry/code-integration) - [Authorization with SpiceDB](/docs/code/authz-spicedb) - [App execution with Airflow](/docs/code/app-execution-airflow)