Ship durable Effect agents on GCP.

00

Copy-paste the project

Every box below is a real shell command. Replace anything in <ANGLE_BRACKETS> once. The flow takes about 25 minutes the first time.

1. A

   Install prerequisites

   Versions known to work: Node 22+, pnpm 9+, Pulumi 3.130+, Docker, gcloud. We install pnpm directly because Node no longer treats corepack as a supported way to ship a package manager.

   setup.shlocal machine

   # macOS — node@22 keeps you on the current LTS line
   brew install node@22 docker
   brew install pulumi/tap/pulumi google-cloud-sdk
   npm install -g pnpm@9    # install pnpm directly, do not use corepack
   
   # Linux (Debian/Ubuntu)
   curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
   sudo apt install -y nodejs docker.io
   curl -fsSL https://get.pulumi.com | sh
   curl https://sdk.cloud.google.com | bash
   sudo npm install -g pnpm@9
   
   # Sanity check — every version should print
   node --version       # v22.x
   pnpm --version       # 9.x
   docker --version
   pulumi version
   gcloud --version

2. B

   Create the GCP project

   Replace PROJECT_ID with something globally unique. Billing account id comes from gcloud billing accounts list.

   gcp-project.shone-time

   export PROJECT_ID=<effect-agents-PROJECT>
   export REGION=us-central1
   export BILLING_ACCOUNT=<XXXXXX-XXXXXX-XXXXXX>
   
   gcloud auth login
   gcloud projects create "$PROJECT_ID" --name="Effect agents"
   gcloud config set project "$PROJECT_ID"
   gcloud billing projects link "$PROJECT_ID" \
     --billing-account "$BILLING_ACCOUNT"
   
   gcloud services enable \
     container.googleapis.com \
     artifactregistry.googleapis.com \
     sqladmin.googleapis.com \
     secretmanager.googleapis.com \
     vpcaccess.googleapis.com \
     servicenetworking.googleapis.com \
     iam.googleapis.com \
     monitoring.googleapis.com \
     logging.googleapis.com
   
   gcloud auth application-default login

3. C

   Scaffold the repo

   The capstone (module 23) is the full shape. Start with just enough to deploy a single runner.

   scaffold.shrepo bootstrap

   mkdir effect-agents && cd effect-agents
   pnpm init
   pnpm add effect @effect/ai @effect/ai-openai \
     @effect/cluster @effect/workflow \
     @effect/platform @effect/platform-node \
     @effect/rpc @effect/sql @effect/sql-pg
   pnpm add -D typescript tsx vitest @effect/vitest
   
   mkdir -p apps/agent-runtime/src apps/agent-gateway/src \
     packages/domain/src packages/event-store/src \
     infra/components

   The smallest runtime that actually boots, joins the shard ring, and serves a health probe. Replace with the full agent runtime once it works.

   apps/agent-runtime/src/main.tsrunnable

   import { Entity, RunnerAddress, Rpc } from "@effect/cluster"
   import { HttpRouter, HttpServer, HttpServerResponse } from "@effect/platform"
   import {
     NodeClusterSocket,
     NodeHttpServer,
     NodeRuntime
   } from "@effect/platform-node"
   import { PgClient } from "@effect/sql-pg"
   import { Config, Effect, Layer, Logger, Option, Schema } from "effect"
   import { createServer } from "node:http"
   
   // A tiny entity that proves the cluster is wired correctly. Every
   // call to Echo(text) for a given entityId routes to the same runner.
   const Echo = Rpc.make("Echo", {
     payload: { text: Schema.String },
     success: Schema.String
   })
   
   const EchoEntity = Entity.make("Echo", [Echo])
   
   const EchoEntityLive = EchoEntity.toLayer(
     Effect.gen(function* () {
       const address = yield* Entity.CurrentAddress
       return EchoEntity.of({
         Echo: ({ payload }) =>
           Effect.succeed(`[${address.entityId}] ${payload.text}`)
       })
     })
   )
   
   // Probes for the StatefulSet from module 09.
   const HealthLive = HttpRouter.empty.pipe(
     HttpRouter.get("/healthz", HttpServerResponse.text("ok")),
     HttpRouter.get("/readyz", HttpServerResponse.text("ready")),
     HttpServer.serve()
   )
   
   const podHost = Config.string("POD_NAME").pipe(
     Config.map((name) =>
       `${name}.effect-agent-runners.agents.svc.cluster.local`
     ),
     Config.withDefault("localhost")
   )
   
   const ClusterLive = Layer.unwrapEffect(
     Effect.gen(function* () {
       const host = yield* podHost
       return NodeClusterSocket.layer({
         storage: "sql",
         runnerHealth: "k8s",
         runnerHealthK8s: {
           namespace: "agents",
           labelSelector: "app=effect-agent-runtime"
         },
         shardingConfig: {
           runnerAddress: Option.some(RunnerAddress.make(host, 50000)),
           runnerListenAddress: Option.some(
             RunnerAddress.make("0.0.0.0", 50000)
           )
         }
       })
     })
   )
   
   const SqlLive = PgClient.layerConfig({
     url: Config.redacted("DATABASE_URL")
   })
   
   const HttpLive = NodeHttpServer.layer(createServer, { port: 8080 })
   
   const MainLive = Layer.mergeAll(EchoEntityLive, HealthLive).pipe(
     Layer.provide(HttpLive),
     Layer.provideMerge(ClusterLive),
     Layer.provideMerge(SqlLive),
     Layer.provide(Logger.json)
   )
   
   Layer.launch(MainLive).pipe(NodeRuntime.runMain)

   apps/agent-runtime/Dockerfilenode:22-slim, no corepack

   # syntax=docker/dockerfile:1.7
   
   # Node 22+ no longer guarantees a working corepack, so we install
   # pnpm directly with npm and pin the version. Multi-stage keeps the
   # production image small and free of build tooling.
   
   FROM node:22-slim AS base
   ENV PNPM_HOME=/pnpm PATH=/pnpm:$PATH
   RUN npm install -g pnpm@9.15.0
   WORKDIR /app
   
   # --- deps: lockfile + manifests only, maximises layer cache hits
   FROM base AS deps
   COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
   COPY apps/agent-runtime/package.json apps/agent-runtime/
   COPY packages packages
   RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
       pnpm install --frozen-lockfile
   
   # --- build: compile TypeScript
   FROM deps AS build
   COPY tsconfig.json ./
   COPY apps/agent-runtime apps/agent-runtime
   RUN pnpm --filter agent-runtime build
   
   # --- runtime: only what node needs to run
   FROM node:22-slim AS runtime
   ENV NODE_ENV=production NODE_OPTIONS=--enable-source-maps
   RUN npm install -g pnpm@9.15.0
   WORKDIR /app
   COPY --from=build /app/node_modules ./node_modules
   COPY --from=build /app/packages ./packages
   COPY --from=build /app/apps/agent-runtime/dist apps/agent-runtime/dist
   COPY --from=build /app/apps/agent-runtime/package.json apps/agent-runtime/
   USER node
   EXPOSE 8080 50000
   CMD ["node", "apps/agent-runtime/dist/main.js"]

4. D

   Pulumi project and stack

   Run inside infra/. Pick TypeScript. Stack name is the environment; start with dev.

   pulumi-init.shinfra/

   cd infra
   pulumi new gcp-typescript --force \
     --name effect-agents-infra \
     --description "Effect agents on GCP"
   
   pulumi stack init dev
   pulumi config set gcp:project "$PROJECT_ID"
   pulumi config set gcp:region "$REGION"
   pulumi config set --secret postgresPassword \
     "$(openssl rand -hex 24)"
   pulumi config set --secret openaiApiKey "<sk-...>"

5. E

   Build and push the runtime image

   Tag with the git sha — never latest. Artifact Registry repo name matches what Pulumi creates next.

   build.shrepo root

   export SHA=$(git rev-parse --short HEAD 2>/dev/null || echo dev)
   export IMAGE="$REGION-docker.pkg.dev/$PROJECT_ID/agents/agent-runtime:$SHA"
   
   gcloud auth configure-docker "$REGION-docker.pkg.dev" --quiet
   
   docker build -t "$IMAGE" -f apps/agent-runtime/Dockerfile .
   docker push "$IMAGE"
   
   cd infra
   pulumi config set agentRuntimeImage "$IMAGE"

6. F

   Pulumi up

   First apply creates VPC, Cloud SQL HA, GKE Autopilot, Artifact Registry, Secret Manager, namespace, RBAC, StatefulSet, gateway Service. Expect 10–15 minutes.

   deploy.shinfra/

   pulumi preview        # review the plan
   pulumi up --yes       # apply
   
   pulumi stack output gatewayUrl
   pulumi stack output cloudSqlConnection
   pulumi stack output kubeconfig --show-secrets > ~/.kube/agents.yaml
   export KUBECONFIG=~/.kube/agents.yaml

7. G

   Smoke test

   Start a run, poll the workflow, open a WebSocket and replay events.

   smoke.shany shell

   export GW=$(cd infra && pulumi stack output gatewayUrl)
   
   RUN=$(curl -sX POST "$GW/agents/session-1/runs" \
     -H 'content-type: application/json' \
     -d '{"runId":"run-1","message":"hello"}')
   echo "$RUN"
   
   EXEC=$(echo "$RUN" | jq -r .executionId)
   curl -s "$GW/runs/$EXEC" | jq .
   
   npx -y wscat -c "${GW/https/wss}/ws/session-1"

01

North star: what you are building

A production deployment is a contract between Cloud Run as the user edge, GKE as the durable runtime, and Cloud SQL as the brain. Every later module sharpens this triangle.

02

Docker-to-GKE mental model

Learn only the Kubernetes objects a Docker operator needs: image, replicas, DNS, secrets, ports, and health. No Helm. No custom controllers.

03

The Effect runtime you will ship

One Node container launches a single Layer graph. Cluster, workflow engine, entities, workflows, SQL, AI providers, and telemetry are services in the same graph.

const podDns = () =>
  `${process.env.POD_NAME}` +
  `.effect-agent-runners.agents.svc.cluster.local`

const ClusterLayer = NodeClusterSocket.layer({
  storage: "sql",
  runnerHealth: "k8s",
  runnerHealthK8s: {
    namespace: "agents",
    labelSelector: "app=effect-agent-runtime"
  },
  shardingConfig: {
    runnerAddress: Option.some(
      RunnerAddress.make(podDns(), 50000)
    ),
    runnerListenAddress: Option.some(
      RunnerAddress.make("0.0.0.0", 50000)
    )
  }
})

const RuntimeLayer = Layer.mergeAll(
  AgentEntityLive,
  AgentWorkflowLive,
  AgentGatewayLive
).pipe(
  Layer.provideMerge(ClusterWorkflowEngine.layer),
  Layer.provideMerge(ClusterLayer),
  Layer.provideMerge(SqlLayer),
  Layer.provideMerge(AiLayer)
)

Layer.launch(RuntimeLayer).pipe(NodeRuntime.runMain)

04

Pulumi stack: one program owns infra and Kubernetes

Pulumi TypeScript first creates the GCP resources, then uses the generated kubeconfig to create namespace, RBAC, services, StatefulSet, secrets, and gateway objects.

import * as gcp from "@pulumi/gcp"
import * as k8s from "@pulumi/kubernetes"
import * as pulumi from "@pulumi/pulumi"

const cfg = new pulumi.Config()
const region = cfg.require("region")

const network = new gcp.compute.Network("agents-vpc", {
  autoCreateSubnetworks: false
})
const subnet = new gcp.compute.Subnetwork("agents-subnet", {
  region,
  network: network.id,
  ipCidrRange: "10.42.0.0/20"
})
const repo = new gcp.artifactregistry.Repository("agents", {
  location: region,
  repositoryId: "agents",
  format: "DOCKER"
})
const db = new gcp.sql.DatabaseInstance("agents-pg", {
  region,
  databaseVersion: "POSTGRES_16",
  settings: {
    tier: "db-custom-2-7680",
    ipConfiguration: { ipv4Enabled: false, privateNetwork: network.id },
    backupConfiguration: { enabled: true, pointInTimeRecoveryEnabled: true },
    availabilityType: "REGIONAL"
  }
})
const cluster = new gcp.container.Cluster("agents-autopilot", {
  location: region,
  enableAutopilot: true,
  network: network.id,
  subnetwork: subnet.id,
  privateClusterConfig: { enablePrivateNodes: true }
})

const k8sProvider = new k8s.Provider("gke", { kubeconfig: makeKubeconfig(cluster) })

// Namespace, ServiceAccount, RBAC, headless Service, StatefulSet,
// Secret, gateway Service — all created with k8sProvider.

05

Build path: local to production

Every module ships a lab outcome. You do not jump to multi-replica GKE on day one.

1. 01

   Local Docker Compose

   One Node container plus Postgres. Use storage: "local" first, then switch to storage: "sql".

   docker-compose.ymllocal dev

   services:
     postgres:
       image: postgres:16
       environment:
         POSTGRES_USER: agents
         POSTGRES_PASSWORD: agents
         POSTGRES_DB: agents
       healthcheck:
         test: ["CMD-SHELL", "pg_isready -U agents"]
         interval: 2s
         timeout: 5s
         retries: 10
       ports: ["5432:5432"]
       volumes: [pgdata:/var/lib/postgresql/data]
   
     runtime:
       build:
         context: .
         dockerfile: apps/agent-runtime/Dockerfile
       depends_on:
         postgres: { condition: service_healthy }
       environment:
         DATABASE_URL: postgres://agents:agents@postgres:5432/agents
         POD_NAME: runtime-local
       ports:
         - "8080:8080"     # health + http gateway
         - "50000:50000"   # cluster socket (only for multi-node compose)
   
   volumes:
     pgdata:

   Run with docker compose up --build. First boot creates the database; subsequent boots reuse it. To simulate two runners, copy runtime as runtime-2 with POD_NAME: runtime-local-2 and a different host port.

2. 02

   Pulumi bootstrap

   Create Artifact Registry, Cloud SQL HA, GKE Autopilot, Secret Manager, IAM, VPC.

3. 03

   Single runner on GKE

   Deploy with replicas: 1. Confirm SQL tables, logs, and one workflow execution.

4. 04

   Three runners

   Scale out. Confirm shard ownership moves and same session id routes consistently.

5. 05

   Cloud Run gateway

   Cloud Run calls GKE gateway over private networking. Start runs, poll workflows, stream events.

6. 06

   Production drills

   Kill a pod, restart the runtime, fail a tool, rotate a secret, restore from backup.

import { it } from "@effect/vitest"
import { Effect, Layer, TestClock } from "effect"
import { TestRunner } from "@effect/cluster"
import { AgentWorkflow, AgentWorkflowLive } from "../src/AgentWorkflow"

// TestRunner.layer runs the whole cluster in-memory: no SQL, no
// network, no real time. Perfect for entity + workflow unit tests.
const TestLayer = AgentWorkflowLive.pipe(
  Layer.provideMerge(TestRunner.layer)
)

it.effect("workflow completes for a simple ask", () =>
  Effect.gen(function*() {
    const executionId = yield* AgentWorkflow.execute(
      { runId: "r1", sessionId: "s1", message: "hi" },
      { discard: true }
    )
    // Advance time so any DurableClock.sleep resolves instantly
    yield* TestClock.adjust("1 hour")

    const result = yield* AgentWorkflow.poll(executionId)
    // assert result is Complete with the expected exit
  }).pipe(Effect.provide(TestLayer))
)

06

Networking without hand plumbing

Effect runners need stable addresses. The public app does not need to join the cluster. Cloud Run calls a gateway; the gateway calls typed entity clients.

07

Security defaults you keep from day one

Least-privilege service accounts, secrets out of source, private DB. Pulumi owns both GCP IAM and Kubernetes RBAC.

// 1. GCP service account the pod will act as
const runtimeGsa = new gcp.serviceaccount.Account("agent-runtime", {
  accountId: "agent-runtime",
  displayName: "Effect agent runtime pods"
})

// 2. Let the Kubernetes service account impersonate the GSA
new gcp.serviceaccount.IAMBinding("agent-runtime-wi", {
  serviceAccountId: runtimeGsa.name,
  role: "roles/iam.workloadIdentityUser",
  members: [pulumi.interpolate`serviceAccount:${project}.svc.id.goog` +
            `[agents/agent-runtime]`]
})

// 3. Give the GSA only what the pod actually needs
new gcp.projects.IAMMember("agent-runtime-sql", {
  project, role: "roles/cloudsql.client",
  member: pulumi.interpolate`serviceAccount:${runtimeGsa.email}`
})
new gcp.projects.IAMMember("agent-runtime-secrets", {
  project, role: "roles/secretmanager.secretAccessor",
  member: pulumi.interpolate`serviceAccount:${runtimeGsa.email}`
})

// 4. Annotate the K8s service account
new k8s.core.v1.ServiceAccount("agent-runtime", {
  metadata: {
    name: "agent-runtime", namespace: "agents",
    annotations: {
      "iam.gke.io/gcp-service-account": runtimeGsa.email
    }
  }
}, { provider: k8sProvider })

08

Observability that sees fibers, workflows, and infra

Agents fail in unusual ways: rate limits, tool failures, partitions, workflow suspension, runner churn. Logs alone are not enough.

09

Operations and failure nuances

The course treats these as required labs, not optional notes.

spec:
  template:
    spec:
      terminationGracePeriodSeconds: 60   # give cluster time to shed shards
      containers:
        - name: agent-runtime
          # Startup probe lets the Layer graph come up before liveness fires.
          # First boot includes opening pg pool, joining the shard ring,
          # and running pending migrations — easy 20s.
          startupProbe:
            httpGet: { path: /healthz, port: 8080 }
            failureThreshold: 30
            periodSeconds: 2
          readinessProbe:
            httpGet: { path: /readyz, port: 8080 }
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /healthz, port: 8080 }
            periodSeconds: 10
          lifecycle:
            preStop:
              exec:
                # Tell the runtime to stop accepting new entity messages,
                # then let in-flight workflow activities finish. The Effect
                # runtime's own scope finalizers release shard locks on
                # SIGTERM; preStop just gives the load balancer time to
                # mark this pod NotReady first.
                command: ["/bin/sh", "-c", "sleep 10"]

10

Effect Cluster: entities, runners, shards, clients

Cluster is typed distributed actors. An Entity is the protocol. An entity layer is the server. Entity.client is the typed remote handle.

const Ask = Rpc.make("Ask", {
  payload: { runId: Schema.String, message: Schema.String },
  success: Schema.String
})
const Cancel = Rpc.make("Cancel", {
  payload: { executionId: Schema.String },
  success: Schema.Void
})

export const AgentEntity = Entity.make("Agent", [Ask, Cancel])

export const AgentEntityLive = AgentEntity.toLayer(
  Effect.gen(function* () {
    const address = yield* Entity.CurrentAddress
    const sessionId = address.entityId

    return AgentEntity.of({
      Ask: ({ payload }) => AgentWorkflow.execute({
        runId: payload.runId,
        sessionId,
        message: payload.message
      }, { discard: true }),

      Cancel: ({ payload }) =>
        AgentWorkflow.interrupt(payload.executionId)
    })
  }),
  { maxIdleTime: "10 minutes" }
)

11

Effect Workflow: durable agent runs

Cluster answers where does this agent live. Workflow answers how do I run this long task durably, retry it, sleep, resume, and inspect status.

class AgentError extends Schema.TaggedError<AgentError>("AgentError")(
  "AgentError",
  { message: Schema.String }
) {}

export const AgentWorkflow = Workflow.make({
  name: "AgentWorkflow",
  payload: {
    runId: Schema.String,
    sessionId: Schema.String,
    message: Schema.String
  },
  success: Schema.String,
  error: AgentError,
  idempotencyKey: ({ runId }) => runId
})

export const AgentWorkflowLive = AgentWorkflow.toLayer(
  Effect.fn(function* (payload, executionId) {
    yield* EventStore.append({
      executionId, type: "started", message: payload.message
    })

    const answer = yield* Activity.make({
      name: "RunAgentTurn",
      success: Schema.String,
      error: AgentError,
      execute: Effect.gen(function* () {
        const assistant = yield* AiAssistant
        return yield* assistant.agent(payload.message)
      })
    }).pipe(Activity.retry({ times: 3 }))

    yield* EventStore.append({ executionId, type: "completed", answer })
    return answer
  })
)

12

WebSocket gateway: realtime over durable work

WebSockets belong at the gateway edge. They never own state. They start workflows, subscribe to persisted events, and reconnect by replaying from lastEventId.

const GatewayLive = HttpRouter.empty.pipe(
  HttpRouter.get(
    "/ws/:sessionId",
    Effect.gen(function* () {
      const req = yield* HttpServerRequest.HttpServerRequest
      const sessionId = req.pathParams.sessionId

      const outbound = EventStore.stream(sessionId).pipe(
        Stream.map((event) => JSON.stringify(event)),
        Stream.encodeText
      )

      const inbound = outbound.pipe(
        Stream.pipeThroughChannel(HttpServerRequest.upgradeChannel()),
        Stream.decodeText(),
        Stream.runForEach((raw) =>
          Effect.gen(function* () {
            const msg = yield* Schema.decodeUnknown(ClientMessage)(
              JSON.parse(raw)
            )
            const clientFor = yield* AgentEntity.client
            const agent = clientFor(sessionId)
            switch (msg._tag) {
              case "Ask":
                yield* agent.Ask({ runId: msg.runId, message: msg.text })
                return
              case "Cancel":
                yield* agent.Cancel({ executionId: msg.executionId })
                return
            }
          })
        )
      )

      yield* inbound
      return HttpServerResponse.empty()
    })
  ),
  HttpServer.serve()
)

13

Capacity, SLOs, and scale model for 5–10M users

10 million users is not one number. Model MAU, peak concurrent users, concurrent WebSockets, active agent runs, tool calls per run, tokens per minute, and database writes per run.

14

Production data architecture

At this scale the hard part is not starting a workflow. The hard part is deciding which state is hot, durable, searchable, replayable, compacted, and deletable.

type AgentEvent =
  | { _tag: "RunAccepted"; executionId: string; runId: string }
  | { _tag: "Token"; executionId: string; index: number; text: string }
  | { _tag: "ToolCall"; executionId: string; tool: string; callId: string }
  | { _tag: "ToolResult"; executionId: string; callId: string; ok: boolean }
  | { _tag: "Completed"; executionId: string; answerRef: string }
  | { _tag: "Failed"; executionId: string; reason: string }

// Persist every event with tenantId, sessionId, executionId,
// monotonic eventId, createdAt, and retention bucket.

-- The event store is your code, not Effect's. This is one
-- reasonable shape; adapt freely. Cluster and workflow state are
-- created by @effect/cluster and @effect/workflow themselves — do
-- not touch those tables.

create table agent_events (
  tenant_id     text        not null,
  session_id    text        not null,
  execution_id  text        not null,
  event_id      bigserial   primary key,
  event_type    text        not null,
  payload       jsonb       not null,
  created_at    timestamptz not null default now()
);

create index agent_events_session_idx
  on agent_events (tenant_id, session_id, event_id);

create index agent_events_exec_idx
  on agent_events (execution_id, event_id);

-- Retention partitioning lives here too. The course leaves the
-- specifics to you because tenancy and compliance shape them.

15

Delivery: CI/CD, migrations, rollouts, rollback

Pulumi makes environments repeatable. Production also needs disciplined release mechanics.

1. A

   PR checks

   Typecheck, tests, lint, Pulumi preview on every PR.

2. B

   Image build

   Tagged by git SHA, signed, pushed to Artifact Registry.

3. C

   Migration gate

   Apply backward-compatible DB changes before deploying the new image.

4. D

   Progressive rollout

   Staging, one runner, then full replica count. Smoke tests after each step.

# Service account the CI uses to run `pulumi up` against prod.
# Bind these roles project-wide unless you can scope tighter.

roles/container.admin          # GKE Autopilot cluster + workloads
roles/cloudsql.admin           # Cloud SQL instances and users
roles/artifactregistry.admin   # repos + image pushes
roles/secretmanager.admin      # versions + IAM bindings
roles/iam.serviceAccountAdmin  # to create the workload-identity GSAs
roles/iam.serviceAccountUser   # to *use* those GSAs in workloads
roles/compute.networkAdmin     # VPC, subnets, Cloud Armor
roles/servicenetworking.networksAdmin  # private services access
roles/storage.admin            # Pulumi state bucket + course buckets
roles/monitoring.admin         # dashboards + alert policies

# Do NOT grant roles/owner. It works, then it leaks.

16

Reliability operations: HA, DR, load, incidents

For a 5–10M user product, start with strong single-region HA and tested disaster recovery. Multi-region active-active comes after this is solid.

17

Governance: tenancy, security, abuse, compliance, cost

Agent systems carry unique blast-radius problems: model spend, tool permissions, prompt injection, retention, and tenant isolation.

18

Multi-region HA: when single-region is not enough

Single-region HA covers most 5–10M-user products. Cross-region buys disaster recovery and lower global latency, but it adds replication lag, failover drills, and workflow-versioning rules. Start with read replicas before active-active.

new gcp.sql.DatabaseInstance("agents-pg-replica", {
  region: "us-east1",
  databaseVersion: "POSTGRES_16",
  masterInstanceName: primary.name,
  // Postgres read replicas can never be auto-failover targets in
  // Cloud SQL — this field is effectively a no-op here. Failover for
  // Postgres uses the primary's REGIONAL availability + a manual
  // promote on the replica during a regional outage.
  replicaConfiguration: { failoverTarget: false },
  settings: {
    tier: "db-custom-2-7680",
    availabilityType: "ZONAL", // replicas don't need HA
    ipConfiguration: {
      ipv4Enabled: false,
      privateNetwork: network.id
    }
  }
})

19

Shard groups per tenant or tier

By default all entities share one shard ring. For premium tenants, large customers, or noisy workloads, route them to a dedicated runner pool with shard groups.

export const AgentEntity = Entity.make("Agent", [Ask, Cancel])
  .annotate(
    ClusterSchema.ShardGroup,
    (entityId) =>
      entityId.startsWith("ent-")
        ? "enterprise"
        : "default"
  )

20

Typed RPC gateway with @effect/rpc

Hand-rolling HTTP between Cloud Run and GKE loses your types at the boundary. @effect/rpc gives a typed schema, no codegen, that both sides import from a shared package.

export class StartRun extends Rpc.make("StartRun", {
  payload: {
    sessionId: Schema.String,
    runId: Schema.String,
    message: Schema.String
  },
  success: Schema.Struct({ executionId: Schema.String })
}) {}

export class PollRun extends Rpc.make("PollRun", {
  payload: { executionId: Schema.String },
  success: WorkflowResult
}) {}

export const AgentRpc = RpcGroup.make(StartRun, PollRun)

21

Cloud Armor and IAP: stricter ingress

Cloud Armor is the WAF and rate limiter in front of your load balancer. IAP enforces Google identity on internal endpoints. Both are Pulumi resources you add once the gateway is stable.

new gcp.compute.SecurityPolicy("gw-armor", {
  rules: [
    {
      action: "deny(403)",
      priority: 1000,
      match: {
        expr: {
          expression:
            "evaluatePreconfiguredExpr('sqli-stable')"
        }
      }
    },
    {
      action: "rate_based_ban",
      priority: 2000,
      rateLimitOptions: {
        rateLimitThreshold: { count: 100, intervalSec: 60 },
        conformAction: "allow",
        exceedAction: "deny(429)",
        enforceOnKey: "HTTP-HEADER",
        enforceOnKeyName: "x-tenant-id"
      },
      match: {
        versionedExpr: "SRC_IPS_V1",
        config: { srcIpRanges: ["*"] }
      }
    },
    {
      action: "allow",
      priority: 2147483647,
      match: {
        versionedExpr: "SRC_IPS_V1",
        config: { srcIpRanges: ["*"] }
      }
    }
  ]
})

22

Cost attribution: tokens to dollars to tenants

At 5–10M users you must answer “which tenant spent how much, on which model, doing what.” Two streams feed it: GCP Billing export and your own token accounting.

const response = yield* LanguageModel.generateText({
  prompt: payload.message
})

// response.usage is { inputTokens, outputTokens, totalTokens,
// cachedInputTokens }. The model name is not on usage — pass it
// through from the workflow payload or your model config so the
// number you store matches the call you actually made.
yield* Activity.make({
  name: "RecordUsage",
  execute: TokenUsage.append({
    tenantId: payload.tenantId,
    executionId,
    model: payload.model,
    inputTokens: response.usage.inputTokens,
    outputTokens: response.usage.outputTokens,
    cachedInputTokens: response.usage.cachedInputTokens ?? 0
  })
})

24

Troubleshooting: what actually goes wrong

Every entry below is a real failure mode from running this stack. Symptoms in the words you'll see in a log or alert, then the fix.

Cluster tables never created; the runner logs say "no shards assigned".

You left storage: "local" from local development. NodeClusterSocket.layer only persists shard ownership and runner registry when storage: "sql" is set and the SQL client layer is provided. Set both, redeploy with one replica, then scale out.

Runners can't reach each other; entity calls time out across pods.

runnerAddress and runnerListenAddress are not the same thing. runnerAddress is the address peers use to call this runner — it must be the pod's StatefulSet DNS (pod-0.svc.cluster.local). runnerListenAddress is what the local server binds to — that one is 0.0.0.0. Using 0.0.0.0 for both is the most common bug; the cluster registers an unreachable address.

A workflow keeps firing duplicates every time the client retries.

Your idempotencyKey is not deterministic. Anything that includes Date.now(), crypto.randomUUID(), or the current pod hostname produces a new execution id every retry. The key must be a stable function of the request — usually a client-provided runId:

good vs bad idempotencyKey

// bad — new execution every call
idempotencyKey: () => crypto.randomUUID()

// good — client owns the run id, retries dedupe
idempotencyKey: ({ runId }) => runId

An Activity retries forever and never surfaces the error.

Activity.make on its own retries until the workflow's suspendedRetrySchedule gives up — effectively forever in dev. Wrap with Activity.retry({ times: N }) for a bounded budget, and consider .withCompensation so a final failure cleans up.

WebSockets disconnect every hour on Cloud Run.

Cloud Run caps every request — WebSockets included — at 60 minutes. There's no setting to extend it. Move the WebSocket gateway to GKE (module 12) and keep Cloud Run for short HTTP calls only.

Cloud SQL: FATAL: sorry, too many clients already.

db-custom-2-7680 caps at ~200 connections. With PgClient.layer's default pool of 10, you'll hit this around 20 pods. Either lower the per-pod pool (driver options on the layer), use the Cloud SQL Auth Proxy with its built-in pooling, or upsize the instance. Adding more pods to "fix" latency makes this worse, not better.

Cloud Run → GKE gateway returns 403 unauthorized.

Three things to check, in order. (1) The Cloud Run service identity has roles/run.invoker on the target — or the gateway is verifying the JWT itself and the audience doesn't match. (2) The ID token's aud claim equals the gateway's URL exactly, scheme and trailing slash included. (3) The token is in Authorization: Bearer, not in a custom header that the load balancer strips.

Pods OOMKilled on GKE Autopilot under load.

Autopilot enforces requests strictly — there's no burst headroom from co-tenants on the node. If your container peaks at 800Mi but requests 512Mi, it gets killed at 512Mi. Set resources.requests.memory to your real p95 plus a margin (e.g. 1Gi for the agent runtime), and set limits.memory equal to requests so Node sees the right heap target.

Workload Identity 401: pod can't reach Cloud SQL or Secret Manager.

The Kubernetes service account is missing the annotation that binds it to the GCP service account. The pod is running under the default KSA with no IAM. Check the spec:

k8s/serviceaccount.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: agent-runtime
  namespace: agents
  annotations:
    iam.gke.io/gcp-service-account: agent-runtime@PROJECT.iam.gserviceaccount.com

And the GSA must have roles/iam.workloadIdentityUser granted to PROJECT.svc.id.goog[agents/agent-runtime]. Module 07 has the Pulumi for both halves.

A pod dies, but its shards never move; entity calls hang.

RunnerHealth.layerK8s is what tells the cluster a pod is gone. If you're using runnerHealth: "k8s", the K8s API client needs read access to pods in the namespace — that's the Role + RoleBinding from module 07. If you've left RunnerHealth.layerNoop in by accident (very common when copy-pasting test setup), dead pods stay "alive" forever and their shards are orphaned.

Old workflow executions vanish after a deploy.

You renamed a workflow. Workflow names are the lookup key — Workflow.make({ name: "AgentWorkflow" }) binds executions to that exact string. Renaming, even cosmetically, abandons every in-flight execution. Keep old names. If you must evolve, add a new workflow alongside and migrate explicitly.

pulumi up sits on Cloud SQL for ten minutes then errors.

Private services access peering wasn't ready when the instance tried to create. Pulumi needs gcp.servicenetworking.Connection to exist and propagate before gcp.sql.DatabaseInstance with privateNetwork set. Add an explicit dependsOn: [privateVpcConnection] on the DB resource — Pulumi will not infer this dependency because the values aren't interpolated.