Running Hermes Agent on Kubernetes: What Breaks, What Doesn't, and a Production-Safe Setup

Hermes Agent is Nous Research's self-improving AI agent. It builds skills from experience, keeps persistent memory across sessions, and connects to Telegram, Discord, Slack, WhatsApp, and Signal out of the box. With 230k+ GitHub stars, plenty of teams now want to run it somewhere more durable than a laptop.

Kubernetes is the obvious destination and the least documented one. There's no official Helm chart. The container image's init sequence needs root in a way that collides with a standard restricted pod security context. Its state model assumes a single writer. And the reload story that works fine in a terminal has no equivalent your GitOps controller can call.

None of that is a reason to avoid Hermes on Kubernetes. It's a reason to know the constraints before you write the manifest. Below: what genuinely breaks, what only looks like it breaks, and a reference deployment to start from.

Tested against: Hermes Agent v2026.8.27 (latest tagged release at publication) · official image nousresearch/hermes-agent · Kubernetes 1.31+ · containerd 2.x

Last verified: 28 August 2026

Hermes ships releases every few days. Before you copy anything below, check it against the version you're actually deploying — and pin that version.

Is there an official Helm chart for Hermes Agent?#

No. Nous Research's own documentation covers install.sh, Docker and Docker Compose, and Nix packages — Kubernetes never comes up as a deployment target. That gap is filled entirely by the community, and the three charts that fill it don't agree on much:

• ultraworkers/hermes-agent-helm-chart — the most feature-complete option: renders a Deployment, PVC, Secret, Service, Ingress, and an Istio VirtualService, plus an "operator-ready" mode defining a HermesTenant CRD (Custom Resource Definition) for an external controller. It's also the only one that encodes the single-writer rule as a hard constraint — more on that below.
• jyje/hermes-agent — listed on Artifact Hub as a verified publisher, distributed over an OCI registry, with multi-arch images and example ArgoCD manifests. It tracks upstream Hermes releases more closely than the other two.
• duyet/hermes-agent — the simplest of the three, splitting persistence into separate data and workspace volumes, with an optional Prometheus ServiceMonitor. Not a verified publisher, so read the templates before trusting the defaults.

Chart versions move independently of Hermes itself, so check the current version on each listing rather than trusting a number in any article, this one included. None of the three is backed by Nous Research. Read the templates end to end before you apply one, and expect to override the security context.

Hermes Agent is stateful: define your persistence boundary first#

This is the decision everything else hangs off, so make it before you pick a chart.

Hermes keeps all of its mutable state — configuration, MEMORY.md, USER.md, session history in a local SQLite database, and every learned skill — under HERMES_HOME. In the official image that's the /opt/data volume, which the Docker docs call "the single source of truth for all Hermes state." HERMES_HOME also scopes the gateway's PID file and systemd service name, which is upstream's own signal that it's the unit of isolation: the documented way to run multiple installations concurrently is to give each one its own HERMES_HOME.

So the accurate invariant is not "Hermes can't scale." It's:

Treat each HERMES_HOME as a single-writer state domain.

You can run many Hermes instances in one cluster. What you must not do is put two active pods behind the same mutable state and assume Kubernetes has handed you horizontal scaling. Nothing arbitrates concurrent writes to that SQLite database and those Markdown files.

In practice, for any deployment with persistence enabled:

• replicaCount stays at 1.
• strategy.type must be Recreate, not RollingUpdate — a rolling update deliberately runs the old and new pod together, which is exactly the two-writer window to avoid.
• ReadWriteOnce is the sensible guardrail on the PVC. RWX isn't automatically unsafe — the invariant is one active writer, not one mount — but RWO enforces it at the storage layer instead of trusting your deployment config, which is the better default.

Need Hermes for more than one team or tenant? One release per tenant, each with its own volume. The ultraworkers chart converged on the same rule and enforces it in values.schema.json: its README states that Hermes "stores mutable data under HERMES_HOME, so this chart treats persistent storage as a single-writer workload," requiring replicaCount: 1 and strategy.type: Recreate whenever persistence is on, and recommending multiple releases over horizontal scaling. Any chart that doesn't enforce this is a data-corruption risk waiting for a bad rollout.

Why the official image fights runAsNonRoot#

The common shorthand — "Hermes has to run as root" — is wrong, and the precise version matters when you're arguing with a platform team about a Pod Security Standard exemption.

The official image uses s6-overlay as its init system. Per the Docker documentation, s6-overlay's /init "runs as root so it can chown the volume on first boot, then drops to the hermes user via s6-setuidgid" for the main program and all supervised services. The image even ships a shim at /opt/hermes/bin/hermes that detects root callers on docker exec and transparently re-execs through s6-setuidgid hermes.

So the agent process itself does not run as root. Only the bootstrap does. But that's still enough to break a restricted security context: a pod spec with runAsNonRoot: true or a non-zero runAsUser blocks the sequence before the agent ever starts, and you get something close to:

/package/admin/s6-overlay/libexec/preinit: fatal: /run belongs to uid 0 instead of 1000,
has insecure and/or unworkable permissions, and we're lacking the privileges to fix it.
s6-overlay-suexec: fatal: child failed with exit code 100

One detail worth getting right, because published examples routinely get it wrong: the hermes user is UID 10000, not 1000. The Dockerfile creates it with useradd -u 10000 -m -d /opt/data hermes and sets no USER instruction — the container starts as root by design and drops privileges itself. Set fsGroup: 1000 and the volume ends up owned by the wrong group.

A reasonable starting security context:

podSecurityContext:
  runAsUser: 0
  runAsNonRoot: false
  fsGroup: 10000
  fsGroupChangePolicy: OnRootMismatch
  seccompProfile:
    type: RuntimeDefault
securityContext:
  allowPrivilegeEscalation: true
  readOnlyRootFilesystem: false
  runAsNonRoot: false
  capabilities:
    drop: ["ALL"]
    add: ["CHOWN", "SETUID", "SETGID"]

That is meaningfully narrower than a privileged pod: every capability is dropped and only the ones the bootstrap needs come back. Treat the exact capability list as a starting point to verify against your image version, not a universal recipe — s6's requirements shift with the image's init scripts, and FOWNER or DAC_OVERRIDE may be needed depending on how your storage class presents volume ownership. Start from drop: ["ALL"], add back only what your logs prove necessary, and re-test on version bumps.

If your cluster enforces the restricted Pod Security Standard, this workload needs a namespace exemption. Forcing the official image to start non-root isn't a setting you've missed — it wasn't built for that.

Worth noting for anyone citing it as a counterexample: Red Hat's OpenShift AI walkthrough does set runAsNonRoot: true, but it deploys a different image (quay.io/aicatalyst/hermes-agent), not the official Nous one. It isn't evidence that the official image runs non-root.

PID 1: what happens when the platform wraps the entrypoint#

This one is specific to orchestrators and rarely covered in Kubernetes guides.

The image's ENTRYPOINT is a dispatcher (/opt/hermes/docker/entrypoint-dispatch.sh), and s6-overlay's suexec requires PID 1. When a platform wraps the entrypoint under its own init, the docs note that /init "would abort with s6-overlay-suexec: fatal: can only run as pid 1" — so the dispatcher instead runs the stage2 bootstrap directly and execs the main wrapper without s6. Upstream names Fly.io Machines, docker run --init, and "some Nomad/Kubernetes setups" as cases that hit this path.

The practical consequences on Kubernetes:

• If something in your stack injects an init process — a shareProcessNamespace: true pod, certain sidecar injectors, a runtime configured to add one — you silently lose s6 supervision. The agent still runs; it just isn't being restarted by s6 any more.
• That isn't necessarily bad. In a cluster you generally want Kubernetes' restart policy to be the supervisor, not a process manager inside the container.
• But it changes your failure model, so know which path you're on. If you want the no-s6 behaviour deliberately, ask for it with --no-supervise rather than relying on an accident of your platform's PID 1 handling.

The v0.15.x containerd restart loop — and how to diagnose it#

An earlier version of this article presented this as a live, confirmed bug. It isn't, and the distinction matters.

In Hermes v0.15.0 through v0.15.2, users running Docker 28.5.2 reported the gateway restarting endlessly, with WARNING gateway.run: Shutdown context: signal=UNKNOWN repeating in the logs. It was a regression from v0.14.x. Issue #35394 documents the failure signature in detail.

What the issue does not do is establish a root cause. The reporter proposed that containerd 2.x — shipped with Docker 28.5.2 — changed signal delivery in a way s6-overlay v3.2.3.0 mishandles, and the issue explicitly labels that a root cause hypothesis. It was closed as duplicate / not planned, which reflects triage routing rather than a decision that the underlying behaviour doesn't matter. Hermes has shipped many releases since, up to v2026.8.27.

So treat this as a failure signature to recognise, not a property of Hermes on containerd:

• Symptom: the gateway restarts every few seconds; logs show signal=UNKNOWN rather than a clean SIGTERM.
• Check your version first. If you're not on v0.15.x, you're probably looking at a different problem — check the exit code, and whether the container or only the supervised process is restarting.
• Workaround for that signature: HERMES_GATEWAY_NO_SUPERVISE=1 (or the --no-supervise flag) disables s6's auto-restart, making the gateway the container's main process and handing recovery to Kubernetes.
• The reported fix at the time was downgrading to v0.14.x, which is not advice worth following two dozen releases later.

Don't set HERMES_GATEWAY_NO_SUPERVISE=1 reflexively. Supervised operation is the documented default for the official image, and disabling it is a real trade: you lose in-container crash recovery in exchange for pod-level restarts. That's often the right call on Kubernetes — but make it deliberately, not as cargo-culted mitigation for a bug you don't have.

What Hermes can hot-reload, and what it can't#

The claim that Hermes has "no hot-reload" is false, and correcting it matters because the real limitation is more interesting.

Hermes documents three reload commands:

• /reload-mcp — "Reload MCP servers from config.yaml"
• /reload-skills — "Re-scan ~/.hermes/skills/ for newly installed or removed skills"
• /reload — "Reload .env variables into the running session (picks up new API keys without restarting)"

The runtime can absolutely pick up new MCP servers and skills without a restart. The gap is the interface:

Hermes can hot-reload MCP servers and skills interactively, but doesn't expose the same lifecycle through its admin API.

Issue #52417 asks for exactly that — POST /api/admin/reload/mcp, /reload/skills, and /reload, authorised with a bearer token via API_SERVER_KEY — noting that the slash commands exist but "the API Server does not seem to provide equivalent HTTP endpoints." It's open, tagged low priority.

That's the actual Kubernetes problem, and it's a shape platform engineers will recognise:

ConfigMap updated -> kubelet syncs file into the pod -> no reconciliation hook -> nothing happens

The file changes on disk. Nothing tells the process. Your options:

1. Restart the workload. kubectl rollout restart deployment/hermes-agent. Blunt but declarative, and with Recreate you're accepting a brief outage anyway. Wire a checksum annotation over the ConfigMap into the pod template so the rollout fires automatically on config change — the standard Helm checksum/config pattern.
2. Drive the runtime reload path from a sidecar or an operator with a channel into the agent. Workable, but you're building the missing hook yourself.
3. Wait for #52417, after which a small controller can watch the ConfigMap and POST the reload — the clean answer, and the reason that issue is worth watching if you run Hermes under GitOps.

Until then, option 1 with a config checksum is the pragmatic default.

Resources, probes, and graceful shutdown#

Three things almost every Hermes-on-Kubernetes writeup omits.

Resources. Nous Research's Docker documentation is specific:

Browser automation (Playwright/Chromium) is called out as the most memory-hungry feature, so if the agent drives a browser, budget above the recommended range. Set a memory limit, but consider leaving CPU unlimited — agent workloads are bursty, and CPU throttling shows up as mysteriously slow tool calls rather than a clean failure.

Probes. The gateway listens on port 8642, which the docs describe as exposing "the gateway's OpenAI-compatible API server and health endpoint." Note that the Dockerfile ships no HEALTHCHECK and no EXPOSE instruction, so there's nothing to inherit — you're defining this yourself. A tcpSocket probe against 8642 is the portable choice; if you've confirmed the health path for your version, an httpGet probe is strictly better. The one that matters most is the startupProbe: first boot does volume chown work and profile reconciliation, and a liveness probe with a short threshold will kill the pod mid-bootstrap and loop forever.

Graceful shutdown. Hermes is stateful, so SIGTERM handling isn't academic. The container exits when its main program exits, and s6 propagates the signal so shutdown is clean. Give it room with terminationGracePeriodSeconds: 30, and test what actually happens to in-flight agent runs, SQLite session writes, and open gateway connections when a pod is evicted. Node upgrades and spot reclaims will do this to you eventually, and Recreate means no second pod is covering the gap.

Two security boundaries, not one#

Most Kubernetes writeups on agents cover pod security and stop. For an agent that executes code, that's half the problem.

Boundary 1 — pod security. Root init, capabilities, seccomp, filesystem, service account. Covered above.

Boundary 2 — agent execution. What the model can actually run, and what it can reach. This is the one that should worry you more, because the pod is the blast radius.

Hermes has real defences here, and they're worth knowing precisely:

• Credential filtering. execute_code blocks environment variables whose names contain KEY, TOKEN, SECRET, PASSWORD, CREDENTIAL, PASSWD, or AUTH. The terminal tool blocks Hermes infrastructure variables (provider keys, gateway tokens). MCP stdio subprocesses receive only PATH, HOME, USER, LANG, LC_ALL, TERM, SHELL, and TMPDIR.
• The bypass is explicit. Variables declared by a skill or listed in env_passthrough skip those filters. That's the mechanism to audit — a skill can legitimately request a credential, and from then on that credential is readable by code the model writes.
• Approval checks are skipped in container backends. Hermes skips dangerous-command approval in the docker, singularity, modal, daytona, and vercel_sandbox backends, on the reasoning that "the container itself is the security boundary." In a Kubernetes pod that assumption is load-bearing: whatever your pod can reach, prompt-injected code can reach. A hardline blocklist (fork bombs, filesystem wipes, device formatting) stays non-overridable.
• SSRF protection exists, and is disableable. Web tools block RFC 1918 ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and loopback by default. Upstream's guidance on relaxing that is blunt: only enable it where "the agent running arbitrary prompt-injected URLs against the local network is an acceptable risk." In a cluster, RFC 1918 is your service mesh, your databases, and the kubelet.

Which leads to the control most often missing: default-deny egress. An agent that browses the web and calls tools is not a normal web app, and "what can this pod reach?" is a more consequential question than which Linux capabilities it holds. Start from deny-all and allow only what's needed:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: hermes-agent-egress
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: hermes-agent
  policyTypes: ["Egress"]
  egress:
    # DNS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
    # HTTPS to the internet, minus internal ranges
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 10.0.0.0/8
              - 172.16.0.0/12
              - 192.168.0.0/16
              - 169.254.169.254/32   # cloud instance metadata
      ports:
        - protocol: TCP
          port: 443

Then add explicit allow rules for the internal services the agent genuinely needs. Blocking 169.254.169.254 matters especially: on a node without IMDSv2 enforced, an agent that can reach instance metadata can often reach the node's IAM role.

On credentials themselves: because the agent has terminal access, anything in its environment is potentially readable by it. On EKS, use IRSA (IAM Roles for Service Accounts) or EKS Pod Identity rather than static access keys in a Secret — Pod Identity is simpler to wire up, IRSA is more broadly documented; match whichever your cluster already uses, and don't run both for one workload. Elsewhere, External Secrets Operator, Vault, or Sealed Secrets all keep plaintext keys out of Git. And scope the IAM role tightly: the agent's permissions are the agent's capabilities.

A minimal deployment that actually works#

Pinned, single-writer, probed, and resource-bounded. Adjust the storage class and image tag, and read it before you apply it.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: hermes-agent-data
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hermes-agent
  labels:
    app.kubernetes.io/name: hermes-agent
spec:
  replicas: 1
  strategy:
    type: Recreate          # never two writers on one HERMES_HOME
  selector:
    matchLabels:
      app.kubernetes.io/name: hermes-agent
  template:
    metadata:
      labels:
        app.kubernetes.io/name: hermes-agent
      annotations:
        # forces a rollout when config changes, since there is no HTTP reload hook
        checksum/config: "REPLACE_WITH_CONFIGMAP_CHECKSUM"
    spec:
      serviceAccountName: hermes-agent
      terminationGracePeriodSeconds: 30
      securityContext:
        runAsUser: 0              # s6 /init needs root to chown the volume
        runAsNonRoot: false
        fsGroup: 10000            # the hermes user is UID 10000
        fsGroupChangePolicy: OnRootMismatch
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: hermes-agent
          image: nousresearch/hermes-agent:v2026.8.27   # pin it; never :latest
          args: ["gateway", "run"]
          ports:
            - name: gateway
              containerPort: 8642
          envFrom:
            - secretRef:
                name: hermes-agent-secrets
          securityContext:
            allowPrivilegeEscalation: true
            readOnlyRootFilesystem: false
            runAsNonRoot: false
            capabilities:
              drop: ["ALL"]
              add: ["CHOWN", "SETUID", "SETGID"]
          resources:
            requests:
              cpu: "500m"
              memory: 2Gi
            limits:
              memory: 4Gi        # raise if skills use Playwright/Chromium
          startupProbe:          # first boot does chown + profile reconciliation
            tcpSocket:
              port: gateway
            periodSeconds: 5
            failureThreshold: 60 # ~5 minutes before giving up
          readinessProbe:
            tcpSocket:
              port: gateway
            periodSeconds: 10
          livenessProbe:
            tcpSocket:
              port: gateway
            periodSeconds: 20
            failureThreshold: 3
          volumeMounts:
            - name: data
              mountPath: /opt/data   # HERMES_HOME — all mutable state
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: hermes-agent-data
---
apiVersion: v1
kind: Service
metadata:
  name: hermes-agent
spec:
  selector:
    app.kubernetes.io/name: hermes-agent
  ports:
    - name: gateway
      port: 8642
      targetPort: gateway

Pair it with the NetworkPolicy above, a Secret (ideally rendered by External Secrets or Sealed Secrets rather than committed), and a ConfigMap for config.yaml if you're managing MCP servers declaratively.

Production checklist#

• Image tag pinned to a specific release — never :latest
• replicas: 1 and strategy.type: Recreate, with one HERMES_HOME per instance
• PVC is ReadWriteOnce, and the storage class supports that access mode
• fsGroup: 10000 matches the image's hermes user
• Capabilities start from drop: ["ALL"], with additions verified against your image version
• Namespace exemption in place if you enforce the restricted Pod Security Standard
• startupProbe generous enough to survive first-boot volume work
• terminationGracePeriodSeconds set, and eviction behaviour tested against in-flight runs
• Memory limit sized for browser automation if skills use it
• Default-deny egress NetworkPolicy, with instance metadata (169.254.169.254) blocked
• No static cloud credentials in Secrets — IRSA / Pod Identity / Vault / External Secrets
• env_passthrough and skill-declared variables audited — they bypass credential filtering
• Config changes trigger a rollout (checksum annotation) or a deliberate reload path
• You know whether s6 supervision is active or bypassed by your platform's PID 1 handling

Known issues by version#

The wider ecosystem: operators, GitOps, and managed paths#

If hand-written manifests feel too manual, community operators exist — none from Nous Research. UndermountainCC/hermes-operator offers a HermesAgent CRD managing pod, PVC, service account, and RBAC declaratively; it's explicitly v1alpha1 and documents that its API can still change, so treat it as early-stage. For GitOps, the jyje chart ships example ArgoCD Application manifests alongside a Sealed Secrets pattern. Whichever path you pick, the single-writer constraint survives it: one release per Hermes instance, whether driven by helm install, a custom resource, or an ArgoCD Application.

If you're on OpenShift, Red Hat published a walkthrough for deploying Hermes Agent on OpenShift AI with vLLM model serving, pairing a GPU-accelerated vLLM server as a KServe InferenceService with a Hermes deployment and PVC. A genuine option if you need local GPU inference — but as noted above, it uses a different container image, so its security posture doesn't transfer to the official one.

And if you're still choosing an agent framework rather than committed to Hermes, kagent is built around Kubernetes primitives from the start rather than adapted onto them afterwards. Hermes solves a different problem: it's a specific, opinionated agent with its own memory, skills, and gateway integrations that happens to need somewhere to run. If Kubernetes-native operation is the priority, kagent is worth the comparison; if you want Hermes' feature set, everything above applies.

Where this connects to governed data#

Everything above keeps Hermes Agent alive and contained. None of it says whether the answers it produces are correct — that depends on what it's allowed to read and how well-defined that data is. The same discipline that makes an agent's infrastructure trustworthy (scoped permissions, no unmanaged state, changes that go through review) reappears in agentic data engineering, where an agent's output earns trust through a harness of checks rather than raw model capability. Point an agent like this at real business data instead of chat platforms and a semantic layer is what stops it guessing at what "revenue" or "active customer" means — part of the same shift toward treating data strategy as AI infrastructure.

Read more about revenue operations, growth strategies, and metrics in our blog and follow us on LinkedIn and Youtube.