Autoscaling & worker pools

In ExoHuman Cloud, agents do not run on one fixed server. Their containers are orchestrated across worker pools — groups of worker machines that each provide one kind of capacity — and each pool scales with demand.

Worker pools

There is one pool per compute profile family: Standard, Performance, High memory and one per GPU type (L4, A100, H100). Custom profiles are placed on the pool that matches their shape.

Each worker machine runs a lightweight worker service that connects outbound to your instance’s control plane, reports its capacity and labels (for example its GPU type), sends a heartbeat and accepts agent containers placed on it.

How an agent run is placed

1. Work arrives for an agent — a chat message, schedule, delegation, webhook or channel message.
2. The control plane queues an execution and looks for a worker in the pool for that agent’s profile that is healthy (recent heartbeat), has free capacity and matches the agent’s labels.
3. The agent’s container starts (or resumes) on that worker and runs the task.
4. When the agent goes idle, its container is suspended and the capacity returns to the pool.

If a worker stops sending heartbeats, it is marked lost and its leased jobs are re-queued on healthy workers. Workers can also be drained for maintenance: they finish current work and accept nothing new.

Scaling behaviour

• Scale out. When queued runs for a profile approach the pool’s free capacity, we add workers to that pool.
• Scale in. When workers sit idle, they are drained and removed. Your agents’ state lives in the control plane and in git, not on the worker, so removing a worker never loses work.
• Per profile. A burst of GPU work scales the GPU pool without changing your CPU pools, and vice versa.
• Scale to zero. Agents that are not running consume no compute and are not billed.

Limits that still apply

Autoscaling provides capacity; your plan and settings decide how much of it you may use at once:

• Concurrent runs per plan — the maximum number of executions running at the same time across your instance. Extra runs wait in the queue.
• Parallel capacity per agent — how many tasks one agent may run at once (default 3).
• GPU availability — GPU pools scale within the accelerators available in the region. Enterprise can reserve capacity.

See Usage limits & budgets.

What this means for you

• You never size servers. Choose a compute profile per agent and a plan with enough concurrent runs.
• Bursty workloads are cheap: twenty agents that each work ten minutes an hour cost about the same as three agents running continuously.
• Long-running interactive features — such as live streaming of an agent’s output or an interactive shell — work on any worker; your instance proxies them for you.

Your own machines as worker nodes

Cloud autoscaling uses the same worker mechanism as the machines you own: connect any Linux server, VM or GPU machine as a worker node, and its capacity joins the fleet. Nodes you add are yours to keep, drain and remove at any time.