mirror of
https://github.com/tiennm99/coolify.git
synced 2026-10-13 08:15:24 +00:00
feat(v5): use bridged Lima hosts for coold dev
Add v5 architecture docs and update dev bootstrap, seeding, and tests to use bridged Lima .local hosts with a container-accessible SSH key.
This commit is contained in:
1 parent
1c091e9e36
commit
068a2979fd
17 files changed
+636
-462
No files matched your search
@@ -0,0 +1,34 @@
|
||||
# Coolify v5 Architecture
|
||||
|
||||
This directory is the canonical architecture reference for Coolify v5 from the
|
||||
Laravel control-plane point of view.
|
||||
|
||||
v5 has three major building blocks:
|
||||
|
||||
```text
|
||||
User / API / Git webhook
|
||||
↓
|
||||
Coolify Laravel control plane
|
||||
↓ HTTP over /run/coolify/flux.sock
|
||||
Flux
|
||||
↓ outbound gRPC Agent.Stream
|
||||
coold on each host
|
||||
↓
|
||||
Podman / networks / firewall / DNS / Corrosion / builder
|
||||
```
|
||||
|
||||
## Documents
|
||||
|
||||
| Document | Purpose |
|
||||
| --- | --- |
|
||||
| [Overview](overview.md) | High-level mental model and data flow. |
|
||||
| [Responsibility split](responsibility-split.md) | What belongs in Coolify, Flux, and coold. |
|
||||
| [Primitives](primitives.md) | Canonical host primitive surface Coolify may dispatch. |
|
||||
| [Deploy flows](deploy-flows.md) | User-functionality flows, starting with `nginx:alpine`. |
|
||||
| [ADR 0001](adr/0001-control-plane-flux-coold-split.md) | Decision record for the v5 split. |
|
||||
|
||||
## Core rule
|
||||
|
||||
Coolify owns user intent and application state. Flux routes requests. coold
|
||||
executes explicit host primitives. coold must not grow app, team, RBAC,
|
||||
deployment, billing, or audit concepts.
|
||||
@@ -0,0 +1,75 @@
|
||||
# ADR 0001: Split v5 into Coolify control plane, Flux broker, and coold host agent
|
||||
|
||||
## Status
|
||||
|
||||
Accepted.
|
||||
|
||||
## Context
|
||||
|
||||
Coolify v5 needs to manage many user resources across many user-owned hosts,
|
||||
including hosts behind NAT or restrictive firewalls. The Laravel application
|
||||
must own product behavior and durable state, but it should not hold thousands of
|
||||
long-lived agent streams or directly expose host runtime sockets.
|
||||
|
||||
Host operations also need a narrow privileged boundary. Podman, firewall, DNS,
|
||||
Corrosion, and build supervision require local host privileges that should not
|
||||
be spread across the Laravel app or arbitrary scripts.
|
||||
|
||||
## Decision
|
||||
|
||||
Coolify v5 uses three distinct building blocks:
|
||||
|
||||
1. **Coolify Laravel control plane** owns user intent, durable product state,
|
||||
RBAC, deployment state machines, placement decisions, secrets, proxy config
|
||||
rendering, notifications, and audit.
|
||||
2. **Flux** owns long-lived agent connectivity. Laravel talks to Flux over a
|
||||
Unix socket. coold agents dial Flux over outbound gRPC. Flux routes typed
|
||||
primitive requests to connected hosts and resolves pending responses.
|
||||
3. **coold** runs once per host. It exposes a closed set of host primitives and
|
||||
owns privileged local execution through Podman, firewall, DNS, Corrosion,
|
||||
host facts, and builder supervision.
|
||||
|
||||
coold must not expose raw Podman passthrough. Every supported operation must be
|
||||
an explicit primitive with validation and a stable protocol shape.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Coolify keeps product complexity in one durable control plane.
|
||||
- Hosts can live behind NAT because coold dials out to Flux.
|
||||
- Privileged host access is isolated to coold.
|
||||
- Flux can scale connection handling separately from Laravel request workers.
|
||||
- The primitive surface can be tested independently from Coolify's app model.
|
||||
|
||||
### Negative
|
||||
|
||||
- More moving parts than a direct SSH/Docker model.
|
||||
- Protocol changes require coordination across Coolify, Flux, and coold.
|
||||
- Some deploy features require new primitives before the v5 path is complete.
|
||||
|
||||
## Boundary rules
|
||||
|
||||
- Coolify may say: "deploy this nginx resource for this team to host H1." coold
|
||||
may not.
|
||||
- Coolify may send: `images.pull`, `containers.create`, `containers.start`,
|
||||
`services.register`, `firewall.allow`. coold may execute those.
|
||||
- Flux may route requests but may not make product decisions.
|
||||
- coold may reject dangerous host operations even when Coolify requested them.
|
||||
|
||||
## First reference flow
|
||||
|
||||
The first validation flow is deploying a Docker image app using `nginx:alpine`.
|
||||
That flow requires no builder and exercises the minimum runtime path:
|
||||
|
||||
```text
|
||||
Coolify state machine
|
||||
→ Flux dispatch
|
||||
→ coold image pull
|
||||
→ coold container create/start
|
||||
→ coold status
|
||||
→ coold service registration
|
||||
→ coold firewall
|
||||
→ Coolify proxy config/reload
|
||||
→ Coolify marks deployment running
|
||||
```
|
||||
@@ -0,0 +1,76 @@
|
||||
# v5 Deploy Flows
|
||||
|
||||
These flows are written from the user-functionality view. Coolify owns the
|
||||
state machine; Flux routes primitive requests; coold executes host operations.
|
||||
|
||||
## Flow: deploy Docker image app `nginx:alpine`
|
||||
|
||||
User intent:
|
||||
|
||||
```text
|
||||
Run nginx:alpine on host H1, expose port 80, and route nginx.example.com to it.
|
||||
```
|
||||
|
||||
### Steps
|
||||
|
||||
| Step | Coolify | Flux | coold |
|
||||
| --- | --- | --- | --- |
|
||||
| Create app | Stores app, source type `docker_image`, image `nginx:alpine`, target host, port, domain. | No action. | No action. |
|
||||
| Start deploy | Creates deployment record and enters deploy state machine. | No action until dispatch. | No action. |
|
||||
| Pull image | Sends `images.pull` to H1. | Routes request to H1's stream. | Pulls image through Podman. |
|
||||
| Prepare network/volume | Sends required `networks.create` / `volumes.create` operations. | Routes requests. | Creates idempotent host resources. |
|
||||
| Create container | Sends `containers.create` with image, name, env, labels, network, port, health check, mounts, DNS. | Routes request. | Applies deny filter and creates container through Podman. |
|
||||
| Start container | Sends `containers.start`. | Routes request. | Starts container. |
|
||||
| Check status | Sends `containers.inspect` or reads status stream. | Routes request. | Reads local Podman state. |
|
||||
| Register endpoint | Sends `services.register`. | Routes request. | Writes this host's endpoint row to Corrosion. |
|
||||
| Allow traffic | Sends `firewall.allow` for proxy-to-container traffic when required. | Routes request. | Writes iptables and nft allow rules. |
|
||||
| Configure ingress | Renders proxy config from Coolify domain/resource state. If reload is needed, sends `containers.exec` or equivalent primitive for the proxy container. | Routes runtime reload request. | Executes the reload primitive only. |
|
||||
| Mark running | Stores final status, deployment result, and user-facing logs. | Resolves final responses. | Keeps container running and reports future state. |
|
||||
|
||||
### Expected data locations
|
||||
|
||||
| Data | Owner |
|
||||
| --- | --- |
|
||||
| App name, team, project, env, domain, image ref, desired port | Coolify database |
|
||||
| Deployment state and history | Coolify database |
|
||||
| Open host stream and pending request IDs | Flux memory |
|
||||
| Pulled image and running container | Host Podman storage |
|
||||
| Endpoint rows for this host | Corrosion via coold |
|
||||
| Firewall tuples and snapshots | Host kernel + `/etc/coolify` via coold |
|
||||
|
||||
### No build path
|
||||
|
||||
`nginx:alpine` is a prebuilt image. This flow does not use Git clone,
|
||||
Dockerfile, buildpacks, Railpack, or the builder subprocess.
|
||||
|
||||
## Flow: delete the nginx app
|
||||
|
||||
| Step | Coolify | Flux | coold |
|
||||
| --- | --- | --- | --- |
|
||||
| User deletes resource | Validates permission and starts cleanup state machine. | No action until dispatch. | No action. |
|
||||
| Stop container | Sends `containers.stop`. | Routes request. | Stops container. |
|
||||
| Remove container | Sends `containers.delete`. | Routes request. | Deletes container. |
|
||||
| Remove endpoint | Sends `services.unregister`. | Routes request. | Removes service endpoint row. |
|
||||
| Remove firewall rule | Sends `firewall.revoke`. | Routes request. | Removes iptables and nft allow rules. |
|
||||
| Update proxy | Removes rendered route and reloads proxy if needed. | Routes reload primitive. | Executes proxy reload primitive. |
|
||||
| Finish cleanup | Marks resource deleted and records outcome. | Resolves responses. | No product state retained. |
|
||||
|
||||
## Flow: Git app with build
|
||||
|
||||
A Git app adds a build phase before container creation.
|
||||
|
||||
```text
|
||||
Coolify resolves source + secrets + build config
|
||||
↓
|
||||
Flux routes build request to a host with builder capability
|
||||
↓
|
||||
coold supervises builder subprocess
|
||||
↓
|
||||
builder writes image/result
|
||||
↓
|
||||
Coolify deploy state machine continues with image/container primitives
|
||||
```
|
||||
|
||||
coold supervises the builder process. The builder owns build implementation.
|
||||
Coolify owns the decision to build, the build configuration, and the deployment
|
||||
state transitions around the build.
|
||||
@@ -0,0 +1,38 @@
|
||||
# v5 Architecture Overview
|
||||
|
||||
Coolify v5 separates product intent from host execution.
|
||||
|
||||
- **Coolify Laravel** is the control plane. It owns users, teams, projects,
|
||||
environments, resources, deployments, domains, secrets, RBAC, audit-worthy
|
||||
state, and the deployment state machines.
|
||||
- **Flux** is the connection broker. Laravel talks to Flux over a Unix socket.
|
||||
coold agents dial Flux over outbound gRPC streams. Flux maps host IDs to
|
||||
connected streams and maps request IDs to pending responses.
|
||||
- **coold** is the per-host executor. It owns local runtime access: Podman,
|
||||
host networks, DNS, firewall mutations, service-discovery sync, host facts,
|
||||
and builder subprocess supervision.
|
||||
|
||||
## Data flow
|
||||
|
||||
```text
|
||||
1. A user/API/webhook asks Coolify to do something.
|
||||
2. Coolify validates permissions and stores desired state in its database.
|
||||
3. Coolify's state machine turns that intent into ordered host primitives.
|
||||
4. Coolify sends each primitive to Flux over /run/coolify/flux.sock.
|
||||
5. Flux forwards the primitive to the target host's open coold stream.
|
||||
6. coold executes the primitive locally and returns a typed result.
|
||||
7. Flux resolves the pending request.
|
||||
8. Coolify records the result and moves the deployment/resource state forward.
|
||||
```
|
||||
|
||||
## Non-goals for Flux and coold
|
||||
|
||||
Flux and coold are not product layers. They do not decide which app to deploy,
|
||||
which user is allowed to deploy, how to roll back, what domains mean, or where
|
||||
business audit belongs. They only handle routing and host execution.
|
||||
|
||||
## Current implementation note
|
||||
|
||||
This directory describes the target v5 architecture. Some primitives are not
|
||||
implemented in coold yet. Until implemented, docs should label them as target
|
||||
primitives and code should not pretend they exist.
|
||||
@@ -0,0 +1,106 @@
|
||||
# v5 Host Primitives
|
||||
|
||||
Coolify deploy logic must send explicit primitives to coold through Flux. There
|
||||
must be no raw Podman passthrough.
|
||||
|
||||
This file is the target primitive catalog. Implementations may lag behind this
|
||||
catalog; code should only call primitives that exist in the current protocol.
|
||||
|
||||
## Images
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `images.pull` | Pull an OCI image ref onto a host. |
|
||||
| `images.list` | List local images. |
|
||||
| `images.delete` | Remove a local image when safe. |
|
||||
|
||||
## Containers
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `containers.create` | Create a container from an explicit spec. |
|
||||
| `containers.start` | Start an existing container. |
|
||||
| `containers.stop` | Stop a container with an optional timeout. |
|
||||
| `containers.restart` | Restart a container. |
|
||||
| `containers.delete` | Delete a stopped or forced container. |
|
||||
| `containers.inspect` | Return detailed runtime state. |
|
||||
| `containers.list` | Return host container summaries. |
|
||||
| `containers.logs` | Stream or read container logs. |
|
||||
| `containers.exec` | Run a command inside a container. |
|
||||
| `containers.healthcheck.run` | Trigger or read a runtime health check. |
|
||||
|
||||
`containers.create` must enforce a deny filter for dangerous host options such
|
||||
as privileged mode, unsafe host mounts, host networking, and disallowed
|
||||
capabilities unless the host is explicitly configured to allow them.
|
||||
|
||||
## Volumes
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `volumes.create` | Create an idempotent named volume. |
|
||||
| `volumes.inspect` | Inspect a host volume. |
|
||||
| `volumes.delete` | Delete an unused volume. |
|
||||
|
||||
## Networks
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `networks.create` | Create an idempotent Podman network. |
|
||||
| `networks.list` | List host networks. |
|
||||
| `networks.delete` | Delete an unused network. |
|
||||
|
||||
Bootstrap-created mesh namespace networks are managed by the v5 cluster init
|
||||
flow. Per-resource or compose networks are runtime primitives.
|
||||
|
||||
## Firewall
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `firewall.allow` | Add an allow tuple and persist it. |
|
||||
| `firewall.revoke` | Remove an allow tuple by ID. |
|
||||
| `firewall.list` | List active/persisted allow rules. |
|
||||
| `firewall.reconcile` | Flush and restore firewall state from snapshots. |
|
||||
|
||||
coold is the sole writer for both firewall planes: iptables for cross-host
|
||||
traffic and nft bridge rules for same-bridge traffic.
|
||||
|
||||
## Service discovery and DNS
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `services.register` | Register this host's container endpoint in Corrosion. |
|
||||
| `services.unregister` | Remove an endpoint from Corrosion. |
|
||||
| `services.endpoints` | Read known endpoints for diagnostics. |
|
||||
| `dns.lookup` | Diagnose internal DNS resolution. |
|
||||
| `dns.stats` | Return DNS server status. |
|
||||
|
||||
## Host facts
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `host.info` | Return host/runtime facts for scheduling and debugging. |
|
||||
| `host.stats` | Return CPU, memory, disk, and container stats snapshot. |
|
||||
| `host.containers` | Return host container summaries. |
|
||||
|
||||
## Builder
|
||||
|
||||
Builds are routed through Flux to a host with builder capability. coold should
|
||||
supervise the builder process but should not implement build logic inline.
|
||||
|
||||
| Primitive | Purpose |
|
||||
| --- | --- |
|
||||
| `build.dispatch` | Start a builder request on a capable host. |
|
||||
| `build.cancel` | Cancel a running build request. |
|
||||
| `build.result` | Return or long-poll build result. |
|
||||
|
||||
## Not primitives
|
||||
|
||||
These belong in Coolify, not coold:
|
||||
|
||||
- `deploy.application`
|
||||
- `rollback.deployment`
|
||||
- `create.preview`
|
||||
- `configure.domain`
|
||||
- `authorize.user`
|
||||
- `send.notification`
|
||||
- `render.proxy.config`
|
||||
@@ -0,0 +1,60 @@
|
||||
# v5 Responsibility Split
|
||||
|
||||
Use this as the first check when deciding where a new feature belongs.
|
||||
|
||||
## Rule of thumb
|
||||
|
||||
| Question | Owner |
|
||||
| --- | --- |
|
||||
| What should happen for this user, team, app, database, or deployment? | Coolify |
|
||||
| Which connected host should receive this request? | Coolify chooses, Flux routes |
|
||||
| How do we get a command to a NATed host? | Flux |
|
||||
| Do this concrete operation on this host. | coold |
|
||||
| Is this allowed for this Coolify user/team? | Coolify |
|
||||
| Is this host operation dangerous even if Coolify asked for it? | coold deny filter |
|
||||
|
||||
## Coolify owns
|
||||
|
||||
- Users, teams, roles, RBAC, API tokens, sessions, SSO/OAuth.
|
||||
- Projects, environments, applications, services, databases, servers.
|
||||
- Resource configuration: source, image, build settings, env vars, domains,
|
||||
ports, health checks, resource limits, volumes, schedules, webhooks.
|
||||
- Deployment state machines: pending, building, pulling, creating, starting,
|
||||
health waiting, cutover, running, failed, rollback, cleanup.
|
||||
- Placement and scheduling decisions.
|
||||
- Proxy/ingress configuration rendering and TLS intent.
|
||||
- Secret storage, encryption, resolution, and injection at deploy time.
|
||||
- Business audit, event history, deployment logs, and user-facing status.
|
||||
- Notifications, billing/subscriptions, cloud-provider integration.
|
||||
|
||||
## Flux owns
|
||||
|
||||
- Long-lived coold stream registry keyed by host ID.
|
||||
- Pending request registry keyed by request ID.
|
||||
- Request routing from Laravel's Unix-socket lane to the selected coold stream.
|
||||
- Timeouts, disconnected-host responses, pending-cap protection, and late result
|
||||
handling.
|
||||
- Host-agent authentication for inbound coold streams.
|
||||
|
||||
Flux does not inspect product meaning. `containers.start` is just a frame to a
|
||||
host; Flux does not know it is part of an nginx deployment.
|
||||
|
||||
## coold owns
|
||||
|
||||
- Local Podman access.
|
||||
- Host runtime primitives: images, containers, volumes, networks, logs, exec,
|
||||
health checks, host facts.
|
||||
- Firewall mutation and reconciliation as the sole kernel firewall writer.
|
||||
- Embedded DNS and service-discovery sync.
|
||||
- Writing this host's service endpoint rows to Corrosion.
|
||||
- Builder subprocess supervision when the host advertises builder capability.
|
||||
- Host-level safety checks such as container-create deny filters.
|
||||
|
||||
coold does not store Coolify secrets, users, teams, app ownership, deployment
|
||||
history, billing data, or business audit.
|
||||
|
||||
## Boundary test
|
||||
|
||||
If a different orchestrator could reuse the same operation with its own app
|
||||
model, it probably belongs in coold. If the operation only makes sense because
|
||||
of Coolify's product model, it belongs in Coolify.
|
||||
Reference in new issue
Block a user