Compose materialises a configs: entry with inline content by writing it into
the container and refuses to do so on a read-only service: "cannot create
config ... : `file` is the sole supported option". The container was created
without /etc/alloy/config.alloy and the deployment failed at start.
Keeping the config inline matters more than the read-only rootfs, so the flag
and its tmpfs go. Every other control stays: no privileged, cap_drop ALL with
only DAC_OVERRIDE added, no-new-privileges, the socket proxy, and the limits.
Replace privileged: true with cap_drop ALL plus DAC_OVERRIDE, no-new-privileges,
a read-only rootfs and memory/pid limits. DAC_OVERRIDE is what lets the uid-0
entrypoint create its storage directory and read the journal and /rootfs; every
other capability stays dropped.
Route prometheus.exporter.cadvisor, discovery.docker and loki.source.docker
through a docker-socket-proxy sidecar on 127.0.0.1:2375 instead of bind-mounting
the socket. POST is refused there, so container create and exec are no longer
reachable. NETWORKS is granted because Docker SD resolves network names per
container and returns no targets without it.
Add an alloy validate step to CI and boot the test container with the shipped
capability set, read-only rootfs and proxy rather than --privileged.
container_memory_usage_bytes counts page cache attributed to the cgroup,
which makes disk-heavy containers (e.g. gitea) appear to use ~all host RAM.
Add container_memory_working_set_bytes (the metric Grafana's Docker
integration dashboard expects), plus container_memory_rss,
container_memory_cache, and container_spec_memory_limit_bytes for
breakdown and limit-percentage panels.
`loki.source.journal "default"` no longer pins `path = "/var/log/journal"`.
Upstream omits the field, which lets Alloy default to BOTH
`/var/log/journal` (persistent) and `/run/log/journal` (volatile). The
explicit path silently dropped journal logs on hosts with volatile-only
storage. Matches the canonical Linux Node integration template.
Also bumps the image six minor versions to current stable. Doc records
the 2026-04-26 re-audit.
Copies the exact 157-metric list from the Linux Node integration's
Metrics anchor as the cadvisor keep-list already does for Docker
(16 metrics). Replaces the earlier `drop node_scrape_collector_.+`
rule, which was the integration page's alternate snippet but didn't
ship the explicit allowlist users see in the docs.
`instance:node_num_cpu:sum` from the Metrics section is intentionally
omitted — it's a recording-rule output computed server-side by
Grafana Cloud's ruler, not produced by the agent.
Doc + README updated to point at the Metrics anchors directly so the
source of truth is unambiguous.
Linux-Node integration:
- replace curated keep-list of ~140 node_* metrics with the upstream
drop rule (drops only node_scrape_collector_*); ships the full
~130+ metric set the integration dashboards expect.
- add loki.source.file for /var/log/{syslog,messages,*.log} alongside
the existing journal scrape, matching the upstream config.
- broaden the /var/log mount to cover both pipelines (was journal only).
Docker integration:
- drop container_memory_working_set_bytes from the cadvisor allowlist;
not part of the documented metric set.
README: refresh "What it collects" + "Mounts" tables, document the
syslog-vs-journald duplication caveat for rsyslog hosts.
- network_mode: host so prometheus.exporter.unix reports real host
interfaces (eth0...) rather than the alloy container's veth pair.
- loki.source.journal: set path = "/var/log/journal" explicitly so it
doesn't silently fall through to /run/log/journal on volatile-journal
hosts.
- cadvisor keep-list: add container_memory_working_set_bytes (drives
several panels on the standard Docker dashboard).
- Drop /dev/kmsg device + extra_hosts:host.docker.internal — neither is
needed by the current keep-lists, and host-network mode makes the
extra_hosts entry meaningless.
- CI: extend Alloy validation beyond `fmt` (syntax-only) by booting
alloy with the embedded config and asserting it stays running, which
catches bad component refs / wrong arg names that fmt accepts.
- README: refresh Mounts table + Security note to match.
prometheus.exporter.unix now reads /rootproc and /rootfs (the existing
host bind mounts) instead of the container's own namespace, so the
filesystem + process metrics actually describe the host. This makes
pid:host unnecessary, so remove it — privileged is enough and pid:host
exposes every host process inside the container.
cadvisor regex: drop fs_inodes/fs_limit/network_tcp_usage; add fs_reads
+ fs_writes + network_(receive|transmit)_(errors|packets_dropped)_total
to match the standard Grafana Cloud docker integration dashboard.
Compose:
- pid: host so prometheus.exporter.unix sees host /proc (cpu, mem,
load, processes) instead of the container's namespace.
- mount /etc/machine-id so loki.source.journal has a stable host id.
bind-mounting /dev/kmsg under volumes: creates the node but leaves the
device-cgroup controller blocking the read (EPERM). cap_drop=[ALL]
clears the default device allow-list, so even with CAP_SYSLOG the
kernel refuses. moving it under devices: adds the cgroup allow rule
alongside the bind-mount, which is what cadvisor actually needs.
clears two startup warnings:
- cadvisor "Could not configure a source for OOM detection" — needs
/dev/kmsg bind-mount and CAP_SYSLOG (kernel.dmesg_restrict=1 default)
- node-exporter "Failed to open /run/udev/data" — diskstats collector
enriches node_disk_* with model/serial/WWN labels from udev
both mounted read-only. CAP_SYSLOG added alongside DAC_OVERRIDE.
grafana cloud loki rejects entries older than 7 days with HTTP 400.
loki.source.docker has no "tail since" option, so on first start it
replays logs from each container's start time — long-running
containers (coolify-proxy, traffmonetizer) produced weeks of backlog
that loki refused to ingest.
insert a loki.process drop stage (older_than=144h) between the docker
source and loki.write.gc so stale entries are filtered before egress.
journal source already bounded by max_age=12h — no filter needed there.
container runs as root (0:0) but cap_drop=[ALL] stripped DAC_OVERRIDE,
so mkdir on /var/lib/alloy/data (alloy-owned in the image) failed with
permission denied. add it back (only DAC_OVERRIDE, nothing else) and
mount /etc/machine-id ro for a stable journal host id. annotate every
volume with the component that uses it.
- merge discovery.relabel "metrics" into prometheus.relabel "filter"
(instance label + keep-filter in one component)
- move logs 'instance' label to loki.write external_labels (DRY, one source)
- drop alloy container log exclusion (~500kb/day trivial; adds visibility
into alloy's own health via loki)
net: -5 lines, one fewer component, same functionality
correctness:
- drop dead ALLOY_HOSTNAME env passthrough (alloy uses constants.hostname)
- tighten netdev regex so iface names like 'lore0' no longer match 'lo'
- widen alloy-container drop to /alloy.* (catches renamed variants)
more info for same cost (~80 extra series, still ~1.6k of 10k):
- add MemFree, disk IOPS (reads/writes_completed), disk saturation
(io_time_weighted), inode tracking (filesystem_files, _files_free),
network drops, container working_set memory, container_last_seen,
machine_scrape_error
- add journal unit + level labels (log querying by service/severity)
- docker discovery refresh 60s -> 5s (fast pickup of new containers)
efficiency:
- loki batch_wait=5s, batch_size=1MiB (~5x fewer push requests)
layout:
- drop redundant expose: [12345] (other containers can reach it anyway)
- node_exporter (host) + cadvisor (containers) via one scrape job
- docker container logs + systemd journal to loki
- filter regex keeps only metrics needed for standard dashboards
- sized for grafana cloud free tier (~10k active series cap)
- shell env driven (no .env file); fail-fast on missing vars
- least-privilege: cap_drop all, no privileged, no pid host, ui not published
tiennm99