Monitoring — Prometheus + Grafana

Scrape the node's execution-layer (bera-reth) and consensus-layer (beacond/CometBFT) Prometheus metrics and render them in Grafana. This is the deployable version of the README "Monitoring: Prometheus + Grafana" row.

The deploy artifacts are in l1/from-source/deploy/monitoring/. This page is the operator-facing version of that directory's README.md.

Deploy artifact — not yet run on real infra

This stack is config-validated but has not been run against live nodes in this repo, and the public testnet is not live yet. Pin specific image releases for production (the artifact uses prom/prometheus:v3.1.0 and grafana/grafana:11.4.0).

What's in the stack

File	Purpose
`docker-compose.yml`	`prometheus` + `grafana` (UIs bound to loopback)
`prometheus.yml`	scrape jobs for `krypton-el:9001` + `krypton-cl:26660`
`grafana/provisioning/datasources/prometheus.yml`	auto-wires the Prometheus datasource

Prerequisites — enable metrics on the nodes

EL (bera-reth): already on — --metrics=0.0.0.0:9001 in the node compose. Verify the path:
bash
```
curl -s http://<el>:9001/metrics | head
```
If empty, try /debug/metrics/prometheus and adjust prometheus.yml accordingly.
CL (beacond/CometBFT): on by default. beacond's baked config forces
toml
```
[instrumentation]
prometheus = true
prometheus_listen_addr = ":26660"
```
so the CometBFT /metrics endpoint is live on :26660 with no config.toml edit. The node compose publishes 26660 to loopback; on the shared krypton-web net it's reachable as krypton-cl:26660.

CL metrics are on by default

The Krypton CL (validator / RPC nodes) exports CometBFT Prometheus metrics on :26660 out of the box — beacond's config provider hard-sets instrumentation.prometheus = true. Only a node that explicitly set prometheus = false would have nothing to scrape. Keep 26660 private (loopback/VPN), never public.

Run

bash

docker network create krypton-web 2>/dev/null || true   # shared net the node stack joins
cp .env.example .env   # set GRAFANA_ADMIN_PASSWORD (or export it)
docker compose up -d
# Grafana  -> http://127.0.0.1:3000  (admin / $GRAFANA_ADMIN_PASSWORD)
# Prometheus -> http://127.0.0.1:9090

.env requires GRAFANA_ADMIN_PASSWORD (compose fails fast without it); GRAFANA_ADMIN_USER defaults to admin, and sign-up is disabled.

Reaching the node's metrics

Prometheus must resolve the node's metrics ports by name. Either:

attach the node stack to the shared krypton-web network and scrape krypton-el:9001 / krypton-cl:26660 (the defaults in prometheus.yml), or
publish those ports to the host and point prometheus.yml at host.docker.internal:<port>.

The scrape config carries external_labels (network: krypton-testnet, chain_id: "473374") and one job per layer — add a target line per node you run.

Dashboards

The datasource is provisioned automatically. Import community dashboards rather than hand-rolling:

Reth — the official Reth Grafana dashboard (ships in reth/etc/grafana/dashboards/).
CometBFT — a CometBFT/Tendermint consensus + p2p dashboard (e.g. from Grafana.com).

What to watch

These tie to the determinism / liveness gates:

block height advancing in lockstep across validators
CometBFT round/step + missed precommits
peer counts and mempool depth
EL state-sync lag
INFLATION_SINK / FEE_SINK growth as a liveness signal

Security

Keep both UIs off the public internet

Prometheus (:9090) and Grafana (:3000) bind to loopback only in this artifact. Expose them only via SSH tunnel / VPN, or behind the Caddy reverse proxywith auth. Never publish them directly.

Set a strong GRAFANA_ADMIN_PASSWORD; store it in your secret manager, not in git — see Secrets (Vault / KMS).
Prometheus retention is 15d in the artifact; size prometheus_data accordingly.

Monitoring — Prometheus + Grafana ​

What's in the stack ​

Prerequisites — enable metrics on the nodes ​

Run ​

Reaching the node's metrics ​

Dashboards ​

What to watch ​

Security ​

See also ​