Agent configuration

Pepper 0.2.0 uses strict TOML: unknown sections and fields are rejected. Relative paths are resolved from the agent's working directory, not from the configuration file's directory.

Complete example

[node]
name = "node-1"                         # default: pepper-node
listen_addr = "127.0.0.1:9001"          # default: 127.0.0.1:9000
# advertise_addr = "10.0.0.10:9001"
# failure_domain = "rack-a"

[node.placement_labels]
# region = "us-east"
# zone = "us-east-1a"
# rack = "rack-a"
# disk_class = "ssd"

[data]
path = "/var/lib/pepper"                # default: ./.pepper

[identity]
key_path = "/var/lib/pepper/node.key"   # default: <data.path>/node.key
generate_if_missing = true

[api]
bind_addr = "127.0.0.1:9080"            # loopback required

[[storage.locations]]
path = "/srv/pepper"
max_capacity_bytes = 100000000000

[network]
bootstrap_peers = ["10.0.0.11:9001"]

[namespace]
enabled = true                           # default: false
consensus_enabled = true                 # default: false
max_namespace_groups = 128
max_consensus_log_bytes = 268435456
max_namespace_write_rate = 1000
max_consensus_command_bytes = 1048576
heartbeat_interval_ms = 100
election_timeout_min_ms = 300
election_timeout_max_ms = 600
snapshot_after_logs = 1000
max_logs_after_snapshot = 128
checkpoint_log_bytes = 67108864
checkpoint_restore_target_ms = 2000
max_staging_leases = 10000
max_staging_bytes = 17179869184
staging_ttl_seconds = 900
read_lease_ttl_seconds = 3600

[replication]
default_factor = 3
repair_interval_seconds = 30

[erasure]
enabled = false
min_size_bytes = 67108864
data_shards = 6
parity_shards = 3

[compute]
enabled = true
runtime = "firecracker"
max_concurrent_jobs = 1
work_dir = "/var/lib/pepper/compute"
firecracker_binary = "/usr/local/bin/firecracker"
firecracker_jailer_binary = "/usr/local/bin/jailer"
firecracker_enable_jailer = false
firecracker_jailer_uid = 65534
firecracker_jailer_gid = 65534
firecracker_jailer_chroot_base = "/srv/jailer"
firecracker_strict_sandbox = true
firecracker_allow_untrusted_rootfs = false
firecracker_allowed_rootfs_cids = []
# firecracker_kernel_image = "/var/lib/pepper/vmlinux"
firecracker_memory_mib = 128
firecracker_vcpu_count = 1
firecracker_max_input_bytes = 1073741824
firecracker_max_output_bytes = 1073741824
firecracker_cgroup_enabled = true
firecracker_cgroup_base = "/sys/fs/cgroup/pepper"

[auth]
# Required for non-loopback P2P; file must be >=32 bytes and mode 0600.
# cluster_secret_path = "/etc/pepper/cluster.secret"
# api_bearer_token = "replace-with-a-secret"

[s3]
enabled = false
region = "us-east-1"
# access_key_id = "pepper"
# secret_access_key_path = "/etc/pepper/s3.secret"
max_clock_skew_seconds = 900

[limits]
max_block_bytes = 67108864
max_object_bytes = 1073741824
max_compute_timeout_seconds = 600
http_requests_per_minute = 6000
rpc_requests_per_minute = 6000
erasure_repair_max_concurrent_shards = 2
# erasure_repair_bytes_per_second = 10485760

[logging]
format = "pretty"                        # pretty or json

Configuration sections

[node]

Name, P2P listen/advertise addresses, failure_domain, and placement_labels.

[data] / [identity]

Metadata/state path and optional identity key path/generation policy.

[api]

Loopback-only HTTP bind address.

[[storage.locations]]

One or more unique block-store paths with explicit capacity ceilings.

[network]

Bootstrap peer socket addresses.

[namespace]

Feature gates, Raft timing/log limits, group/write admission, staging, and read leases.

[replication] / [erasure]

Default replicated durability, repair cadence, and Reed-Solomon policy.

[compute]

Firecracker, jailer, rootfs, VM, I/O, concurrency, and cgroup controls.

[auth]

Cluster secret file and optional local API bearer token.

[s3]

Opt-in region, access key, secret file, and signing clock skew.

[limits]

Block/object/compute, HTTP/RPC, and erasure-repair limits.

[logging]

pretty or json output.

Namespace defaults and rules

enabled / consensus_enabled

Both default false. consensus_enabled requires enabled; S3 requires both.

max_namespace_groups

128 groups hosted by this node.

max_consensus_log_bytes

256 MiB per-node consensus log admission ceiling.

max_namespace_write_rate

1,000 writes in the configured admission window.

max_consensus_command_bytes

1 MiB; must remain below max_consensus_log_bytes.

heartbeat / election

100 ms heartbeat, 300 ms minimum election, 600 ms maximum; must satisfy 0 < heartbeat < min < max.

snapshot_after_logs / max_logs_after_snapshot

1,000 and 128; retained post-snapshot logs must stay below the snapshot threshold.

checkpoint_log_bytes

64 MiB; must remain below max_consensus_log_bytes.

max_staging_leases / max_staging_bytes

10,000 leases and 16 GiB of staged content.

staging_ttl_seconds / read_lease_ttl_seconds

900 seconds and 3,600 seconds.

Security and validation

api.bind_addr must be loopback. A non-loopback P2P listener requires a cluster secret containing at least 32 bytes with Unix mode 0600 or stricter.
node.advertise_addr is required for an unspecified listener such as 0.0.0.0 and must be routable unicast.
At least one unique, nonzero-capacity storage location is required. Replication is 1–64; the repair interval is 5–86,400 seconds.
Erasure data/parity counts must be nonzero, parity cannot exceed data, and their sum cannot exceed 32.
Compute runtime must be firecracker. Input/output caps are 1 byte–16 GiB; jailer UID/GID must be nonzero when enabled.
S3 access keys are visible ASCII, regions contain letters/digits/hyphens, a secret path is required, and clock skew is 1–3,600 seconds.
max_block_bytes is at most 64 MiB; max_object_bytes is at most 1 TiB; configured rate/concurrency limits must be nonzero.

Firecracker rootfs policy

Each job supplies rootfs_cid. By default it must appear in firecracker_allowed_rootfs_cids and use a raw, object, or erasure CID. firecracker_allow_untrusted_rootfs=true is an explicit development-only opt-out because host tools parse the ext4 image before boot. The rootfs must contain executable /pepper-guest-agent.