Buckets, S3, and filesystems

Pepper 0.2.0 adds two namespace-backed data models: versioned object buckets and structurally shared filesystem trees. The optional S3 gateway maps common AWS S3 operations onto durable Pepper buckets.

Native buckets

pepper bucket create <ALIAS>
pepper bucket put <BUCKET> <KEY> <PATH> [--content-type TYPE] [--if-generation N] [--if-cid CID] [--request-id ID]
pepper bucket get <BUCKET> <KEY> -o <PATH> [--revision N]
pepper bucket head <BUCKET> <KEY> [--revision N]
pepper bucket delete <BUCKET> <KEY> [PRECONDITIONS]
pepper bucket list <BUCKET> [--prefix TEXT] [--limit N] [--cursor CURSOR] [--revision N]
pepper bucket versions <BUCKET> <KEY>

Each put stores content as an ordinary Pepper object and publishes a canonical bucket descriptor. Overwrites and deletes append immutable versions; deletes create tombstones instead of erasing history. Listing is lexicographic, bounded, and root-bound for stable pagination.

Enable S3

[namespace]
enabled = true
consensus_enabled = true

[s3]
enabled = true
region = "us-east-1"
access_key_id = "pepper-dev"
secret_access_key_path = "/etc/pepper/s3.secret"
max_clock_skew_seconds = 900

S3 is opt-in and shares the loopback HTTP listener. The secret file is read when the agent starts. Authentication supports SigV4 Authorization headers, presigned URLs, browser POST policies, and AWS streaming payload signatures. Configure clients with Pepper's API URL as a custom endpoint and the same region and credentials.

AWS_ACCESS_KEY_ID=pepper-dev \
+AWS_SECRET_ACCESS_KEY='<secret>' \
+aws --endpoint-url http://127.0.0.1:9080 --region us-east-1 \
+  s3api create-bucket --bucket assets

AWS_ACCESS_KEY_ID=pepper-dev AWS_SECRET_ACCESS_KEY='<secret>' \
+aws --endpoint-url http://127.0.0.1:9080 --region us-east-1 \
+  s3 cp ./photo.jpg s3://assets/photo.jpg

S3 compatibility in 0.2.0

Addressing

Path-style and virtual-hosted bucket routing.

Objects

PUT/GET/HEAD/DELETE, byte ranges, metadata, content type, conditional requests, copy, and multi-delete.

Listing

ListBuckets and ListObjectsV2 with bounded pagination and prefix/delimiter behavior.

Integrity

Content-MD5 plus CRC32, CRC32C, SHA-1, and SHA-256 checksum headers.

Multipart

Create, upload/part-copy, list parts/uploads, complete, abort, durable recovery, and incomplete-upload cleanup.

Bucket controls

Tagging, CORS, and lifecycle rules for aborting incomplete multipart uploads.

S3 compatibility is intentionally scoped. It is not a claim of complete AWS S3 parity: ACLs, IAM, server-side encryption, object lock, ListObjectVersions, and many cloud control-plane features are not implemented.

Immutable filesystem trees

pepper fs create <ALIAS>
pepper fs commit <FS> <SOURCE> --base-revision N [--message TEXT] [--request-id ID]
pepper fs checkout <FS> <DESTINATION> [--revision N]
pepper fs history <FS>
pepper fs diff <FS> <A> <B>
pepper fs restore <FS> <REVISION> <DESTINATION>
pepper fs rollback <FS> <REVISION> [--request-id ID]
pepper fs clone-from-root <FS> <ROOT_CID> [--request-id ID]

A commit uploads regular files as objects, builds canonical inode and Merkle-map directory DAGs, and atomically advances the filesystem revision. Unchanged subtrees retain their CIDs, making history and cloning structurally shared. Checkout and restore write through a temporary tree before replacing the destination.

Filesystem boundaries

Regular files and directories are supported, including ordinary permission bits.
Symlinks, hard links, sparse files, devices, sockets, setuid/setgid/sticky bits, ownership, ACLs, extended attributes, and platform-specific attributes are rejected or not preserved.
Paths must be UTF-8. Restoration rejects unsafe traversal and destination symlink attacks.
fs mount is a placeholder in this release and returns an error; use checkout, edit, and commit.