Storage & backups
Artifacts live in blob stores; metadata — repositories, components, assets, users, permissions — lives in Postgres. Each repository is bound to one blob store when it is created, so you pick storage per repository: a hot npm proxy on EBS, a large Docker registry on S3, shared release files on EFS.
Content is addressed by SHA-256 and deduplicated within a store: the same layer pushed to ten images, or a wheel cached by two proxies, is stored once. Reference counts are tracked in Postgres.
Store types
Section titled “Store types”file — PVC, EBS, EFS, NFS
Section titled “file — PVC, EBS, EFS, NFS”A directory on a mounted volume. In Kubernetes this is a PersistentVolumeClaim, and its StorageClass decides the backing:
| Backing | StorageClass example | Access mode | Replicas |
|---|---|---|---|
| EBS (gp3) | gp3 (EBS CSI) |
ReadWriteOnce | 1 (Deployment uses Recreate) |
| EFS | efs-sc (EFS CSI) |
ReadWriteMany | many |
| Any RWX (NFS, CephFS, Filestore…) | — | ReadWriteMany | many |
| Plain PVC / hostPath / local | cluster default | ReadWriteOnce | 1 |
Layout: blobs/sha256/<aa>/<digest> plus temp/ for in-progress Docker
uploads. Writes go to a temp file and are atomically renamed, so readers never
see partial blobs. The Helm chart refuses to render more than one replica
when any file store uses ReadWriteOnce.
s3 — AWS S3 and compatibles
Section titled “s3 — AWS S3 and compatibles”type: s3s3: bucket: my-entrepot-artifacts region: eu-west-2 prefix: prod # optional key prefix endpoint: "" # set for MinIO / Ceph RGW / R2 forcePathStyle: false # true for MinIO accessKeyId: "" # empty = AWS default chain (IRSA, env, instance profile) secretAccessKey: "" # write-only, encrypted at rest redirectDownloads: true # 307 large downloads to presigned S3 URLs- On EKS use IRSA: annotate the chart’s service account with the role ARN.
The role needs
s3:GetObject,s3:PutObject,s3:DeleteObject,s3:ListBucketands3:AbortMultipartUploadon the bucket (see the IAM policy). - Uploads are spooled to the pod’s
/tmp(anemptyDir, seetmp.sizeLimit) while hashing, then sent with multipart upload. - With
redirectDownloads, large downloads get a307to a presigned URL, so the pods never stream large artifacts. Clients must be able to reach the S3 endpoint.
Which store for which repository?
Section titled “Which store for which repository?”| Repository | Good fit | Why |
|---|---|---|
| Docker hosted/proxy | S3 with redirectDownloads |
large, write-once layers; offloads bandwidth |
| npm / PyPI proxies | EBS or EFS | many small files, latency-sensitive metadata |
| Release files, Helm, Apt, Yum | S3 or EFS | shared, durable, easy to replicate |
| Evaluation / single node | the default PVC |
simplest |
Declaring and managing stores
Section titled “Declaring and managing stores”Declare stores in configuration (and the Helm chart’s blobStores); they are
created or updated at startup:
blobStores: - name: default type: file volume: {storageClass: gp3, accessMode: ReadWriteOnce, size: 200Gi} - name: shared type: file volume: {storageClass: efs-sc, accessMode: ReadWriteMany, size: 1Ti} - name: s3 type: s3 s3: {bucket: my-bucket, region: eu-west-2, redirectDownloads: true}The chart mounts each file store’s PVC at /data/<name> (or reuses
volume.existingClaim).
curl -fsS -H "Authorization: Bearer $ADMIN" -X POST https://entrepot.example.com/api/v1/blob-stores -d '{ "name": "s3-eu", "type": "s3", "s3": {"bucket": "my-bucket", "region": "eu-west-2", "redirectDownloads": true}}'curl -fsS -H "Authorization: Bearer $ADMIN" -X POST https://entrepot.example.com/api/v1/blob-stores -d '{ "name": "scratch", "type": "file", "file": {"path": "/data/scratch"}}'curl -fsS -H "Authorization: Bearer $ADMIN" https://entrepot.example.com/api/v1/blob-stores/s3-eu # includes usageresource "entrepot_blob_store" "s3_eu" { name = "s3-eu" type = "s3" s3 = { bucket = "my-bucket" region = "eu-west-2" redirect_downloads = true }}filestores must live understorage.allowedRoots(/datain the chart), so an API caller can’t point a store at arbitrary paths. The volume itself must already be mounted there.- A store is checked to be reachable and writable before it is saved.
- A store can’t be deleted while repositories use it (
409). GET /api/v1/blob-stores/{name}reports usage: blob count, total bytes and the number of repositories.
Garbage collection
Section titled “Garbage collection”Deleting assets only drops references. Unreferenced blobs are garbage-collected every 15 minutes, after a one-hour grace period, so in-flight uploads are never collected. Run it on demand with:
curl -fsS -H "Authorization: Bearer $ADMIN" -X POST https://entrepot.example.com/api/v1/maintenance/gc# → {"removed": 42}# or, inside a pod / with the binary:entrepot gc -config config.yamlBackups
Section titled “Backups”- Postgres holds the metadata: back it up with your usual tooling (RDS
snapshots,
pg_dump, CloudNativePG backups). - Blob stores: snapshot the volumes (EBS snapshots, AWS Backup for EFS) or enable S3 versioning and replication.
- The secrets key (
secrets.key, orauth.sessionSecretif you don’t set one): keep it with the backups, or the encrypted settings can’t be restored.
Blobs are immutable and content-addressed, so an incremental copy is always consistent once the matching database backup is restored. Take the database backup first and the blob snapshot after it: blobs without a database row are simply garbage-collected, while a database row whose blob is missing would be a broken asset.