Evaluate the Blob v2 key inventory safely

This operations page is for administrators of self-hosted Workspace installations. Use it when you prepare a root-key rotation, consider removing an old key from the root-key bundle, or need to understand a Blob inventory blocker.

After the inspection, you will know whether Blob v1 or Blob v2 data still depends on an old kid. Remove a key only when distribution, drain, and the specific removal gate have no open finding.

Choose the correct tool

The root-key inventory requires privileged local access to the encrypted server configuration, the root-key bundle, PostgreSQL, and the storage root. It is not a tenant-bound remote operation.

TaskTool and boundary
Operate a running server through authorized tenant APIsUse nucli. It does not read root-key bundles or scan local Blob storage.
Run a documented local numin contractUse only the commands described in numin. The current public numin contract does not include root-key inventory.
Start the Workspace serverUse the server entrypoint only as the runtime. Do not treat it as a general administration CLI.
Produce a Blob v2 root-key inventoryUse only the approved, version-matched operations procedure supplied for your installation. Stop if your deployment provides no such contract. Do not replace it with direct database access or custom storage scripts.

This task does not need a dedicated nucli skill. Such a skill could only guide an existing authenticated server contract; it must not reimplement local cryptography or storage scanning in the client.

What the inventory protects

Workspace uses a versioned root-key bundle. A kid identifies a key version, not the secret key material. New writes use the active kid; older Blobs may still require an older decrypt-enabled kid.

blob_key_inventory bundle Root-key bundle active_kid + decrypt_kids inventory Local key inventory bundle->inventory gates Distribution, drain, and removal gates inventory->gates database PostgreSQL StorageObject metadata database->inventory storage Storage root Blob v1 and Blob v2 auth Authenticated Blob v2 header + metadata and object binding storage->auth auth->inventory decision Keep the old kid or remove it under control gates->decision

The inventory separates three questions:

  1. Which container format is present?
  2. Which authenticated kid does the container require?
  3. Does missing, invalid, or uncovered data block key removal?

Why the cleartext header is not enough

A Blob v2 container starts with a bounded cleartext header. It contains only the format nucleus-blob-container.v2, the crypto domain nucleus/blob.v2, the kid, a random salt, and the chunk size. Tenant, store, object, filename, media type, and payload metadata remain encrypted.

The inventory never counts the cleartext kid directly. It first resolves the existing root-key domain key and then authenticates the first encrypted metadata frame:

  • AES-GCM binds the exact header as Additional Authenticated Data.
  • The Blob key is derived from the domain key, salt, tenant ID, store ID, and object ID.
  • The decrypted metadata frame must match the expected tenant, store, object, and media type.
  • The kid enters the inventory count only after authentication succeeds.

Changes to the header, kid, salt, or object binding therefore fail closed. An unknown or no longer decrypt-enabled kid is never counted as valid data.

What the bounded check does not do

The key inventory does not read every payload frame. It is not a complete document-integrity check.

CheckKey inventoryFull Blob read
Format and header contractYesYes
Authenticated kidYesYes
Tenant, store, and object bindingYesYes
First encrypted metadata frameYesYes
Every payload frameNoYes
Final trailer, size, and SHA-256NoYes
Data after the trailerNoYes

This boundary is safe for key retirement. Once the authenticated metadata frame proves that a container uses an old kid, that dependency is counted. Later payload corruption cannot hide the dependency on the old key.

Prerequisites

Before you request an inventory report, confirm that:

  • you use the exact Workspace version that matches the database and Blob v2 format of the installation,
  • the approved procedure has read-only access to the encrypted configuration, its configuration key, PostgreSQL, and the complete storage root,
  • every expected server and companion process reports its bundle fingerprint and active kid through the approved observability path,
  • the current root-key bundle remains backed up and contains every required decrypt_kid,
  • a current backup and documented change window exist.

Never place root-key values, derived keys, configuration keys, database passwords, or Blob payloads in tickets, chats, shell histories, or inventory records. A kid and bundle fingerprint are identifiers rather than key material, but you should still treat them as operational metadata.

Read the inventory result

A complete report contains at least this Blob view:

FieldMeaningSafe response
blob_objectsNumber of cryptographically classified Blob v1 and Blob v2 objectsCompare it with the expected storage population.
blob_formatsCount per authenticated or canonically inspected formatnucleus-blob-container.v2 must appear after v2 writes exist.
blob_kidsValid Blob count per authenticated kidAny count for an old kid blocks its removal.
blob_missingDatabase objects without a reachable payloadA value above zero blocks drain and retirement.
blob_invalidUnknown, manipulated, or unauthenticated containersA value above zero blocks drain and retirement.
coverage_pending or scope_pendingRoot-key domains without a complete inventory or runtime contractA non-empty value blocks removal.

The aggregate gates answer different questions:

GateWhat it checksRequired result
distributionEvery expected process uses the same bundle state and active kid.status:clear
drainNo invalid or missing data remains and no relevant write still uses an old key.status:clear
retirementDistribution, drain, and every individual removal gate are clear.status:clear
retirement_gate[<kid>]Dependencies of one non-active kid.For the key you intend to remove: status:clear

active_write_key is not an error. The active kid must not be removed and therefore does not receive a normal clear status.

Handle blockers safely

An old kid still has Blob data

Keep the kid in decrypt_kids. Blob v2 currently has no generally approved relocation, downgrade, or rekey path. Do not move the file manually and do not replace it with a Blob v1 container.

Wait for an explicit, version-bound, dry-run-capable offline contract. A normal product write may replace a Blob only when its product contract explicitly permits that operation.

blob_missing is above zero

Do not delete the database row to make the count disappear. Inspect backups, the storage mount, permissions, and rollout completeness. Restore a missing payload only through a documented restore or recovery contract.

blob_invalid is above zero

Preserve the container and the redacted report for investigation. Check the binary version, bundle, storage consistency, and possible incomplete restore first. Do not try unrelated keys and do not edit the header. An invalid container is a blocker, not a zero-count object.

Distribution is blocked

Do not activate or remove a key. Confirm that every expected process role reports the same bundle fingerprint and active kid. A locally verified state is not enough for a distributed rollout.

Coverage is open

Treat the uncovered domain as unknown cold data. Keep every old key until an approved inventory or runtime-observation contract closes the gap.

Follow the safe decision sequence

  1. Obtain the report through the approved, version-matched, read-only operations procedure for your installation.
  2. Inspect distribution and every expected process role first.
  3. Inspect blob_missing, blob_invalid, the format counts, and the KID counts.
  4. Inspect drain, followed by the removal gate for the specific old kid.
  5. Record identifiers, counts, statuses, and redacted reasons only. Never record secret key material or payloads.
  6. Remove the old kid only through an explicitly approved bundle contract and only when both retirement and the specific removal gate are clear.
  7. Roll out the changed bundle completely, then verify the fingerprint, active kid, reads, and error counters again.

Confirm success

The inspection is complete when:

  • nucleus/blob.v2 appears as a fully inventoried at-rest domain,
  • blob_missing=0 and blob_invalid=0,
  • blob_kids contains no data for the kid you intend to remove,
  • every expected process role reports a consistent bundle state,
  • distribution, drain, retirement, and the specific removal gate have the required clear status,
  • the approved rollout produces no unknown-kid, verify, decrypt, or read errors.

An interrupted or partial report is not success. Keep the old kid until you can repeat the inspection completely.

Next steps

  • Use numin only for its explicitly documented local offline contracts.
  • Follow the approved backup, rollout, and root-key procedures supplied for your installation. Stop if no version-matched procedure exists.