Previewkernable is not released yet: these docs describe work in progress, and things may change before 1.0.Not released yet.Request early access →
Skip to content

Data over time ​

The log is kept forever, so everything about its data is designed to change without rewriting it.

Where kernable keeps what ​

DataKeptDeletableVerifiableWho reads it
events (the log)foreverneverhash chain, worker signatureslisteners and subscriptions, per grant
commandsforever, with their outcomeneverclient signaturesthe kernel; kernable verify
vault valuesuntil the person is erased; secrets until their command completesby an erasing eventreferences in the logkeys granted vault_read
snapshotsuntil replaced, erased, or of an old schemayes: a cacheworker signatureworkers
filesuntil deletedyes, but not by an erasuresigned requestskeys granted data_read
cache valuesuntil they expire or the kernel restartsyessigned requestskeys granted cache
read model storesin your processyours to managenothe read model

The log only grows: there is no compaction, on purpose, because a verifiable history cannot lose events. Events are small JSON, so a store of millions of events is usually a few gigabytes. Keep data that should expire in the cache, large binaries in files, and back up the store and the vault (Back up and restore).

Event versions ​

Every event carries its payload's schema version (from 1), signed and part of the hash. To change a payload, raise the version and add an upcast from the old one:

ts
export const shopDomain = domain("shop");
shopDomain.event("ItemAdded", 2).upcast(1, (d: { sku: string; qty: number }) => ({ sku: d.sku, count: d.qty }));

Aggregates and listeners get every event upcast to the version they know; stored events stay as they were:

PositionStored asSchemaA reader on version 2 gets
41{"sku":"apple","qty":2}1{"sku":"apple","count":2}, upcast
57{"sku":"pear","count":3}2{"sku":"pear","count":3}

When to raise the version: whenever a reader on the old version would misread the new payload, by renaming, removing, or changing the meaning or type of a field. A new optional field needs no new version if every reader can ignore it; it does if old readers must not miss it. Upcasts chain: a version-1 payload reaches version 3 through upcast(1, …) and upcast(2, …).

Rules:

  • A process that meets a newer version stops instead of misreading it: it logs, retries, and keeps its position until it is updated. Listeners after it wait too, because of the order. So deploy readers before writers.
  • A worker cannot write an older version than the store holds: the kernel refuses it. Old and new workers never write side by side.
  • A handler marked raw gets events as stored, for archives and exports.

Formats and protocol ​

  • Protocol versions. Every request carries the protocol version of its SDK; the kernel accepts a range and names the side to update otherwise.
  • Store formats. A store records its data format. A kernel opens only its own format and refuses others at startup. A frozen store of the current format is kept in kernable's tests and must stay readable.
  • Before the first release the protocol and every format stay at version 1 and change in place, without migrations. Use SDKs and a kernel of the same build, and start a new store after updating a development build: an older one may be refused or read wrongly.

Personal data: the vault ​

Events cannot be deleted, but personal data must be deletable. Fields declared personal (with the person they belong to) or secret go to the vault instead of the log; the log keeps a salted reference:

json
{ "name": { "$vault": "41a4ffe62b20…" }, "email": { "$vault": "c56522f357b1…" } }
Events cannot be deleted, but personal data can: it goes to the vault, and the log keeps only a reference.
EventPersonal dataRejection / mismatch
  • Workers and listeners granted vault_read get the values; everyone else gets null.
  • Secrets are deleted once their command completes.
  • An erasing event deletes a person's values in every stream and domain of its tenant, and the snapshots that may contain them. Read models must forget the person when they see it; rebuilt, they read null.
  • The vault keeps each value encrypted with the vault key, together with its subject and whether it is secret. An erasure leaves a proof without the person: the SHA-256 of the subject, the erasing command, how many values went, and when.
  • The log records every erasure that happened. The kernel stores the erasing event first and erases right after, before it answers; after a crash in between it finishes the erasure when it starts, before it serves anything. An erasing command that is rejected erases nothing.
  • Files and cache values are not erased with the person: they are not part of the vault. Delete a person's files yourself when you erase them (Erase a person).

Snapshots and read model stores ​

  • Snapshots of aggregates live in their own store, not in the log: a cache that workers send every 100 events (snapshotEvery) and rebuild whenever it is gone or of another schema. By default the kernel keeps them in SQLite next to the event store, so they survive a restart. Without a usable snapshot, a worker gets all of an aggregate's events, in several messages if they do not fit one. The kernel accepts a snapshot only from the worker that ran the command, signed by it, and only at exactly the version after the command's events; it stores it with the hash of that event. A worker that cannot read a snapshot (an unknown schema) rejects it, and the kernel drops it and sends all events instead.
  • Read models can keep their state in a store in their own process: data and position are written together per batch, so a restart continues where it left off. A new schema rebuilds from the first event.