Previewkernable is not released yet: these docs describe work in progress, and things may change before 1.0.Not released yet.Request early access →
Skip to content

Protocol ​

The kernel speaks gRPC over a Unix socket or TCP (with mutual TLS), package kernable.v1. This page is generated from the comments in proto/kernable/v1/kernable.proto. The byte layouts of everything that is signed are in Signatures, rejection codes and status codes in Errors. To implement a client in another language, follow Write an SDK.

How to read this page ​

The sections follow the file. Each service lists its RPCs: who calls them, the permission (grant action) they need, and how they fail. Each message lists its fields with their protobuf numbers; a field without a description means what its name says. Qualified means domain.Type; an unqualified type comes with a separate domain field.

Call flow per role ​

Every call carries the caller's grant in the metadata entry kernable-grant, and protocol in its request or first stream message.

  • Client: signs a Command and calls CommandService.Submit. The kernel checks the signature, the grant and the middlewares before, queues the command, and returns a CommandResult once the command is accepted, committed or settled (see Wait). Queries go through QueryService.Query to a read model. EventService.Subscribe streams the log without registering. Files and cache values go through DataService and CacheService.
  • Worker: opens WorkerGateway.Session, sends Register, signs the Challenge and gets Registered. Then it receives an Execute per command and answers each with a signed Result (new events, optionally a snapshot) or Rejection.
  • Listener: opens ListenerService.Listen, sends Hello, signs the Challenge and gets Ready. Then it receives Batches and acknowledges each up_to with an Ack. A read model also answers QueryCalls with a QueryAnswer, a middleware answers MiddlewareCalls with a Verdict.
  • Deploy tooling and root keys: MembershipService.Apply sets what a service consists of; TrustService.Revoke revokes a key. Grants have no RPC: they travel with every call.

Conventions ​

kernable protocol, version 1. Payloads (of commands, events, queries, answers and snapshots) are opaque bytes to the kernel. Signatures cover the byte layouts in SIGNING.md, never the protobuf encoding.

Protocol versions: every request and stream opening (SubmitRequest, QueryRequest, SubscribeRequest, DataRequest, CacheRequest, RevokeRequest, ApplyRequest, RetryRequest, Register, Hello) carries protocol, the version its sender speaks. It is never signed. The kernel accepts a range of versions and refuses others with FAILED_PRECONDITION, naming the range and whether the SDK or the kernel needs an update. 0 (unset) is never accepted. Until the first release the version stays 1 and changes in place: SDKs and kernel must come from the same build.

Names: a domain is an identifier of letters, digits, - and _ that starts with a letter; by convention it is lowercase (orders), like aggregate types, while command, event and query types are capitalized (PlaceOrder); both cases are valid, and names are case-sensitive. Command, event, query and aggregate types are unique within their domain. Requests and events carry the domain and the type separately, unqualified (orders and PlaceOrder). Registrations, subscriptions, middlewares, memberships and grants use the qualified name domain.Type (orders.PlaceOrder), where orders.* stands for a whole domain and * for everything.

Times are milliseconds since the Unix epoch. A request with issued_at is valid 60 s around the kernel's clock; outside that window it fails with UNAUTHENTICATED.

Authorization: every call presents the caller's Grant in the gRPC metadata entry kernable-grant (see Grant). Rejections by a worker, a middleware or a read model are normal outcomes (Rejected, with a code); everything else (a bad signature, a missing permission, a timeout) is a gRPC status.

Shared ​

Signature ​

An Ed25519 signature and the key that made it.

FieldTypeNumberDescription
key_idstring1lowercase hex of the 32-byte public key
sigbytes264 bytes: Ed25519 over the message layout in SIGNING.md

Command ​

A request to change one aggregate. Signed by the client over kernable.v1.command; the kernel queues it and hands it to a worker that registered its aggregate and command type.

FieldTypeNumberDescription
command_idstring1Chosen by the client (the SDKs use a random UUID). The kernel runs a command ID once: submitting it again returns the stored result, or waits for it.
tenant_idstring2
domainstring3of the aggregate and the command type
aggregate_typestring4unqualified: Order
aggregate_idstring5chosen by the client
command_typestring6unqualified: PlaceOrder
payloadbytes7
client_sigSignature8Over kernable.v1.command, by a key granted submit for domain.command_type.
contextRequestContext9signed with the command, stored with its events
vault_refsrepeated string10References to the values the payload keeps in the vault (see VaultValue), in payload order.
deliver_atint6411Signed, and only when set: a scheduled command, delivered at this time (ms since epoch) and not before. Submit answers accepted at once; until then the command waits beside its aggregate's queue, so the aggregate's other commands pass. Middlewares check it when it is submitted and again when it is due. At most the kernel's --schedule-max-days ahead.
deliver_untilint6412Signed, and only when set: a scheduled command that cannot run by this time (ms since epoch), after an outage say, is rejected expired instead of running late.

RequestContext ​

Who sends a request and in which context. Signed by the client together with the request, and stored with every event the request causes, so audit and listeners can see it. The kernel only acts on expected_version and bypass_authorization; the rest is for the domain. Options for a single call (how long to wait) are not part of it; see SubmitRequest.

FieldTypeNumberDescription
sender_tenantstring1The sender: the real principal, also when acting on behalf of someone else.
sender_identitystring2
sender_accountstring3
sender_sessionstring4
on_behalf_ofstring5authorize as this identity of the target tenant instead
expected_versionoptional uint646Commands only: rejected with version_mismatch unless the aggregate is at this version when the command runs (0: the aggregate has no events yet). Absent: no check.
correlation_idstring7shared by everything one original request causes
causation_idstring8the command that caused this one, if any
trace_idstring9
bypass_authorizationbool10Privileged switch that tells the domain to skip its own authorization: only keys granted privileged for bypass_authorization may set it (otherwise PERMISSION_DENIED).
attributesrepeated Attribute11Signed in the given order. Business data only, never security decisions.

Attribute ​

FieldTypeNumberDescription
keystring1
valuestring2

Event ​

A stored event: a NewEvent of a worker, completed by the kernel when it committed it.

FieldTypeNumberDescription
global_positionuint641position in the whole log, from 1
tenant_idstring2
domainstring3always the domain of the command that caused it
aggregate_typestring4unqualified
aggregate_idstring5
versionuint646the aggregate's version after this event, from 1
event_typestring7unqualified: OrderPlaced
payloadbytes8
command_idstring9the command that caused it
created_atint6410ms since epoch, when the kernel committed it
hashbytes11SHA-256 link of the hash chain, see SIGNING.md
producer_sigSignature12The worker's signature over its result (kernable.v1.result); the same on every event of a command.
contextRequestContext13of the command that caused the event
schemauint3214version of the payload format of event_type, from 1
vault_refsrepeated string15see NewEvent
erasesstring16see NewEvent
origin_partitionstring17An event its tenant brought along when it moved here from another partition (S4): where it was first committed. Listeners do not get these; the original is delivered where it is.
origin_positionuint6418

Snapshot ​

An aggregate's state after version, made by a worker: a cache, the events stay the truth. The kernel drops it when it cannot be used (see Rejection) and after an erasure.

FieldTypeNumberDescription
versionuint641the aggregate version the state is after
schemastring2state format of the aggregate definition; workers reject unknown ones
statebytes3opaque to the kernel
worker_sigSignature4over kernable.v1.snapshot, see SIGNING.md

NewEvent ​

An event proposed by a worker; the kernel assigns position, version, time and hash.

FieldTypeNumberDescription
event_typestring1Unqualified, of the command's domain; an empty or qualified type rejects the command (invalid_event_type).
payloadbytes2
schemauint323Version of the payload format, from 1. Must be the one declared in Register.event_schemas (1 if not declared), otherwise invalid_event_schema.
vault_refsrepeated string4References to the values the payload keeps in the vault, in payload order.
erasesstring5A subject (a person) whose vault values this event erases, everywhere; empty for none. The worker's key needs vault_erase for the event type (otherwise erase_not_allowed).
origin_partitionstring6Set by the kernel only, on events a tenant brought from another partition when it moved there: where the event was first committed. A worker's result with them is refused.
origin_positionuint647

Vault ​

VaultValue ​

A personal or secret value that a command or event keeps outside the signed log. The payload holds {"$vault": ref} instead, and vault_refs lists the ref. The ref commits to the value: ref = hex(SHA-256(kernable.v1.vault message)), see SIGNING.md, over a random salt, the subject, secret and the value, so it reveals nothing once erased. The kernel keeps values encrypted and deletable, and only with a vault key (kernable start --vault-key); without one, a submit with vault values fails with FAILED_PRECONDITION and a result with vault values is rejected (no_vault). Personal values belong to a subject and are erased by an event that erases it; secret values are deleted when their command completes.

FieldTypeNumberDescription
refstring1lowercase hex, see above
subjectstring2personal: whose value it is; empty for secrets
secretbool3
saltbytes416 random bytes, only on writes
valuebytes5the JSON encoding of the field

Trust ​

Grant ​

What a key may do. The kernel trusts root keys only (kernable start --root); a root key signs a grant over kernable.v1.grant (kernable grant). The grantee's process presents it with every call, hex-encoded, in the gRPC metadata entry kernable-grant; it must name the key that signed the request (otherwise PERMISSION_DENIED). The kernel keeps the newest grant per key (by issued_at) and ignores older ones. A grant has no time window; it ends when a newer grant replaces it or the key, or the root key that signed it, is revoked. Streams whose key loses a permission they use end with PERMISSION_DENIED.

FieldTypeNumberDescription
key_idstring1the grantee
permissionsrepeated Permission2
succeedsstring3An older key this key replaces (empty: none). Listeners and aggregates registered with that key may connect with key_id and keep their registration.
issued_atint644ms since epoch
root_sigSignature5

Permission ​

One action and the types it covers. Types are *, domain.* or a qualified domain.Type, except where noted. The action * stands for all actions.

  • submit: submit commands of these types (CommandService.Submit).
  • run: run commands of these types as a worker (Register).
  • subscribe: read events of these types with EventService.Subscribe.
  • listen: receive events of these types as a listener (Hello.event_types).
  • query: ask queries of these types (QueryService.Query).
  • answer: answer queries of these types as a read model (Hello.query_types).
  • check: check commands and queries of these types as a middleware (Hello.middlewares).
  • privileged: set a privileged switch; the type is the switch, bypass_authorization.
  • vault_read: get the vault values of these event types in listener batches, and of these command types in middleware checks.
  • vault_erase: write events of these types that erase a subject (NewEvent.erases).
  • data_read, data_write: get, or put and delete, the files of a domain (domain.* or *).
  • cache: use the cache of a domain (domain.* or *).
  • manage: apply the membership of these services (service names, or *).
  • replicate: follow the kernel as its standby (*).
  • export: export the whole kernel (*, ExportService in export.proto).
FieldTypeNumberDescription
actionstring1
typesrepeated string2

TrustService ​

Revocation of keys, by root-key tooling (kernable revoke).

RPCRequestResponseDescription
RevokeRevokeRequestRevokeResponseRevokes key_id from since on, for good. Signed by a root key over kernable.v1.revoke and valid 60 s around issued_at. The kernel stores it signed with its own key (kernable.v1.revocation), refuses the key from then on and ends its streams; queued commands signed with it and received at or after since are rejected with key_revoked. A revoked root key takes the grants it signed with it. Revoking a key again does nothing. Errors: PERMISSION_DENIED (not a root key, or a revoked one), UNAUTHENTICATED (invalid signature, outside the time window), INVALID_ARGUMENT (key_id is not a key ID).

RevokeRequest ​

FieldTypeNumberDescription
key_idstring1the key to revoke
sinceint642ms since epoch; may lie in the past, e.g. when the key leaked
issued_atint643ms since epoch
root_sigSignature4
protocoluint325not signed

RevokeResponse ​

Memberships ​

ProcessService ​

Business processes, run by the kernel (design/processes.md).

RPCRequestResponseDescription
RegisterProcessRegistrationProcessRegisteredRegisters a process model (JSON, see design/processes.md "The model format"). Signed over kernable.v1.process by a key granted process for the process's qualified name, and submit for each command, query for each query and listen for each event the model names; valid 60 s around issued_at. A model equal to the newest version registers nothing new; a different one becomes the next version. Running cases stay on their version. Errors: UNAUTHENTICATED, PERMISSION_DENIED, INVALID_ARGUMENT (the model is invalid: the message names the step and why; a command type no worker registered and no membership lists).
RemoveProcessRemovalProcessRemovedRemoves a process: it starts no new cases, its running cases finish (as when a membership no longer lists it). Signed over kernable.v1.process_remove by a key granted process for it; valid 60 s around issued_at. A later registration makes it active again. Errors: UNAUTHENTICATED, PERMISSION_DENIED, NOT_FOUND (no such process).
ResolveProcessResolutionProcessResolvedResolves a case's incident (design/processes.md, "Incidents"): retry runs its step again (a new attempt), skip goes on to the step's next (refused for a step without a single next), cancel ends the case with the end cancelled (also a case without an incident). Signed over kernable.v1.process_resolve by a key granted process for the process; valid 60 s around issued_at. Errors: UNAUTHENTICATED, PERMISSION_DENIED, NOT_FOUND (no such running case), FAILED_PRECONDITION (no incident to retry or skip; a step without a single next; the case's ID names cases of several tenants: give the tenant).
ActProcessActingProcessActedSets as whom the process acts (design/processes.md, "With the kit"): the sender tenant, identity and session of context go with every command and query the engine sends for the process from now on (its correlation and causation stay the engine's). Kept with the process's state, replicated to the standby and to every partition. Without it a process acts as process:<name>. Idempotent. Signed over kernable.v1.process_act by a key granted process for it; valid 60 s around issued_at. Errors: UNAUTHENTICATED, PERMISSION_DENIED, NOT_FOUND (no such process), FAILED_PRECONDITION (a partition that takes its processes from the control partition).

ProcessActing ​

FieldTypeNumberDescription
namestring1qualified: shop.fulfilment
contextRequestContext2
issued_atint643ms since epoch
sigSignature4over kernable.v1.process_act
protocoluint325not signed

ProcessActed ​

FieldTypeNumberDescription
changedbool1false: it acted so already

ProcessResolution ​

FieldTypeNumberDescription
processstring1qualified: shop.fulfilment
case_idstring2the case's aggregate ID: its key and sequence, o1#1
tenant_idstring3the case's tenant; empty: the one tenant that has the case
actionstring4retry | skip | cancel
issued_atint645ms since epoch
sigSignature6over kernable.v1.process_resolve
protocoluint327not signed

ProcessResolved ​

FieldTypeNumberDescription
tenant_idstring1the case's tenant

ProcessRemoval ​

FieldTypeNumberDescription
namestring1qualified: shop.fulfilment
issued_atint642ms since epoch
sigSignature3over kernable.v1.process_remove
protocoluint324not signed

ProcessRemoved ​

FieldTypeNumberDescription
removedbool1false: it was removed already

ProcessRegistration ​

FieldTypeNumberDescription
modelbytes1the model as JSON
issued_atint642ms since epoch
sigSignature3over kernable.v1.process
protocoluint324not signed

ProcessRegistered ​

FieldTypeNumberDescription
namestring1qualified: shop.fulfilment
versionuint322from 1
newbool3false: equal to the newest version, nothing registered

MembershipService ​

What a service consists of, applied by deploy tooling.

RPCRequestResponseDescription
ApplyApplyRequestApplyResultReplaces the membership of membership.service. Signed over kernable.v1.membership by a key granted manage for the service, and valid 60 s around issued_at. Apply it once per deploy (kernable apply, or a deploy script), not from every process: the newest issued_at wins, and an older membership is refused. Errors: UNAUTHENTICATED (invalid signature, outside the time window), INVALID_ARGUMENT (no valid service name, unqualified worker entries, empty listener names), PERMISSION_DENIED, FAILED_PRECONDITION (a membership issued at the same time or later is applied).

Membership ​

What a service consists of. Once a service has one, the kernel waits (readiness) for what it lists, and treats workers and listeners of the service that it does not list as dynamic. Workers and listeners name their service in Register.service and Hello.service. A service without a membership, and a process without a service, count as listed.

  • workers: the aggregate types the service runs (qualified: shop.Cart). Commands of a listed aggregate are accepted, and queued, even before a worker registered it. An aggregate a worker runs but the service does not list is forgotten once its last worker leaves.
  • listeners: listener names. A listed listener counts for readiness, settling and the order of delivery; an unlisted one keeps its position but counts for nobody.
  • middlewares: names of the service's listeners that check requests; they count like listeners. A listener of the service that is not listed here cannot connect as a middleware, and taking one off the list removes its check: a check never appears or disappears without a signed membership.
FieldTypeNumberDescription
servicestring1a name like a domain: letters, digits, - and _
workersrepeated string2qualified aggregate types
listenersrepeated string3
middlewaresrepeated string4
metadatarepeated Attribute5free, e.g. version and description; signed in order
issued_atint646ms since epoch
processesrepeated string7Qualified process names (shop.fulfilment); signed only when there are any. A process a membership no longer lists starts no new cases; its running cases finish.

ApplyRequest ​

FieldTypeNumberDescription
membershipMembership1
sigSignature2over kernable.v1.membership
protocoluint323not signed

ApplyResult ​

Partitions ​

PartitionService ​

Where each tenant's data lives when several kernels (partitions) share the tenants. Every kernel started with --partition serves the tenants the map assigns to it and refuses the others with FAILED_PRECONDITION wrong_partition: …; the SDKs read the map and send each request to its tenant's partition.

RPCRequestResponseDescription
MapMapRequestPartitionMapThe newest map this kernel holds. NOT_FOUND without one (a kernel without --partition serves every tenant); with one, the call needs a grant of any key (kernable-grant): the map names tenants and addresses.
ApplyPartitionMapMapResultApplies a new map on the control partition (the kernel started with --partition but without --control); the other partitions take it from there within seconds. Signed over kernable.v1.partitions by a key granted manage for *, valid 60 s around issued_at. Errors: UNAUTHENTICATED, PERMISSION_DENIED, INVALID_ARGUMENT (unknown partitions, a tenant twice), FAILED_PRECONDITION (not the control partition; an epoch not above the one held).
MovingMoveStatusRequestMoveStatusAsked of the target of a move (needs a grant of any key): whether it copied the tenant.

MapRequest ​

FieldTypeNumberDescription
protocoluint321

MapResult ​

Partition ​

A partition: its name and the addresses of its kernels (two for a pair with a standby).

FieldTypeNumberDescription
namestring1
addressesrepeated string2

TenantPartition ​

FieldTypeNumberDescription
tenantstring1
partitionstring2

TenantMove ​

A tenant moving between partitions, or moved (S4). While it moves, both partitions answer its requests with UNAVAILABLE. Once moved, source_head is the source's head when it was copied (every original event of the tenant is at or before it) and target_after the target's head after the copies: an SDK delivers the target's events after it only once the source's stream has passed source_head, so a tenant's events keep their order across the move.

FieldTypeNumberDescription
tenantstring1
fromstring2
tostring3
source_headuint644
target_afteruint645
epochuint646of the map that started the move: tells this move from earlier ones

PartitionMap ​

Tenants listed in tenants live on their partition; every other tenant on default.

FieldTypeNumberDescription
epochuint641the newest epoch wins; each new map has a higher one
partitionsrepeated Partition2
tenantsrepeated TenantPartition3
defaultstring4a partition's name
controlstring5the partition that holds the map and the trust records
issued_atint646ms since epoch
sigSignature7over kernable.v1.partitions
protocoluint328not signed
movingrepeated TenantMove9being copied to to; the tenant waits meanwhile
movedrepeated TenantMove10done, with their heads
copyingrepeated TenantMove11Copied to to while the tenant runs on from (its finished commands, up to the source's head as of then), so that the move after it stops the tenant only for the rest.

MoveStatus ​

What a target partition knows of a tenant moving to it.

FieldTypeNumberDescription
donebool1
source_headuint642
target_afteruint643
epochuint644of the move it copied (TenantMove.epoch)
precopybool5copied while the tenant ran (copying): the move's first part only

MoveStatusRequest ​

FieldTypeNumberDescription
tenantstring1
protocoluint322

Clients ​

CommandService ​

Commands, sent by clients and by handlers of listeners.

RPCRequestResponseDescription
SubmitSubmitRequestCommandResultSubmits a command and waits as wait says. Needs submit for the qualified command type, and a worker that registered it (or a membership that lists its aggregate), otherwise NOT_FOUND. Middlewares before the command may reject it; the outcome is then rejected. A command ID that was submitted before is not run again: Submit returns its stored result, or waits for it. If the command does not complete within the kernel's submit timeout, Submit fails with DEADLINE_EXCEEDED, but the command stays queued: resubmit it to get the result. Other errors: UNAUTHENTICATED (missing or invalid signature), PERMISSION_DENIED, INVALID_ARGUMENT, FAILED_PRECONDITION (see listener; vault values without a vault), UNAVAILABLE (a middleware is not connected, or the kernel shuts down), DEADLINE_EXCEEDED (a middleware did not answer in time).
CancelCancelRequestCancelResultCancels a scheduled command (Command.deliver_at) before it is due: the command completes as rejected cancelled, so its ID answers that from then on. Needs submit for the command's type; middlewares check it as a cancel of that type (MiddlewareCall.cancel), and may reject it. A command that is due already, running or completed answers too_late; an unknown ID not_found.

CancelRequest ​

FieldTypeNumberDescription
command_idstring1
tenant_idstring2
contextRequestContext3
issued_atint644ms since epoch; must be within 60 s of the kernel's clock
client_sigSignature5over kernable.v1.cancel
protocoluint326not signed

CancelResult ​

FieldTypeNumberDescription
command_idstring1
cancelledCancelled2
rejectedRejected3too_late, not_found, or by a middleware

Cancelled ​

SubmitRequest ​

FieldTypeNumberDescription
commandCommand1
waitWait2
protocoluint323not signed
vaultrepeated VaultValue4the values of command.vault_refs
listenerstring5Set by SDKs when a listener's handler submits (not signed): the listener's name. A listener handles its events one after another, so it cannot wait for itself: a settling submit then waits only for listeners of a lower order, and a command the listener itself checks as a middleware before is refused (FAILED_PRECONDITION).

Wait ​

How long Submit waits before it responds.

ValueNumberDescription
WAIT_SETTLED0Until every listener that counts has acknowledged the command's last event (default). If that takes longer than the kernel's settle timeout, Submit returns the committed events with settled false.
WAIT_COMMITTED1Until the command's events are stored, or it is rejected.
WAIT_ACCEPTED2Until the command is stored in the queue; resubmit it later to get the result.

CommandResult ​

The outcome of a command; exactly one of committed, rejected or accepted is set.

FieldTypeNumberDescription
command_idstring1
committedCommitted2the worker's events, stored
rejectedRejected3by the worker, a middleware or the kernel; nothing was stored
acceptedAccepted4queued, not run yet (WAIT_ACCEPTED only)

Committed ​

FieldTypeNumberDescription
eventsrepeated Event1may be empty: a command can produce no events
settledbool2True once every listener that counts has acknowledged the events (WAIT_SETTLED only).

Accepted ​

Rejected ​

Why a command or query was refused. Codes set by the kernel are listed in the errors reference; workers, middlewares and read models choose their own.

FieldTypeNumberDescription
codestring1machine-readable, e.g. version_mismatch
messagestring2for people
keystring3Set by the kernel only, never by a worker: for unique_taken, the name of the key another aggregate holds (never its value); for requirement_missing, the first requirement of the result that does not hold. Stored with the rejection.
requirementRequirement4

EventService ​

Plain event streams, for clients that read the log without registering as a listener.

RPCRequestResponseDescription
SubscribeSubscribeRequeststream EventStreams stored events with a global position after after_position: first the backlog, then live, until the client cancels. Not registered, so it counts for nobody and keeps no position. Needs subscribe for the requested types (PERMISSION_DENIED otherwise, also later, when the key loses it). Errors: UNAUTHENTICATED (invalid signature, outside the time window), UNAVAILABLE (the kernel shuts down).

SubscribeRequest ​

Signed by a key granted subscribe for the requested types, over kernable.v1.subscribe.

FieldTypeNumberDescription
after_positionuint6410: from the first event
event_typesrepeated string2qualified (orders.OrderPlaced, orders.*); empty = all the key may read
issued_atint643ms since epoch; must be within 60 s of the kernel's clock
client_sigSignature4
protocoluint325not signed

Queries ​

QueryService ​

Every query goes through the kernel, also between parts of one process, so that signatures, grants and the request context always apply. Read models answer them (see Hello).

RPCRequestResponseDescription
QueryQueryRequestQueryResultAsks a query and returns the answer of one read model that answers its type and has processed min_position (several are taken in turn). Needs query for the qualified query type. Middlewares before the query may reject it, middlewares after it may reject the response; the outcome is then rejected. Errors: UNAUTHENTICATED (invalid signature, outside the time window), PERMISSION_DENIED, INVALID_ARGUMENT, UNAVAILABLE (no connected read model answers the type, or a middleware is not connected), DEADLINE_EXCEEDED (no read model answered, or reached min_position, within the kernel's query timeout, or a middleware did not answer in time), FAILED_PRECONDITION (see listener).

QueryRequest ​

Signed by the client over kernable.v1.query.

FieldTypeNumberDescription
query_idstring1Chosen by the client (the SDKs use a random UUID); returned in QueryResult. Queries are not stored, so an ID is not checked for reuse.
tenant_idstring2
domainstring3
query_typestring4unqualified: GetOrder
payloadbytes5
contextRequestContext6
min_positionuint647Answer only after the read model has processed this global position (0: right away), e.g. the position of the last event of a command the client just submitted.
issued_atint648ms since epoch; must be within 60 s of the kernel's clock
client_sigSignature9
protocoluint3210not signed
listenerstring11Set by SDKs when a listener's handler asks (not signed); see SubmitRequest.listener. The query is never routed to that listener, nor to one that could reach min_position only after that listener, and refused (FAILED_PRECONDITION) if no other read model answers it or if the listener itself checks the query as a middleware.
wait_onlybool12Signed, and only when set: answer nothing; return an empty response once a read model that answers query_type has processed this kernel's head (as of the request) and min_position. Not checked by middleware, and any partition takes it, whatever the tenant: the SDKs send it to every partition before a query across tenants (allPartitions), so that its answer holds every event committed before it.
targetTarget13Signed, and only when set: the aggregate the query is about, as the query type declares it (the SDKs fill it from the query's data). A middleware decides on it as on a command's aggregate, e.g. "only the order's owner may read it"; the read model answers from the data.

Target ​

The aggregate a query is about; see QueryRequest.target.

FieldTypeNumberDescription
aggregate_typestring1qualified: shop.order
aggregate_idstring2

QueryResult ​

FieldTypeNumberDescription
query_idstring1
responsebytes2the read model's answer, opaque to the kernel
rejectedRejected3by the read model or a middleware

Data and cache ​

DataService ​

Files of a domain, kept by the kernel under (tenant, domain, key), apart from the event log: they are neither signed into it nor erased with the vault. Every request is signed by a key granted data_read (get) or data_write (put, delete) for the domain, over kernable.v1.data.<op>, and valid 60 s around issued_at. A put replaces the file under its key. A put or delete runs at most once: the same signed request again is a replay, also after it failed (it may have taken effect); send it signed anew. Errors: UNAUTHENTICATED (invalid signature, outside the time window), PERMISSION_DENIED, INVALID_ARGUMENT (domain, key or size), ALREADY_EXISTS (a replay), FAILED_PRECONDITION (a put or delete signed before the kernel started: sign it again).

RPCRequestResponseDescription
PutDataDataRequestDataResult
GetDataDataRequestDataResult
DeleteDataDataRequestDataResult
ListDataDataRequestDataListThe files whose keys start with key (a prefix, may be empty), up to 1,000 per page, in the order of their keys' bytes, after after; granted data_read, checked as <domain>.ListData. To find files no event names, or what a tenant stores.

DataRequest ​

FieldTypeNumberDescription
tenant_idstring1
domainstring2
keystring3any string of 1 to 512 bytes
databytes4put only; at most --max-data-bytes (16 MiB by default)
content_typestring5put only, e.g. application/pdf
issued_atint646ms since epoch
client_sigSignature7
protocoluint328not signed
contextRequestContext9Who asks and in which context, as for a command; signed. Middlewares checking <domain>.PutData, .GetData or .DeleteData decide on it.
afterstring10list only: the next of the page before; signed

DataResult ​

FieldTypeNumberDescription
foundbool1get and delete: whether the file existed; put: true
databytes2get only
content_typestring3get only
sha256bytes4put and get: the SHA-256 of the file's data, for an event to name

DataList ​

FieldTypeNumberDescription
filesrepeated DataEntry1
nextstring2the last key of this page if there may be more: send it as after

DataEntry ​

FieldTypeNumberDescription
keystring1
content_typestring2
sizeuint643bytes

CacheService ​

A shared cache of a domain: values under (tenant, domain, key) that may expire. Not durable; a kernel restart may empty it. Signed like DataService requests over kernable.v1.cache.<op>, by a key granted cache for the domain, with the same errors.

RPCRequestResponseDescription
SetCacheCacheRequestCacheResult
GetCacheCacheRequestCacheResult
DeleteCacheCacheRequestCacheResult

CacheRequest ​

FieldTypeNumberDescription
tenant_idstring1
domainstring2
keystring3any string of 1 to 512 bytes
valuebytes4set only; at most 1 MiB
ttl_msuint645set only; 0 = until deleted or the kernel restarts
issued_atint646ms since epoch
client_sigSignature7
protocoluint328not signed
contextRequestContext9Who asks and in which context, as for a command; signed. Middlewares checking <domain>.SetCache, .GetCache or .DeleteCache decide on it.

CacheResult ​

FieldTypeNumberDescription
foundbool1get and delete: whether the key had a live value; set: true
valuebytes2get only

Listeners ​

ListenerService ​

Durable, named, ordered event listeners (read models and reactors).

Event N reaches a listener of order k only after every listener that counts with a lower order has acknowledged N; equal orders run in parallel. Delivery is at least once.

A listener can also be a middleware: before the kernel accepts a command or routes a query whose type the middleware covers, it asks every such middleware, in chain order, and refuses the request at the first rejection. A registered middleware that is not connected refuses every request it covers (UNAVAILABLE): the chain fails closed.

RPCRequestResponseDescription
Listenstream ListenerMessagestream ListenerDeliveryA listener's stream. The listener sends Hello, the kernel a Challenge, the listener a ChallengeResponse, the kernel Ready; then the kernel sends batches, queries and checks, and the listener acks, answers and verdicts. Needs listen for the event types, answer for the query types and check for the middleware types. Errors: INVALID_ARGUMENT (malformed Hello, or an ack beyond the delivered events), PERMISSION_DENIED (missing permission, a name registered with another key, a middleware its membership does not list), ALREADY_EXISTS (a listener of that name is connected, or a live one uses a registered name), UNAUTHENTICATED (challenge failed), RESOURCE_EXHAUSTED (an ack would park more than 10,000 events).
RetryRetryRequestRetryResultDelivers parked events of a registered listener again (see Ack.parked), or dismisses them. Signed over kernable.v1.retry by a key granted manage for the listener's service (* for a listener without one), and valid 60 s around issued_at. Errors: UNAUTHENTICATED (invalid signature, outside the time window), PERMISSION_DENIED, NOT_FOUND (no such listener).

RetryRequest ​

FieldTypeNumberDescription
listenerstring1
positionsrepeated uint642the parked events; empty: all of them
dismissbool3remove them instead of delivering them again
issued_atint644ms since epoch
sigSignature5over kernable.v1.retry
protocoluint326not signed

RetryResult ​

FieldTypeNumberDescription
positionsrepeated uint641the parked events it applied to; requested ones not parked are left out

ListenerMessage ​

From the listener; exactly one field is set.

FieldTypeNumberDescription
helloHello1first message
challenge_responseChallengeResponse2second message
ackAck3
answerQueryAnswer4to a QueryCall
verdictVerdict5to a MiddlewareCall

ListenerDelivery ​

From the kernel; exactly one field is set.

FieldTypeNumberDescription
challengeChallenge1after Hello
readyReady2after a valid ChallengeResponse
batchBatch3
queryQueryCall4Delivered in order with the batches: the answer reflects every batch sent before it.
checkMiddlewareCall5Also in order with the batches; sent only once the listener has caught up with the events committed before it connected.
waitingWaiting6instead of Ready, while another instance of a single group is active

QueryCall ​

A query for a read model to answer with a QueryAnswer of the same call_id.

FieldTypeNumberDescription
call_idstring1
queryQueryRequest2as the client sent it, with its signature

QueryAnswer ​

FieldTypeNumberDescription
call_idstring1
responsebytes2
rejectedRejected3

MiddlewareCall ​

A request for a middleware to check: pass it with an empty verdict or reject it. Answer with a Verdict of the same call_id, also after (where it may only count for a query's response).

FieldTypeNumberDescription
call_idstring1
commandCommand2as the client sent it, with its signature
queryQueryRequest3as the client sent it, with its signature
dataDataRequest4A file or cache request, before it runs; without the file or value itself.
cacheCacheRequest5
cancelCancelRequest11A cancel of a scheduled command, as the client sent it; scheduled is that command.
vaultrepeated VaultValue6Before a command only: its vault values, if the key may read them (granted vault_read).
whenWhen7
command_resultCommandResult8After only: the command's result. The middleware only observes it; its verdict is ignored.
query_resultQueryResult9After only: the query's result. A verdict that rejects a response replaces it.
typestring10The qualified type being checked: shop.AddItem, shop.GetCart, or for files and the cache shop.PutData, shop.GetData, shop.DeleteData, shop.SetCache, shop.GetCache, shop.DeleteCache; for a cancel the scheduled command's type.
scheduledCommand12A cancel only: the scheduled command it cancels, as stored.

Verdict ​

FieldTypeNumberDescription
call_idstring1
rejectedRejected2absent: the request passes

When ​

When a middleware runs.

ValueNumberDescription
WHEN_BEFORE0Before the kernel accepts a command or routes a query; a rejection refuses it. Fails closed: while the middleware is away, the requests it covers are refused.
WHEN_AFTER1Once the outcome is known. A query's response is sent only if the middleware passes it (fails closed, like before). A command's result, and a rejected query, it only observes, best effort: it is told once the command completes, nobody waits for it, and it misses what happens while it is away.

Middleware ​

A listener's registration as a middleware, see Hello.middlewares.

FieldTypeNumberDescription
orderint321Position in the chain: lower runs first, equal orders by listener name. Negative orders are reserved for kernable's kits: they need a grant of reserved_order.
typesrepeated string2Command and query types it checks: *, a domain (orders.*) or a qualified type. Not empty; needs check for each.
whenWhen3

Hello ​

The first message of a listener's stream. A registered listener (not live) is stored under its name with its key, order, event types and middlewares, and keeps its position across connections; it counts (see Membership) also while disconnected.

FieldTypeNumberDescription
namestring1unique; one connection per name at a time
key_idstring2the key that answers the challenge; registered with the name
orderint323Order of delivery, from 0 (negative orders are reserved for kernable's kits and need a grant of reserved_order): event N reaches this listener only after every listener that counts with a lower order acknowledged N.
event_typesrepeated string4qualified (orders.OrderPlaced, orders.*); empty = all the key may read
from_positionoptional uint645Start after this position instead of the stored one, e.g. 0 to rebuild an in-memory read model. Absent: after the stored position, or at the newest event on the first connect, so a new listener never reacts to what happened before it existed.
query_typesrepeated string6qualified query types this listener answers
middlewaresrepeated Middleware7At most one per when; none: not a middleware. A listener of a service with a membership must be listed in its middlewares.
protocoluint328
servicestring9The service this listener belongs to; its membership (see Membership) decides whether it counts and whether it may be a middleware. Empty: no service, it always counts.
livebool10Not registered: no stored position, starts at the newest event, does not count for readiness, settling or the order of delivery, and is forgotten when it disconnects. For reactions that only matter while the process runs. Cannot be a middleware, nor use the name of a registered listener.
instancestring11One of several processes that run this listener under one name, e.g. a read model on each node of a pair: 1 to 128 letters, digits, -, _ and ., unique in the group. Empty: the only one, and a second connection of the name is refused. The group counts for readiness, settling and the order of delivery as one listener: through an event once one instance acknowledged it, ready once one instance has caught up. Every instance of a group must agree on single.
singlebool12With instance: one instance at a time gets events, so a reactor's side effects happen once; the others get Waiting, and the first of them takes over after the group's position when the active one leaves. Without: every instance gets every event, from its own position, and queries and checks go to any instance that has reached what they need; such an instance cannot park events. Not for live listeners.

Waiting ​

Another instance of the same single group is active; this one gets Ready and its deliveries when it takes over.

FieldTypeNumberDescription
activestring1the active instance

Ready ​

The kernel accepted the listener; deliveries follow.

FieldTypeNumberDescription
after_positionuint641the listener receives events after this position
head_positionuint642newest event when the listener connected: up to it, it replays

Batch ​

Events for the listener, in order of global position.

FieldTypeNumberDescription
eventsrepeated Event1only the listener's event types; may be empty
up_touint642last position the batch covers; acknowledge it once processed
vaultrepeated VaultValue3The values the events refer to, if the key may read them (granted vault_read) and they are not erased; a missing value reads as null.
retriesrepeated Event4Parked events an operator asked to deliver again (RetryRequest), to handle before events and regardless of the listener's position. Sent alone, with up_to the last position delivered. Their vault values are in vault.

Ack ​

The listener has processed everything up to position (a Batch.up_to). Not signed; affects only waiting and ordering. An ack beyond the delivered batches ends the stream.

FieldTypeNumberDescription
positionuint641
parkedrepeated Parked2Events of the acknowledged batches (or their retries) the listener gave up on: it moves on, and the kernel keeps them, with the reason, until an operator retries or dismisses them (ListenerService.Retry). Each is above the previously acknowledged position or one of the retries, and at most position. Not for live listeners.

Parked ​

An event a listener gave up on.

FieldTypeNumberDescription
positionuint641
codestring2Why: a Rejected code (not_found), or handler_error for any other error. At most 128 bytes.
messagestring3The rejection's message; for another error only its type (it can hold personal data, and the kernel keeps it): the listener logs the whole error. At most 1,024 bytes.
attemptsuint324how often it was handled

Workers ​

WorkerGateway ​

Workers run the commands of their aggregates.

RPCRequestResponseDescription
Sessionstream WorkerMessagestream KernelMessageA worker's stream. The worker sends Register, the kernel a Challenge, the worker a ChallengeResponse, the kernel Registered; then the kernel sends Execute, at most max_concurrency at a time, and the worker answers each with a Result or a Rejection. Needs run for every registered command type. When the stream ends, the kernel hands its unfinished commands to another worker. Errors: INVALID_ARGUMENT (malformed Register), PERMISSION_DENIED (missing permission, or an aggregate registered with another key), FAILED_PRECONDITION (the store holds a newer event schema than the worker writes), UNAUTHENTICATED (challenge failed, or an invalid result, rejection or snapshot signature).

WorkerMessage ​

From the worker; exactly one field is set.

FieldTypeNumberDescription
registerRegister1first message
challenge_responseChallengeResponse2second message
resultResult3
rejectionRejection4

KernelMessage ​

From the kernel; exactly one field is set.

FieldTypeNumberDescription
challengeChallenge1after Register
registeredRegistered2after a valid ChallengeResponse
executeExecute3
execute_moreExecuteMore4the rest of an Execute's history

Register ​

The first message of a worker's stream.

FieldTypeNumberDescription
worker_idstring1this process, for logs and the UI
key_idstring2the key that answers the challenge and signs results
aggregatesrepeated WorkerAggregate3The aggregates the worker runs; at least one. The kernel registers each one: it refuses commands for an aggregate type or command type no worker registered, and is not ready while a registered aggregate has no worker connected. Replicas register the same aggregates, with the same key (or its successor, see Grant.succeeds).
max_concurrencyuint324commands in flight at once; 0 counts as 1
event_schemasrepeated EventSchema5The payload version the worker writes per event type; types not listed are written at 1. The kernel refuses a worker, or a result, that would write an older version of a type than the store already holds, and a result whose events differ from what was declared.
protocoluint326
servicestring7The service this worker belongs to; its membership (see Membership) decides whether the kernel waits for the worker's aggregates. Empty: no service, they are always waited for.

WorkerAggregate ​

FieldTypeNumberDescription
aggregate_typestring1qualified: orders.Order; once per Register
command_typesrepeated string2qualified, of the same domain: orders.PlaceOrder; not empty

EventSchema ​

FieldTypeNumberDescription
event_typestring1qualified, of a domain of the worker's aggregates: orders.OrderPlaced
schemauint322from 1

Challenge ​

Proof of key possession, for workers and listeners.

FieldTypeNumberDescription
noncebytes132 random bytes

ChallengeResponse ​

FieldTypeNumberDescription
sigbytes1Ed25519 over kernable.v1.challenge with the nonce, by Register/Hello.key_id

Registered ​

FieldTypeNumberDescription
session_idstring1currently the worker_id of the Register message

Execute ​

A command for the worker to run. Answer with a Result or a Rejection of the same lease_id before deadline; after it, the command goes to another worker and a late answer is ignored.

FieldTypeNumberDescription
lease_idstring1
commandCommand2as the client sent it, with its signature
historyrepeated Event3events after snapshot.version, or all events without one
expected_versionuint644The aggregate's current version (0: no events yet); the result's events follow it.
snapshotSnapshot5the latest snapshot, if any
received_atint646ms since epoch when the kernel accepted the command; stable across retries
deadlineint647ms since epoch when the lease expires and the command goes to another worker
vaultrepeated VaultValue8the values the command and the history refer to, unless erased
morebool9The history does not fit one message; ExecuteMore messages of the same lease_id follow, in order, with the rest of it (other leases' messages may come between). Run the command once the last one (more false) has arrived.

ExecuteMore ​

The next part of an Execute's history, with the vault values it refers to.

FieldTypeNumberDescription
lease_idstring1
historyrepeated Event2
vaultrepeated VaultValue3
morebool4another ExecuteMore of this lease follows

Result ​

The events a command produced (possibly none). Signed by the worker over kernable.v1.result, which covers the command and expected_version but not the lease. If another command changed the aggregate meanwhile, the kernel runs the command again (Execute with the new history), a few times before it rejects it with concurrency_conflict.

FieldTypeNumberDescription
lease_idstring1
eventsrepeated NewEvent2
worker_sigSignature3by Register.key_id
snapshotSnapshot4Optional: the state after these events, at expected_version + len(events), signed over kernable.v1.snapshot by the same key. Only with at least one event; an invalid snapshot ends the stream like an invalid result signature. Not stored after an erasure.
vaultrepeated VaultValue5the values of the events' vault_refs
keysrepeated UniqueKey6The full set of unique keys and facts the aggregate holds after these events (empty: none), and the facts other aggregates must hold for the command to commit. The kernel checks them atomically with the commit: a key another aggregate holds rejects the command with unique_taken, a missing fact with requirement_missing. At most 64 keys, 64 facts and 64 requirements; key and fact names are valid type names (no dots) of at most 128 bytes; no key twice (name, scope and value hash), no fact twice. Otherwise invalid_index. A command that leaves its aggregate without events holds nothing. An aggregate whose key another one holds (a duplicate from before the key was declared) is rejected with unique_taken until its key changes or becomes null.
factsrepeated string7
requiresrepeated Requirement8
index_sigSignature9Over kernable.v1.index by Register.key_id; required when keys, facts or requires are set. Apart from worker_sig, which every event stores and kernable verify checks against the events alone.

UniqueKey ​

A unique key of an aggregate: no other aggregate holds the same key at the same time.

FieldTypeNumberDescription
namestring1declared by the aggregate; a valid type name (no dots), at most 128 bytes
globalbool2unique across tenants; otherwise within the command's tenant
value_hashbytes3SHA-256 of the value's UTF-8 bytes (the domain normalises the value)
subjectstring4Personal: the person the value belongs to. The kernel keeps the key in the vault, and erasing the subject frees it. Empty: not personal.

Requirement ​

A fact another aggregate of the command's tenant must hold (or, absent, must not hold) when the command commits. The aggregate is named by its ID, or by a unique key of its type that it holds: the key's name and the SHA-256 of its value, as in UniqueKey, in the command's tenant (a global key held in another tenant names none). No aggregate holding the key, like an aggregate without events, holds no fact. Personal keys cannot name an aggregate.

FieldTypeNumberDescription
domainstring1
aggregate_typestring2unqualified
aggregate_idstring3empty when named by a key
factstring4
keystring5the key's name, instead of aggregate_id
value_hashbytes6with key; never returned in a rejection
absentbool7the aggregate must not hold the fact

Rejection ​

The worker refuses the command: it is stored as rejected and nothing is written. Signed by the worker over kernable.v1.rejection. A rejection with code snapshot_stale means the worker cannot read the snapshot it got (unknown schema): the kernel drops the snapshot and executes the command again with all events.

FieldTypeNumberDescription
lease_idstring1
reasonRejected2
worker_sigSignature3by Register.key_id