diff --git a/docs/enhancements/per-second-usage-billing/README.md b/docs/enhancements/per-second-usage-billing/README.md new file mode 100644 index 00000000..4eafb7ea --- /dev/null +++ b/docs/enhancements/per-second-usage-billing/README.md @@ -0,0 +1,337 @@ +--- +status: draft +stage: alpha +latest-milestone: "v0.x" +--- + +# Per-Second Usage Billing for Compute + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [Where Compute Fits](#where-compute-fits) + - [User Stories](#user-stories) + - [How It Works](#how-it-works) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [The Instance Controller Is the Metering Point](#the-instance-controller-is-the-metering-point) + - [What It Tracks per Instance](#what-it-tracks-per-instance) + - [How the Running Time Becomes Usage](#how-the-running-time-becomes-usage) + - [Downtime and Crash Recovery](#downtime-and-crash-recovery) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) + - [Feature Enablement and Rollback](#feature-enablement-and-rollback) + - [Monitoring Requirements](#monitoring-requirements) + - [Dependencies](#dependencies) + - [Scalability](#scalability) + - [Troubleshooting](#troubleshooting) +- [Implementation History](#implementation-history) +- [Infrastructure Needed](#infrastructure-needed) + +## Summary + +Today, projects are not billed for the compute they use. This enhancement makes +Compute **meter and hand off exact, per-second usage** — how many vCPUs and how +much memory each customer's workloads actually consumed — to the platform +billing pipeline. Customers get billed for what they truly run, **only while +they are running**, and nothing else. + +Compute's job here is narrow and precise: **know how long an instance actually +ran** and report that accurately. It does not set prices, decide who owes what, +or generate invoices — those already live in the platform billing stack. Compute +measures; the billing platform prices and invoices. + +## Motivation + +Compute currently gives every customer a fixed pool of resources and enforces +limits against it (see [quota enforcement](../quota-enforcement/README.md)), but +nothing records how much of that pool a customer actually used. That means: + +- **Customers who run little pay the same as those who run a lot.** Consumption + is invisible, so usage can't factor into fair, consumption-based billing. +- **There is no accurate, auditable number** beyond a reservation — no count of + hours used, no basis for usage-based pricing, no way for a customer to see + their real footprint. + +Billing against actual, per-second consumption fixes both. It gives the platform +an accurate, trusted measure of what each project used, for the exact time it +used it — and it gives customers a bill that reflects reality, not a ceiling. + +### Goals + +- Compute records, for each project, the **actual running time** of its compute + workloads, counted per second. +- That running time is broken out by resource — **vCPUs** and **memory** — so the + billing platform can price them (possibly differently) later. +- **Customers are billed only for time they actually ran** — nothing accrues + while a workload is stopped, and suspension stops accrual entirely. +- The usage Compute reports is **exact and trustworthy**: no double-counting on + retries, no gaps on faults, and it survives operator and controller restarts. +- The meter definitions Compute emits match what the billing platform expects, + so usage lands cleanly instead of being rejected. +- Compute's contribution is **hand-off only**. It measures and reports; pricing + and invoicing stay in the billing platform. + +### Non-Goals + +- **Pricing, rate cards, or invoices.** Compute does not decide what a vCPU-hour + costs or bill anyone. It produces usage; the platform turns that into money. +- **Metering things Compute doesn't run.** Billing is for **running** instances + only — while one is available and running. Reserved-but-idle capacity is not + billed through this mechanism. +- **Per-second bookkeeping on the wire.** Billing is per-second in *denomination + and accuracy*, but events are batched, so usage is not a billion little events. + This keeps the pipeline efficient while billing stays exact. +- **Replacing capacity reservation / quota.** Quota enforcement keeps limiting + what a project *may* run; this enhancement measures what it *did* run. Both + coexist. +- **Attributing usage to a billing account.** Compute reports which **project** + used the resources; mapping project → account (and organizing it by the right + customer) is the billing platform's job. + +## Proposal + +### Where Compute Fits + +Compute runs customer workloads and knows, moment to moment, what is actually +running. It turns that into usage records and hands them to the platform's +billing pipeline, which meters them against price and sends the result to +billing: + +![How Compute's usage reaches the billing pipeline](./architecture-context.png) + +### User Stories + +**Consumption-based billing.** A customer runs a small workload for a few hours +and a large one for a week. Their invoice reflects each: measured vCPU and +memory for exactly the time each instance ran — no more, no less. + +**Stop means stop.** A customer stops a workload on Friday night. Saturday +onwards accrues nothing. When they start it again Monday, billing resumes from +that point — there is no charge for the gap. + +**Suspended costs nothing.** A customer's project is suspended. Instances stop +running, so they stop costing anything, the moment that happens. Billing +resumes only if (and when) the instance actually runs again. + +**An operator trusts the number.** Whether an event is retried, a controller +restarts, or write hiccups occur, the usage that lands is the same quantity it +should be — never double-counted, never silently dropped. + +### How It Works + +**Only running instances bill.** An instance accrues usage while it is running +and stops accruing the moment it leaves the running state (stopped, terminated, +or suspended). Usage is therefore a measure of *actual uptime*, not potential. + +**Per second, batched.** An instance's running time is accounted for in seconds, +but recorded in intervals. Any workload that has been running continuously for +the reporting interval reports its accumulated seconds in one record — e.g. a +workload that ran for five minutes reports 300 seconds (times its resource +size) in one event, not 300 separate events. + +**What gets recorded, per project:** + +- **CPU** — vCPU-seconds: how many vCPUs the instances ran, for how long. +- **Memory** — memory byte-seconds: how much memory the instances used, for how + long. + +**When a workload stops**, the final partial interval is reported up to the +moment it stopped, so the exact time it ran is captured. + +The sequence an instance follows over its life: + +```mermaid +stateDiagram-v2 + [*] --> Running: Instance starts + Running --> Running: Usage accrues while running + Running --> Stopped: Instance stops + Stopped --> Running: Instance restarts + Stopped --> [*]: Terminated + Running --> [*]: Circle = no accrual when not running +``` + +Nothing about this changes how customers run their workloads. They create, +start, stop, and delete instances exactly as today; measuring usage is +invisible to them. + +## Design Details + +### The Instance Controller Is the Metering Point + +Compute already has a component — the **Instance Controller** — that shepherds +every instance through its life: when it starts, when it becomes available, +when it stops, and when it's removed. Only it sees, centrally, that an +instance is actually **running**. Because it already knows this, it is the +natural and only place to record "how long did this instance run." There is no +separate metering service or per-runtime timers; the controller simply adds +usage accounting wherever it already observes an instance changing state. + +### What It Tracks per Instance + +For each instance, the controller remembers two things across its lifetime: + +- **When the current running stretch began** — the moment the instance became + available/running, so the clock can start without it having to be observed + every single second. +- **How far usage has already been reported** — so that on the next report it + only covers new time, never re-counts what was already handed off. + +This bookkeeping belongs to the instance and persists with it, so it survives +the controller itself restarting — metering doesn't lose its place when a +process is recycled. (If the instance is created or its state is reset, the +tracking starts fresh from that point.) + +> [!NOTE] +> This tracking is **persisted with the instance** (in the cluster's API +> storage), not held in the controller's memory — so the controller restarting +> or going down never loses it. + +### How the Running Time Becomes Usage + +On any of the transitions the controller already handles, it turns the tracked +running time into usage: + +- **While running**, it reports in intervals: what has accumulated since the + last report is handed off as one record (CPU + memory for that window). +- **When the instance stops, terminates, or suspends**, it reports the final + partial interval up to the moment of the transition, then stops — so the + exact runtime is captured and nothing accrues once it's no longer running. + +A signed-off report hand-off that fails is retried for the **same window**, so +no double-count on a retry. The controller therefore never has to be exact at +any single instant — it just needs to be exact about *when the running state +changed*, which it already observes. + +### Downtime and Crash Recovery + +The metering bookkeeping is stored with the instance itself, not in the +controller's memory, so an outage does **not** erase running time: + +- **Controller restarts (or is down briefly).** Instances keep running; on + restart the controller reads back each instance's markers and bills from the + last successfully-reported point forward to now. The downtime is recovered + and billed — nothing is missed, it is simply billed late. +- **Controller is down for a long stretch.** Running time keeps accumulating and + is all billed on recovery. Correct, but a larger single catch-up window and a + longer unbilled lag while it is down. + +Two genuinely bounded limits apply: + +- **The instance is deleted while the controller is down.** Its markers are + deleted with it, losing the unreported window — bounded to at most one + reporting interval (default five minutes), because normal operation flushes + at least that often. +- **A crash between handing usage to the pipeline and recording that it was + handed off.** On restart that window is re-reported; whether it double-counts + depends on the pipeline deduplicating by event *content* (quantity) rather + than only by event *identity* — a check the billing platform must honor. + +### Risks and Mitigations + +- **Usage events are rejected because the meter isn't registered/declared.** + Compute's meter definitions and the billing platform's expectations must agree + one-for-one. Mitigation: Compute ships the declarations and emits consistently + against them; a mismatch surfaces loudly in monitoring (see + [Monitoring Requirements](#monitoring-requirements)) rather than silently + underbilling. +- **A transient write failure could, naively, double-count or drop usage.** + Mitigation: recording is designed so a retry produces the *same* usage for the + same interval, and a crash loses at most a bounded, small window. The reported + number stays trustworthy. +- **Operators can't tell whether usage is flowing.** Mitigation: dedicated + signals surface usage being emitted and usage being rejected/errored, so "is + billing working?" is answerable at a glance. +- **This depends on the platform billing pipeline existing and being wired.** + Compute's side can be developed, gated, and tested independently, and only + begins reporting once enabled and pointed at the pipeline. + +## Drawbacks + +- **The platform's first dependency on the billing pipeline for compute.** Until + it's proven, Compute's usage telemetry is new infrastructure carrying the + billing path. This is mitigated by shipping behind a feature gate and + monitoring hand-off closely before broad enablement. +- **Billing is exact, which means it's transactional in nature.** Accurate + per-second usage is more operational care than a flat capacity charge — but + it's exactly what fair, consumption-based billing requires. + +## Alternatives + +- **Bill against reserved capacity (instances *could* run), not actual usage.** + Rejected. It's the status quo's implicit model and doesn't reward efficiency + or reflect real consumption; a customer who never starts a workload would + still be capped-and-charged as if they had. Billing actual usage is the + differential this enhancement exists to provide. +- **Emit a record every single second.** Rejected. Per-second accuracy does not + need per-second events, which would impose unreasonable volume and pipeline + load for no additional billing fidelity (the batching in + [How It Works](#how-it-works) is exact). +- **Have each runtime separately report and reconcile usage.** Rejected in favor + of a single, authoritative metering point in Compute, so all runtimes report + consistently through one path and there is one number to trust. + +## Production Readiness Review Questionnaire + +> Trimmed to the subsections relevant at alpha, following the precedent of other +> compute enhancements. Beta-targeted subsections (rollout/upgrade/rollback +> planning) are deferred. + +### Feature Enablement and Rollback + +- [x] Feature gate + - Usage emission ships behind a feature gate, **off by default**, and is only + enabled once configured and pointed at the billing pipeline. A deployment + that hasn't opted in emits nothing. + - Enabling/disabling takes effect on restart with the changed configuration; + no control-plane downtime. + - Rollback is turning the feature off — usage emission stops; no existing + behavior is affected. + +### Monitoring Requirements + +- [x] Other (dedicated usage/billing telemetry) + - Signals distinguish usage being **emitted** (successfully handed off) from + usage **errored/rejected** (not reaching the pipeline). A rising rejection + rate with flat emission is the "billing may be silently broken" alarm — + especially the case where events are rejected for an undeclared meter. + +### Dependencies + +- The **platform billing pipeline** and the **meter/resource-type definitions** + it fans out from Compute. Usage is only billable if Compute's declared meters + and the pipeline's expectations agree, and events actually reach the pipeline. + If the pipeline is unreachable, emission blocks and surfaces the error (no + silent underbilling). + +### Scalability + +- Usage volume is bounded by the **number of running instances** times the + reporting interval — not by per-second count. A large fleet adds proportionally + more records, but each instance stays a small, infrequent contributor, so the + load scales linearly and manageably with fleet size rather than with wall-clock + seconds. + +### Troubleshooting + +- **"No usage is landing."** Confirm emission is enabled and configured, and that + the pipeline is reachable. If events are being rejected, check that Compute's + declared meters are present and registered on the platform side. +- **"Usage for X looks wrong."** Confirm the instance actually ran for the window + in question (stopped/suspended time is not billable), and consult the + emission/rejection signals to rule out a hand-off problem. + +## Implementation History + +- **Proposed here.** Initial capability and hand-off design. + +## Infrastructure Needed + +- The **platform billing pipeline** must be available and wired to accept + Compute's usage records, and Compute's meter/resource-type declarations must be + registered on the platform side (just-in-time as emission is enabled). +- No new customer-facing surface: measuring usage requires no customer behavior + change. diff --git a/docs/enhancements/per-second-usage-billing/architecture-context.png b/docs/enhancements/per-second-usage-billing/architecture-context.png new file mode 100644 index 00000000..9064030a Binary files /dev/null and b/docs/enhancements/per-second-usage-billing/architecture-context.png differ diff --git a/docs/enhancements/per-second-usage-billing/architecture-context.puml b/docs/enhancements/per-second-usage-billing/architecture-context.puml new file mode 100644 index 00000000..761f5841 --- /dev/null +++ b/docs/enhancements/per-second-usage-billing/architecture-context.puml @@ -0,0 +1,21 @@ +@startuml architecture-context +!$NEW_C4_STYLE = 1 +!include https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master/C4_Component.puml +!include https://raw.githubusercontent.com/datum-cloud/enhancements/main/enhancements/datum-theme.puml + +Person(customer, "Customer", "Runs compute workloads; billed for what they use") + +System(billing, "Billing Platform", "Turns measured usage into invoices") + +System_Boundary(compute, "Compute") { + Container(instances, "Instances", "Virtual machines / workloads running on behalf of the customer") + Container(controller, "Instance Controller", "Knows when an instance is running and records how long it ran", "Measures vCPU + memory usage per project") +} + +Rel_U(customer, compute, "Runs workloads") +Rel_D(instances, controller, "Is observed while running") +Rel_R(controller, billing, "Reports measured usage") +Rel_U(billing, customer, "Invoices based on usage") + +SHOW_LEGEND() +@enduml