Skip to content

Commit 714265f

Browse files
[K2] Add K2 documentation (#33858)
* Add K2 documentation * AI feedback * Add CODEOWNERS * re-run CI --------- Co-authored-by: Marc Selwan <marc@marcinthecloud.com>
1 parent ed083b2 commit 714265f

13 files changed

Lines changed: 1292 additions & 0 deletions

File tree

‎.github/CODEOWNERS‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -521,6 +521,14 @@ package.json @cloudflare/content-engineering
521521
/src/content/release-notes/images.yaml @deannalam @dochne @renandincer @third774 @cloudflare/product-owners
522522
/src/icons/images.svg @deannalam @dochne @renandincer @third774 @cloudflare/product-owners
523523

524+
# k2
525+
526+
/src/content/docs/k2/ @Marcinthecloud @mwylde @sejoker @cloudflare/product-owners
527+
/src/assets/images/k2/ @Marcinthecloud @mwylde @sejoker @cloudflare/product-owners
528+
/src/content/changelog/k2/ @Marcinthecloud @mwylde @sejoker @cloudflare/pm-changelogs @cloudflare/product-owners
529+
/src/content/directory/k2.yaml @Marcinthecloud @mwylde @sejoker @cloudflare/product-owners
530+
/src/icons/k2.svg @Marcinthecloud @mwylde @sejoker @cloudflare/product-owners
531+
524532
# key-transparency
525533

526534
/src/content/docs/key-transparency/ @lschull @mgalicer @cloudflare/appsec-reviewers @cloudflare/product-owners

‎src/components/landing/BuildFromScratch.astro‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ const primitives = [
7272
tags: [
7373
{ label: "R2", href: "/r2/" },
7474
{ label: "Basin", href: "/basin/" },
75+
{ label: "K2", href: "/k2/" },
7576
{ label: "D1", href: "/d1/" },
7677
{ label: "KV", href: "/kv/" },
7778
{ label: "Hyperdrive", href: "/hyperdrive/" },

‎src/components/landing/sidebar-data.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,7 @@ export const sidebarSections: SidebarSection[] = [
128128
link("Basin SQL", "/basin-sql/"),
129129
],
130130
},
131+
link("K2", "/k2/"),
131132
link("D1", "/d1/"),
132133
link("KV", "/kv/"),
133134
link("Hyperdrive", "/hyperdrive/"),

‎src/content/directory/k2.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
id: FKFjPd
2+
name: K2
3+
4+
entry:
5+
title: K2
6+
url: /k2/
7+
group: Developer platform
8+
additional_groups: [Storage]
9+
10+
meta:
11+
title: Cloudflare K2 docs
12+
description: A durable event stream
13+
author: "@cloudflare"
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
---
2+
title: Concepts
3+
description: Learn about the key ideas behind K2, including streams, records, subscriptions, and leases.
4+
pcx_content_type: concept
5+
products:
6+
- k2
7+
sidebar:
8+
order: 3
9+
---
10+
11+
This page introduces the concepts underlying K2.
12+
13+
## Streams
14+
15+
The core primitive in K2 is what we call a _stream_ — an ordered, durable, log of
16+
events. The stream sits between producers writing events and consumers
17+
reading them. Unlike a traditional queue, where consumption removes items,
18+
production and consumption in a log are completely decoupled. Writes append
19+
events to the log, while reads merely advance a pointer (or _offset_)
20+
within the log.
21+
22+
This has some useful properties:
23+
24+
- Writes and reads are completely independent, so we never run out of
25+
space or otherwise block writes due to slow reads
26+
- We can support multiple independent readers consuming the entire stream
27+
(pub-sub style) as readers do not affect each other or the log
28+
- We can support historical replay, as data is only removed based on a configurable
29+
time-to-live (TTL)
30+
31+
Each stream has a name and a retention period. When you create a stream, K2 assigns it a unique ID,
32+
which producers and consumers use to communicate with it. Producers write to a stream through
33+
an HTTP endpoint (with or without authentication), a Workers binding, or both.
34+
35+
## Records
36+
37+
Records are the data written to a stream. Each record has the following fields:
38+
39+
- `content`: a binary message, containing arbitrary data; base64-encoded in the HTTP APIs
40+
- `headers`: an optional map of string keys to string values that can be used to describe the data in the content
41+
42+
Headers are useful for describing the content, without needing to deserialize it. Common use
43+
cases for headers include:
44+
45+
- Storing the encoding, so the reader knows how to deserialize the content
46+
- Representing data used to route or filter events, improving efficiency by avoiding deserialization
47+
when not necessary
48+
- Annotating content in a pass-through pipeline without needing to modify the underlying data
49+
50+
Records can be up to 1 MB, counting across both content and headers.
51+
52+
## Subscriptions
53+
54+
Reads from K2 are performed via _subscriptions_. Each subscription will receive all messages
55+
in the stream, and multiple consumers can share a single subscription.
56+
57+
This enables K2 to support two delivery strategies: reads can be shared amongst
58+
a set of consumers (such that each consumer gets a subset of the messages), or
59+
delivered to all consumers independently (such that each consumer gets all
60+
messages). These strategies can also be mixed, with multiple groups of
61+
consumers which each get a subset of the messages.
62+
63+
Subscriptions are created with an initial position in the log: either `earliest`, which receives
64+
all retained (not deleted according to the TTL) data in the stream, or `latest` which receives
65+
all events from the time the subscription is created.
66+
67+
## Leases
68+
69+
Once a subscription is created, clients can consume from it by POSTing to the subscription's
70+
`/consume` endpoint. This returns a list of messages which this client is expected to process.
71+
These messages are _leased_ to that client for a particular amount of time — the _lease period_,
72+
which is 5 minutes. The client is expected, before the lease expires, to either _ack_ the messages,
73+
telling the subscription that they are successfully consumed, or _nack_ them, indicating a processing
74+
failure. Events owned by a nack'd or timed-out lease will be redelivered on a subsequent call to
75+
consume.
Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
title: Configuration
3+
description: Create, update, and delete K2 streams, and configure their inputs, authentication, CORS, and retention.
4+
pcx_content_type: configuration
5+
products:
6+
- k2
7+
sidebar:
8+
order: 5
9+
---
10+
11+
import { Details } from "~/components";
12+
13+
K2 streams can be created, updated, and deleted with the REST API.
14+
15+
To create, update, or delete streams, your API token needs the `K2 Config Write` permission. To get or list streams, it needs the `K2 Config Read` permission.
16+
17+
{/* TODO: Add Dashboard and Wrangler instructions once they support K2 streams. */}
18+
19+
## Stream settings
20+
21+
| Setting | Type | Required | Default | Description |
22+
| ------------------- | ------- | -------- | ------------------- | ----------------------------------------------------------------------------------------------- |
23+
| `name` | string | Yes | None | 1 to 128 letters, numbers, or underscores. Must be unique in your account. Not case-sensitive. |
24+
| `retention_seconds` | integer | No | `604800` | How long K2 retains records, from `3600` (one hour) to `2592000` (30 days). |
25+
| `http` | object | Yes | None | Configures the HTTP input. Refer to [HTTP input](#http-input). |
26+
| `worker_binding` | object | No | `{ enabled: true }` | Configures the Workers binding input. Refer to [Workers binding input](#workers-binding-input). |
27+
28+
At least one of `http` or `worker_binding` must be enabled.
29+
30+
You cannot rename a stream after you create it.
31+
32+
### Inputs
33+
34+
An input is a way for producers to write records to a stream. K2 supports two inputs:
35+
36+
- **HTTP:** Producers send records to the stream's `/produce` endpoint.
37+
- **Workers binding:** A Worker sends records through a binding.
38+
39+
### HTTP input
40+
41+
| Field | Type | Required | Description |
42+
| ---------------- | ---------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------- |
43+
| `enabled` | boolean | Yes | Enables the `/produce` endpoint. |
44+
| `authentication` | boolean | No | Requires an API token with the `K2 Produce` permission to produce. If omitted or `false`, anyone with the stream endpoint can produce. |
45+
| `cors.origins` | array of strings | No | Origins allowed to produce from a browser. Refer to [CORS](#cors). |
46+
47+
:::caution
48+
If you do not set `authentication` to `true`, the `/produce` endpoint is public. Anyone who knows the stream ID can write records to the stream.
49+
:::
50+
51+
#### Authentication
52+
53+
When `authentication` is `true`, producers using the HTTP API must send an API token in the
54+
`Authorization: Bearer <TOKEN>` header. The token must have permission to
55+
produce to K2 streams (`K2 Produce`) in the account that owns the stream.
56+
57+
#### CORS
58+
59+
Configure `cors.origins` to allow browsers to produce records from a web page. Each entry must be one of the following:
60+
61+
- An `http://` or `https://` origin, such as `https://example.com`. Origins cannot include a path, query string, fragment, or credentials.
62+
- `*`, to allow any origin. If you use `*`, it must be the only entry.
63+
64+
You can configure up to five origins. Each origin must be unique.
65+
66+
### Workers binding input
67+
68+
| Field | Type | Required | Description |
69+
| --------- | ------- | -------- | ------------------------------------------------------- |
70+
| `enabled` | boolean | Yes | Allows Workers to produce to the stream with a binding. |
71+
72+
If you omit `worker_binding` when you create a stream, the Workers binding input is enabled.
73+
74+
### Retention
75+
76+
`retention_seconds` sets how long K2 retains records after it receives them.
77+
The value must be between `3600` (one hour) and `2592000` (30 days). The
78+
default is `604800` (seven days).
79+
80+
K2 deletes expired records in the background. Records can remain readable for some time after their retention period ends, so do not rely on retention to remove data at an exact time.
81+
82+
## Update a stream
83+
84+
To change a stream's settings, send a `PATCH` request with the settings to change. You can update `retention_seconds`, `http`, and `worker_binding`. Include at least one of these fields.
85+
86+
```sh
87+
curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/k2/streams/$STREAM_ID" \
88+
--request PATCH \
89+
--header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
90+
--header "Content-Type: application/json" \
91+
--data '{
92+
"retention_seconds": 86400,
93+
"worker_binding": { "enabled": false }
94+
}'
95+
```
96+
97+
The response contains the updated stream:
98+
99+
```json output
100+
{
101+
"success": true,
102+
"errors": [],
103+
"messages": [],
104+
"result": {
105+
"id": "241fa65b438a4d539a19371f58bfdae0",
106+
"name": "orders",
107+
"retention_seconds": 86400,
108+
"endpoint": "https://241fa65b438a4d539a19371f58bfdae0.k2.cloudflarestorage.com",
109+
"http": {
110+
"enabled": true,
111+
"authentication": true
112+
},
113+
"worker_binding": {
114+
"enabled": false
115+
},
116+
"created_at": "2026-09-24T21:19:19.246Z",
117+
"modified_at": "2026-09-29T14:25:54.712Z"
118+
}
119+
}
120+
```
121+
122+
When you update an input, the new object replaces the existing one. For
123+
example, to add a CORS origin to the HTTP input, send the complete `http`
124+
object, including `enabled` and `authentication`. To disable an input, set it
125+
to `{ "enabled": false }`. You cannot disable both inputs.

0 commit comments

Comments
 (0)