ClickHouse

Streamtime runs ClickHouse, the columnar analytics database, on a Kubernetes fleet (the Altinity distribution) and keeps it monitored and up to date. You choose how much data it ingests and how long it keeps it; Streamtime sizes the storage from that.

Key Features

  • Sized from your data: storage follows the expected ingestion rate, the retention period and the number of shards.
  • Shards and replicas: scale out with more shards, and add replicas per shard for availability.
  • Tenancy: Shared, Isolated or Dedicated fleets.
  • Monitoring and alerts: metrics, logs and alert rules for every cluster.

Settings

Basic Configuration

  • Identifier: lowercase letters, digits, - and _, up to 24 characters.
  • ClickHouse Provider: Altinity.
  • Retention Days: how long ingested data is kept before it ages out.
  • Expected Ingestion Rate: sustained writes in MB/s (default 10), used to size storage.

Fleet Selection: the Target Kubernetes Fleet. The cloud, tenancy and region you choose only narrow the list.

Advanced Configuration

  • Shards (default 1) and Replicas per shard (default 1).
  • Resources: CPU and memory requests and limits per ClickHouse host.
  • Users, profiles and server settings, passed to ClickHouse as they are.
  • The chart Version, to upgrade (see below).

Before you start

  • A fleet in Healthy state with room for the ClickHouse workload (see Fleet management).
  • For the API: an access token in STREAMTIME_TOKEN and your organization identifier in place of <org>. Replace api.example.com with your Streamtime API address.

The API examples below share this setup:

export STREAMTIME_TOKEN=<your-access-token>
export API=https://api.example.com/organizations/<org>
import os
from streamtime_sdk.client import AuthenticatedClient

client = AuthenticatedClient(
    base_url="https://api.example.com",
    token=os.environ["STREAMTIME_TOKEN"],
)
ORG = "<org>"

Uses the generated client in streamtime-sdk/typescript/client.

import { client } from "./client/client.gen";

client.setConfig({
  baseUrl: "https://api.example.com",
  headers: { Authorization: `Bearer ${process.env.STREAMTIME_TOKEN}` },
});
const org = "<org>";
package main

import (
    "bytes"
    "crypto/rand"
    "encoding/hex"
    "encoding/json"
    "net/http"
    "os"
)

const api = "https://api.example.com/organizations/<org>"

// newKey returns a random idempotency key.
func newKey() string {
    b := make([]byte, 16)
    rand.Read(b)
    return hex.EncodeToString(b)
}

// call sends one request with the access token and a JSON body (or none).
// An optional idempotency key is sent as the Idempotency-Key header.
func call(method, path string, body any, idempotencyKey ...string) (*http.Response, error) {
    var buf bytes.Buffer
    if body != nil {
        if err := json.NewEncoder(&buf).Encode(body); err != nil {
            return nil, err
        }
    }
    req, err := http.NewRequest(method, api+path, &buf)
    if err != nil {
        return nil, err
    }
    req.Header.Set("Authorization", "Bearer "+os.Getenv("STREAMTIME_TOKEN"))
    req.Header.Set("Content-Type", "application/json")
    if len(idempotencyKey) > 0 {
        req.Header.Set("Idempotency-Key", idempotencyKey[0])
    }
    return http.DefaultClient.Do(req)
}

Saving a ClickHouse resource and provisioning it are separate steps in the API: create or change it, then reconcile its new generation. The reconcile returns 202 with the operation it started; the same Idempotency-Key returns that operation again, and a 409 means the generation is out of date.


Create a ClickHouse cluster

  1. In the sidebar, select ClickHouse, then Create ClickHouse.
  2. Basic Configuration: enter the Identifier, choose the ClickHouse Provider, and set the Retention Days and Expected Ingestion Rate.
  3. Fleet Selection: choose the Target Kubernetes Fleet.
  4. Advanced Configuration: set the Shards and Replicas per shard, and the resources if you need to. The Estimated Additional Monthly Cost updates as you go.
  5. Create it. It shows Pending, then Provisioning, then Healthy.
# 1. Save it. The response includes "generation" and "configuration_state".
curl -X POST "$API/clickhouses/" \
  -H "Authorization: Bearer $STREAMTIME_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "identifier": "events",
    "provider": "altinity",
    "fleet": "<fleet>",
    "retention_days": 30,
    "advanced_configuration": {"shards": 1, "replicas": 2, "ingestion_rate_mb_per_sec": 10}
  }'

# 2. Provision it
curl -X POST "$API/clickhouses/events/reconcile/" \
  -H "Authorization: Bearer $STREAMTIME_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"generation": 1}'

Only identifier is required to save a draft; it can be provisioned once configuration_state is valid (a fleet is set).

import uuid

from streamtime_sdk.api.click_house import organizations_clickhouses_reconcile_create
from streamtime_sdk.models import ReconcileRequest

resp = client.get_httpx_client().post(
    f"/organizations/{ORG}/clickhouses/",
    json={
        "identifier": "events",
        "provider": "altinity",
        "fleet": "<fleet>",
        "retention_days": 30,
        "advanced_configuration": {"shards": 1, "replicas": 2, "ingestion_rate_mb_per_sec": 10},
    },
)
resp.raise_for_status()

organizations_clickhouses_reconcile_create.sync(
    ORG,
    "events",
    client=client,
    body=ReconcileRequest(generation=resp.json()["generation"]),
    idempotency_key=str(uuid.uuid4()),
)
import { organizationsClickhousesCreate, organizationsClickhousesReconcileCreate } from "./client";

const { data: saved } = await organizationsClickhousesCreate({
  path: { org_identifier: org },
  body: {
    identifier: "events",
    provider: "altinity",
    fleet: "<fleet>",
    retention_days: 30,
    advanced_configuration: { shards: 1, replicas: 2, ingestion_rate_mb_per_sec: 10 },
  },
});

await organizationsClickhousesReconcileCreate({
  path: { org_identifier: org, identifier: "events" },
  headers: { "Idempotency-Key": crypto.randomUUID() },
  body: { generation: saved!.generation },
});
resp, err := call("POST", "/clickhouses/", map[string]any{
    "identifier":     "events",
    "provider":       "altinity",
    "fleet":          "<fleet>",
    "retention_days": 30,
    "advanced_configuration": map[string]any{
        "shards": 1, "replicas": 2, "ingestion_rate_mb_per_sec": 10,
    },
})
if err != nil {
    return err
}
defer resp.Body.Close()

var saved struct {
    Generation int `json:"generation"`
}
if err := json.NewDecoder(resp.Body).Decode(&saved); err != nil {
    return err
}
resp, err = call("POST", "/clickhouses/events/reconcile/",
    map[string]any{"generation": saved.Generation}, newKey())
streamtime clickhouse create \
  --identifier events \
  --fleet <fleet> \
  --shards 1 --replicas 2 \
  --retention-days 30

The command also starts provisioning once the settings are complete; otherwise run streamtime clickhouse reconcile events when they are.

Not available in MCP yet. Create the cluster from the UI, API or CLI.


Check status

status is one of Pending, Provisioning, Updating, Healthy, Error or Deleting.

  1. In the sidebar, select ClickHouse. Each entry shows its status.

  2. Open it. Overview shows its status, settings and endpoints.

curl -s "$API/clickhouses/events/" -H "Authorization: Bearer $STREAMTIME_TOKEN"

# Collect a fresh status report (waits for the answer)
curl -s -X POST "$API/clickhouses/events/status/" -H "Authorization: Bearer $STREAMTIME_TOKEN"
from streamtime_sdk.api.click_house import organizations_clickhouses_retrieve

res = organizations_clickhouses_retrieve.sync(ORG, "events", client=client)
print(res.status)
import { organizationsClickhousesRetrieve } from "./client";

const { data } = await organizationsClickhousesRetrieve({ path: { org_identifier: org, identifier: "events" } });
console.log(data?.status);
resp, err := call("GET", "/clickhouses/events/", nil)
// Decode "status" from resp.Body.
streamtime clickhouse list
streamtime clickhouse get events

Ask your assistant, for example: “List my ClickHouse clusters” or “Is the events ClickHouse cluster healthy?”

It uses list_clickhouses and get_clickhouse.


Change the configuration or upgrade

Add shards or replicas, change the resources or the retention, or upgrade to a new Version (advanced_configuration.chart_config.main.version). Send the whole advanced_configuration, not just the keys you change.

  1. Open it from ClickHouse and select Configuration (or Edit).
  2. Change the settings, then save. Streamtime applies the change, and the status shows Updating, then Healthy.
# 1. Save the change. The response includes the new "generation".
GEN=$(curl -s -X PATCH "$API/clickhouses/events/" \
  -H "Authorization: Bearer $STREAMTIME_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"advanced_configuration": {"shards": 2, "replicas": 2, "ingestion_rate_mb_per_sec": 10}}' | jq .generation)

# 2. Apply it
curl -X POST "$API/clickhouses/events/reconcile/" \
  -H "Authorization: Bearer $STREAMTIME_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d "{\"generation\": $GEN}"

The SDK has no typed call for this update yet, so save it with the SDK’s HTTP client, then apply it:

import uuid

from streamtime_sdk.api.click_house import organizations_clickhouses_reconcile_create
from streamtime_sdk.models import ReconcileRequest

resp = client.get_httpx_client().patch(
    f"/organizations/{ORG}/clickhouses/events/",
    json={"advanced_configuration": {"shards": 2, "replicas": 2, "ingestion_rate_mb_per_sec": 10}},
)
resp.raise_for_status()

organizations_clickhouses_reconcile_create.sync(
    ORG, "events",
    client=client,
    body=ReconcileRequest(generation=resp.json()["generation"]),
    idempotency_key=str(uuid.uuid4()),
)
import { organizationsClickhousesReconcileCreate } from "./client";

const { data: saved } = await client.patch({
  url: "/organizations/{org_identifier}/clickhouses/events/",
  path: { org_identifier: org },
  body: {"advanced_configuration": {"shards": 2, "replicas": 2, "ingestion_rate_mb_per_sec": 10}},
});

await organizationsClickhousesReconcileCreate({
  path: { org_identifier: org, identifier: "events" },
  headers: { "Idempotency-Key": crypto.randomUUID() },
  body: { generation: (saved as { generation: number }).generation },
});
resp, err := call("PATCH", "/clickhouses/events/", json.RawMessage(`{"advanced_configuration": {"shards": 2, "replicas": 2, "ingestion_rate_mb_per_sec": 10}}`))
if err != nil {
    return err
}
defer resp.Body.Close()

var saved struct {
    Generation int `json:"generation"`
}
if err := json.NewDecoder(resp.Body).Decode(&saved); err != nil {
    return err
}
resp, err = call("POST", "/clickhouses/events/reconcile/",
    map[string]any{"generation": saved.Generation}, newKey())

Changing the settings isn’t available in the CLI yet; save them from the UI or API. Then apply the change:

streamtime clickhouse reconcile events

Not available in MCP yet. Change the configuration from the UI or API.


View metrics, logs and alerts

Open it from ClickHouse and select Monitoring for its metrics and logs, and Alert Rules (where shown) for the alerts that watch it. Alerts that fire also appear on the fleet’s Monitoring tab.

Metrics and logs are read through the fleet that runs it:

curl -s "$API/fleet/<fleet>/metrics/overview/" -H "Authorization: Bearer $STREAMTIME_TOKEN"
curl -s "$API/fleet/<fleet>/alerts/" -H "Authorization: Bearer $STREAMTIME_TOKEN"
resp = client.get_httpx_client().get(f"/organizations/{ORG}/fleet/<fleet>/alerts/")
print(resp.json())
const { data: alerts } = await client.get({
  url: "/organizations/{org_identifier}/fleet/{fleet_id}/alerts/",
  path: { org_identifier: org, fleet_id: "<fleet>" },
});
console.log(alerts);
resp, err := call("GET", "/fleet/<fleet>/alerts/", nil)
# The fleet's logs from the last hour
streamtime logs fleet <fleet> --since 60

Reading metrics or alerts isn’t available in the CLI yet.

Ask your assistant, for example: “Any alerts on the events ClickHouse workload today?” or “Show its error logs from the last hour.”

It uses get_fleet_alerts, get_fleet_logs, list_fleet_metrics and get_fleet_metric on the fleet that runs it.


Delete

Deleting removes what Streamtime installed for it on the fleet.

  1. In the sidebar, select ClickHouse.
  2. Open its menu in the list, select Delete, and confirm.

The status shows Deleting, then it disappears from the list.

curl -X DELETE "$API/clickhouses/events/" -H "Authorization: Bearer $STREAMTIME_TOKEN"

A draft that was never provisioned is removed at once; otherwise the teardown starts. Poll GET $API/clickhouses/events/ until it returns 404.

from streamtime_sdk.api.click_house import organizations_clickhouses_destroy

organizations_clickhouses_destroy.sync_detailed(ORG, "events", client=client)
import { organizationsClickhousesDestroy } from "./client";

await organizationsClickhousesDestroy({ path: { org_identifier: org, identifier: "events" } });
resp, err := call("DELETE", "/clickhouses/events/", nil)
streamtime clickhouse delete events --yes

Not available in MCP yet. Delete it from the UI, API or CLI.