Video Tutorial: BYOK with Apache Kafka Cluster

Get Started

The fastest way to deploying a streamtime private cloud is by bringing your own Kubernetes cluster, also known as BYOK. This guide covers the steps to bootstrap a Kubernetes fleet using BYOK.

Pre-requisitres

Any CNCF Kubernetes. We recommend the latest: 1.3.x
You must provide a kubeconfig with a static, long-lived token
Network connectivity between streamtime network and the k8s API endpoint. If private networking is required, you must ensure routability. We recommend atleast 128 available IP addresses (based on the number of clusters you intend to host)
CSI with default storage class set
A load balancer controller.
Node sizing based on our guidelines, based on # of max tenants

Creating a BYOK Kubernetes Fleet

Creating a BYOK fleet in Streamtime is a guided 4-step process:

Choose Bootstrap Provider
- Click “Create Kubernetes Fleet” in the Streamtime UI.
- Select your bootstrap provider. For BYOK, click the “BYOK” option (other options: AWS, GCP, OCI, Azure).
Configure Tenancy & Sizing
- Select tenancy mode (shared, isolated, or dedicated).
- Set your base domain, max tenancy (maximum number of Kafka clusters allowed), and max Kafka units (total throughput capacity).
- Specify the number of Kafka units you want to allocate for this fleet.
Placement Configuration (Optional)
- Optionally, specify provider and region details for placement metadata.
Advanced Configuration
- Upload your Kubernetes kubeconfig file.
- Click deploy. Once the fleet is created, you will see the status as “Healthy”.

Creating an Apache Kafka Cluster on BYOK

Kafka cluster creation is also a 4-step guided process:

Choose Cluster Type
- Select the cluster type (CFK, Apache Kafka, WarpStream, Redpanda).
- For BYOK, select “Apache Kafka”.
Provider & Tenancy
- Choose provider as BYOK.
- Select tenancy mode (must match the BYOK fleet’s tenancy).
- Specify the number of Kafka units required.
Fleet Selection
- Select the BYOK Kubernetes fleet you created in the previous steps.
Advanced Configuration
- Choose authentication identity provider, storage options, Apache Kafka version, and enable/disable auto-upgrade as needed.

API Reference

Streamtime provides a comprehensive REST API for programmatically managing fleets and Kafka clusters. Below are examples for the most common CREATE operations.

Authentication

All API requests require authentication using a Bearer token:

curl -H "Authorization: Bearer YOUR_API_TOKEN" \
     -H "Content-Type: application/json" \
     https://<streamtime-api-endpoint>/organizations/<your-org-id>/...

BYOK Fleet

curl -X POST https://<streamtime-api-endpoint>/organizations/<your-org-id>/infrastructure/ \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "identifier": "additional-alpaca",
    "infra_type": "byok",
    "domain": "<your-domain>.com",
    "max_tenants": 4,
    "max_kafka_units": 17,
    "tenancy_mode": "Shared",
    "advanced_config": {
        "kubeconfig": "<your-kubeconfig-string>"
    },
    "placement_config": {},
    "organization": "acme"
}'

Response:

   {
    "id": "bfca99b0-35a5-4490-9cc4-40296910f76d",
    "organization": "acme",
    "domain": "<your-domain>.com",
    "identifier": "additional-alpaca",
    "provisioner_name": null,
    "destroyer_name": null,
    "infra_type": "byok",
    "tenancy_mode": "Shared",
    "max_tenants": 4,
    "max_kafka_units": 17,
    "kubeconfig": null,
    "kubeconfig_str": null,
    "metadata": {},
    "prometheus_endpoint": null,
    "opencost_endpoint": null,
    "loki_endpoint": null,
    "loadbalancer_dns": null,
    "placement_config": {},
    "advanced_config": {
        "kubeconfig": "<your-kubeconfig-string>"
    },
    "advanced_properties": {},
    "status": "Pending",
    "created_at": "2025-07-17T04:37:33.316063Z",
    "updated_at": "2025-07-17T04:37:33.316090Z",
    "network": null
}

2. Create Kafka Cluster

Apache Kafka Cluster

curl -X POST https://<streamtime-api-endpoint>/v1/clusters \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "identifier": "dutch-mammal",
    "cluster_type": "strimzi",
    "kafka_units": 3,
    "provisioner_name": "",
    "tenancy_mode": "Shared",
    "advanced_config": {
        "number_of_zones": "1",
        "cluster_access": "Internal & External",
        "private_access": false,
        "version": "3.8.0",
        "oauth": "okta-dev-47119201",
        "tiered_storage_bucket_name": "apache-kafka-32421",
        "tiered_storage_provider": {
            "aws_account": "platformatory",
            "aws_region": "us-east-1"
        },
        "tiered_storage_create_bucket": true
    },
    "tags": [
        {
            "key": "environment",
            "value": "non-prod"
        }
    ],
    "organization": "<your-org-id>",
    "cloud_provider": "byok",
    "region": "us-east-1",
    "infrastructure": "additional-alpaca"
}'

Response:

{
    "id": "1a0b72d5-96b9-4123-9cec-71446b6bf45c",
    "organization": "<your-org-id>",
    "infrastructure": "additional-alpaca",
    "identifier": "dutch-mammal",
    "provisioner_name": "",
    "destroyer_name": null,
    "version_updater_name": null,
    "cluster_type": "strimzi",
    "tenancy_mode": "Shared",
    "kafka_units": 3,
    "cloud_provider": "byok",
    "region": "us-east-1",
    "tags": [
        {
            "key": "environment",
            "value": "non-prod"
        }
    ],
    "advanced_config": {
        "number_of_zones": "1",
        "cluster_access": "Internal & External",
        "private_access": false,
        "version": "3.8.0",
        "oauth": "okta-dev-47119201",
        "tiered_storage_bucket_name": "apache-kafka-32421",
        "tiered_storage_provider": {
            "aws_account": "platformatory",
            "aws_region": "us-east-1"
        },
        "tiered_storage_create_bucket": true
    },
    "console_url": null,
    "client_properties": null,
    "new_version": null,
    "s3_manifests_file_path": null,
    "status": "Pending",
    "created_at": "2025-07-17T09:49:48.972570Z",
    "updated_at": "2025-07-17T09:49:48.972586Z"
}