Skip to main content

ClickHouse Configuration

ClickHouse serves as the primary analytics database for storing profiling data and metrics. It provides high-performance columnar storage optimized for analytical queries.

Configuration Modes​

Connect to existing ClickHouse​

This mode connects to your existing ClickHouse cluster, including ClickHouse Cloud, on-premises clusters, and single-node instances.

Supported ClickHouse Deployments

  • ClickHouse Cloud: Managed ClickHouse service with automatic scaling and management
  • ClickHouse Clusters: Multi-node distributed setups with shards and replicas
  • Single-node ClickHouse: Standalone ClickHouse server installations

Database Setup Options​

You have two options for setting up the required databases:

  1. Manual Setup: Create databases and users manually using SQL commands.
  2. Automatic Setup: Enable autoCreateDBs to let zymtrace create databases during migration

Option 1: Manual Database Setup​

When connecting to an existing ClickHouse cluster, we strongly recommend that you create a dedicated user, assign appropriate permissions, and create a new database for zymtrace.

If you're using a distributed ClickHouse cluster with multiple shards/replicas, you have two options:

  1. Use the {cluster} macro (recommended): A common convention where {cluster} is configured as a macro that resolves to your cluster name
  2. Find the actual cluster name: Query your ClickHouse installation to get the specific cluster name
-- Check if the {cluster} macro is available
SELECT * FROM system.macros;

-- Find available clusters in your ClickHouse installation
SELECT cluster FROM system.clusters;

SQL Commands for Database Setup​

Choose the appropriate commands based on your ClickHouse deployment.

Caution:

This script will drop the zymtrace_profiling database if it exists. Run it only if this is your first time setting up zymtrace with ClickHouse, or if you intentionally want to reset the database, knowing that this will permanently delete any existing data.

ClickHouse Cloud and Single Node​

For single-node ClickHouse installations or ClickHouse Cloud:

-- Clean slate: remove any existing profiling database
DROP DATABASE IF EXISTS zymtrace_profiling SYNC;
DROP DATABASE IF EXISTS zymtrace_metrics SYNC;

CREATE DATABASE zymtrace_profiling;
CREATE DATABASE zymtrace_metrics;

CREATE USER IF NOT EXISTS zymtrace_user
IDENTIFIED WITH sha256_password BY 'YOUR NEW PASSWORD HERE';

GRANT
    SELECT,
    INSERT,
    UPDATE,
    ALTER,
    DELETE,
    CREATE,
    DROP,
    SHOW,
    OPTIMIZE,
    TRUNCATE,
    SYSTEM SYNC REPLICA
ON zymtrace_profiling.* TO zymtrace_user;

GRANT
    SELECT,
    INSERT,
    UPDATE,
    ALTER,
    DELETE,
    CREATE,
    DROP,
    SHOW,
    OPTIMIZE,
    TRUNCATE,
    SYSTEM SYNC REPLICA
ON zymtrace_metrics.* TO zymtrace_user;

Note that SYSTEM SYNC REPLICA is only strictly required when running in ClickHouse Cloud and optional when using a self-hosted single node instance.

Make sure to replace YOUR NEW PASSWORD HERE with a secure password.

Reference - https://clickhouse.com/docs/sql-reference/statements/grant

Option 2: Automatic Database Setup​

Alternatively, you can enable autoCreateDBs: true in your configuration to let zymtrace automatically create the required databases during migration. This option requires that your ClickHouse user has CREATE permissions on the server.

When to use automatic setup:

  • Development environments where quick setup is preferred
  • When you have administrative access to grant broad CREATE permissions
  • Testing scenarios where database recreation is acceptable

When to use manual setup:

  • Production environments requiring strict permission control
  • When following security best practices with minimal required permissions
  • Enterprise environments with database administration policies
info

When using ClickHouse Cloud, ensure you use port 8443 and enable secure connection. For ClickHouse clusters, specify the cluster name to enable distributed queries.

Helm Configuration​

ClickHouse Cloud / Single Node​

This configuration applies to both ClickHouse Cloud and standalone single-node ClickHouse installations.

clickhouse:
  mode: "use_existing"
  use_existing:
    host: "https://abc123def.us-east-1.aws.clickhouse.cloud:8443" # For ClickHouse Cloud, 
           # OR "http://clickhouse.internal.company.com:8123" # For single-node
    user: "default" # For Cloud, or "zymtrace_user" for single-node
    password: "your-cloud-password" # For Cloud, or "secure-password" for single-node
    database: "zymtrace"
    secure: true # For Cloud, or false/true for single-node depending on TLS
    # clusterName not needed for Cloud or single-node
    # autoCreateDBs: true  # or create the DBs yourself using the SQL above

HTTP Interface Requirement​

Protocol Requirement

Only the ClickHouse HTTP interface is supported due to limitations in the official ClickHouse Rust client. The native protocol port is not supported. Always use the HTTP interface port:

  • HTTP: Port 8123 (typically for internal/dev environments)
  • HTTPS: Port 8443 (recommended for production and ClickHouse Cloud)

URL Format Requirements​

The host field must include:

  • Protocol: http:// or https://
  • Hostname/IP: The ClickHouse server address
  • Port: The HTTP interface port

Next Steps​

After configuring ClickHouse, proceed to configure the other storage components:

Configure Postgres | Configure Object Storage