Skip to main content

Databases & storage overview

zymtrace uses ClickHouse, Postgres and S3-compatible Object Storage as its primary storage backends. For a deeper understanding of how these components interact, refer to the architecture section.

To accommodate different operational needs, zymtrace provides two deployment modes:

  1. Create Mode (create): Deploys and manages the storage service within your cluster. This is ideal for quick setup and seamless integration with zymtrace.
  2. Existing Mode (use_existing): Connects to your existing ClickHouse, Postgres, or S3 storage, eliminating the need to manage additional infrastructure.

Storage Components​

ClickHouse​

ClickHouse serves as the primary analytics database for storing profiling data and metrics. It provides high-performance columnar storage optimized for analytical queries.

Configuration options:

  • Create Mode: Deploy a new ClickHouse instance within your cluster
  • Use Existing: Connect to ClickHouse Cloud, on-premises clusters, or single-node instances

→ Configure ClickHouse

Postgres​

Postgres handles identity management and symbol database storage. It stores user authentication data and debugging symbol metadata.

Configuration options:

  • Create Mode: Deploy a new Postgres instance within your cluster
  • Use Existing: Connect to existing Postgres databases or GCP Cloud SQL

→ Configure Postgres

S3-Compatible Object Storage​

Object storage is used for storing large binary files such as debugging symbols and other artifacts.

Configuration options:

  • Create Mode: Deploy a new MinIO instance within your cluster
  • Use Existing: Connect to MinIO, AWS S3, or Google Cloud Storage

→ Configure Object Storage

Deployment Considerations​

When choosing between Create and Use Existing modes, consider:

Create Mode Benefits:

  • Quick setup and deployment
  • Automatic configuration and integration
  • Simplified management within your cluster
  • Ideal for development and testing environments

Use Existing Benefits:

  • Leverage existing infrastructure investments
  • Better for production environments with established data governance
  • Reduced operational overhead for storage management
  • Better resource utilization across multiple applications

Next Steps​

  1. Choose your deployment mode for each storage component
  2. Follow the detailed configuration guides for each storage type
  3. Update your custom_values.yaml file with the chosen configurations

Quick Reference​

Once you've configured all storage components, deploy the backend using Helm:

helm upgrade backend zymtrace/backend -f custom_values.yaml