zymtrace symbolization service
What is symbolization?​
Symbolization is the process of converting machine addresses from stack traces in ingested profiles into human-readable source code line mappings. This process is essential for understanding program behavior, analyzing performance, and debugging effectively.
In the context of profiling, a frame represents a single function call in a program's execution stack. A symbol for a particular frame consists of a list of <file name, function name, source line>
triplets. This is necessary because inlining may associate multiple functions and source lines with a single frame.
Native frames collected by the zymtrace whole-system profiler may originate from:
- System libraries (part of container images or the host OS)
- Operating system daemons
- Third-party software
- Native libraries used by in-house software
Our symbolization approach​
zymtrace delivers symbolization through two methods:
- Global symbolization service – Automatically retrieves and processes symbols for public and open-source software
- Custom symbol upload – Allows users to upload their own symbols for proprietary software.
Our solution provides:
- Seamless integration: Works without requiring debug symbols on the host
- Zero configuration: Automatically symbolizes common libraries with minimal input
- Flexibility: Supports both public and private symbols
- Efficiency: Uses GSYM format for minimal storage and efficient lookups
- Security/privacy: The processed symbols can be cloned on-premise
- Comprehensive coverage: Supports multiple Linux distributions and Docker containers
Global symbolization service​
zymtrace automatically crawls debugging packages into a global symbolization service, enabling efficient and automated symbol retrieval. This service:
- Crawls Debian, Alpine, and Ubuntu repositories, including popular Docker containers, for debug information.
- Processes and stores debug symbols as GSYM files, a compact and efficient format for symbol information.
- Indexes and stores these files in Google Cloud Storage.
- Allows customers to clone this symbolization database for on-premises use.
This approach ensures that common libraries and system components are automatically symbolized with minimal user effort.
Custom symbols​
While the global symbolization service covers many common libraries and kernel components, zymtrace also provides a way to extract and upload symbols for proprietary software via our command-line tool. The tool can either be run on-demand or integrated into CI/CD piplines.
This process is documented on the uploading symbols page.