Introduction
Welcome to the Iceburst.io Documentation! Iceburst is an open-source real-time data lake for monitoring and security.
Last updated
Welcome to the Iceburst.io Documentation! Iceburst is an open-source real-time data lake for monitoring and security.
Last updated
Our approach is simple: ingest all the telemetry and security data in an S3 bucket as Parquet files in Iceberg table format and query them using DuckDB with milliseconds retrieval and zero egress cost. Parquet is converted to Arrow format in-memory for efficient query processing.
S3 as a storage layer architecture enables us to offer so many advantages out-of-the-box like separation of storage and compute, time travel, fault tolerance, infinite concurrency reads, fast recovery, and a better developer experience.
Simple to adopt
There is no need to install and maintain a Kubernetes or Kafka cluster to start using Iceburst. We have a simple S3 Exporter for OpenTelemetry to kickstart your setup.
Simple to scale
When it comes time to scale up to terabytes per day, simply switch to Iceburst Exporter for OpenTelemetry and we will do all the heavy lifting of setting up Kafka for data streaming and Flink jobs for stream processing.
Improved efficiency
Better query performance and new query functionality drastically improve the overall efficiency. In addition, the cost savings that is realized because of S3 as the data store.
Simple to adopt
There is no need to install and maintain a Kubernetes or Kafka cluster to start using Iceburst. We have a simple S3 Exporter for OpenTelemetry to kickstart your setup.
Simple to scale
When it comes time to scale up to terabytes per day, simply switch to Iceburst Exporter for OpenTelemetry and we will do all the heavy lifting of setting up Kafka for data streaming and Flink jobs for stream processing.
Improved efficiency
Better query performance and new query functionality drastically improve the overall efficiency. In addition, the cost savings that is realized because of S3 as the data store.
