Cloudflare Blog: Log analytics using ClickHouse.
Characteristics of logs
- Unpredictable
- Semi-structured
- Contextual
- Write-heavy
producer, shipper, queue, consumer, datastore (Elasticsearch)
Elasticsearch bottlenecks
- Mapping Explosion
- Multi-tenancy support
- Cluster operational tasks
- Garbage collection
producer, shipper, queue, inserter, datastore (ClickHouse)
Elasticsearch is great for full-text search and ClickHouse is great for analytics.