Comment by PeterCorless

Comment by PeterCorless 15 hours ago

2 replies

I'd argue that since most observability use cases are "write once, read hardly ever" they aren't really transactional (OLTP-oriented). You're doing inserts at a high rate, quick scans in real time, and a few exploratory reads when you need. Generally you're not doing upserts, overwriting existing data; you're doing time-series, where each "moment" has to be captured atomically.

Given that, it makes sense that if you have an OLAP data, like Apache Pinot, you can do the same and better than OLTP. You can do faster aggregations. Quicker large range or full table scans.

(Disclosure: I work at StarTree, which is powered by Apache Pinot)

prabhatsharma 14 hours ago

Not sure, if you got OTLP (Opentelemetry Protocol) and OLTP (Online Transaction Processing) mixed up.

Pinot is cool.

OpenObserve is similar to Pinot but built in rust with Apache arrow Datafusion as the underlying technology and for a different, targeted and tailored use case of Observability.

  • PeterCorless 13 hours ago

    Oh Jeez! Yes, apologies. I even just wrote a whole explainer for our internal folks on OTLP vs. OLTP. (Lolfail!)

    And that's cool! I'll look into OpenObserve.