Comment by h1fra

Comment by h1fra 3 days ago

4 replies

I have one biff against otel, it's not possible to stream a trace. You have to build a big object in memory that is not suitable for a long-running process. And because of that it's not possible to start it somewhere and finish it somewhere else

thephyber 3 days ago

I'm under the impression this is wrong. A trace is made from one or more spans. So long as the context is propagated[1] from one service to another, any number of spans referring to the same trace can be generated anywhere at any time. The trace+spans don't have to be created in the order of a stack.

[1] https://opentelemetry.io/docs/concepts/context-propagation/

  • phillipcarter 3 days ago

    This is correct. Traces are made up of spans that can be created within the same process, different processes, different machines, etc. and all emitted asynchronously.

eterm 3 days ago

I don't think that's a limitation of the specification. You can create spans and emit events for those spans without holding objects in memory.

malkia 2 days ago

There is no concept of streaming trace. You are propagating a random ID, and expect other NODES (yours, or outside of your control) to re-emit these, and if need to create new ones.

The agreement is that these nodes would either emit all these events to (eventually) a common place (push), or something is going to gather them (pull).

I think at Google, some of these used to be still on the machines, and the tools would pull them directly, and back then it was possible to mark certain for preservance (that was long time ago - 2014, so I'm sure things have changed).

Also I was on the over-excited edge, because I was not aware what was this, but I was on call (a small team in ads), and had to page up to the Bigtable/Megastore or was it Spanner team, and they simply asked me to bump some tracing bits up for like 30 seconds, then something magically showed up - and I was - wtf!

I think it was then it clicked with me how useful this is, .... but also how much wasteful (in terms of resources) it could be if you don't end up looking there.