Span events vs logs

A send fails with “insufficient funds”. Do you record that as a span event or as a log? Until this year the usual answer was the event. OpenTelemetry has since changed its recommendation, and both answers are still around.

What is a span event?

A name, a timestamp and optional attributes, attached to one span (OpenTelemetry traces). It marks a single moment during that span. The rule of thumb from the docs: if the timestamp matters, use an event; if it doesn’t, use an attribute.

Why is OpenTelemetry moving events to logs?

To have one way to emit events. The announcement deprecates Span.AddEvent and Span.RecordException, and new code should emit events through the Logs API. Existing span events stay part of the OTLP trace model. Instrumentations move in their next major versions, and an opt-in variable, OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN, lets them emit both during the move.

So if your backend shows exceptions on spans today, that keeps working. Just don’t build new features on span events.

What does an exception event carry?

The exception convention names the event exception and gives it exception.type, exception.message and exception.stacktrace (exception conventions). recordException fills all three from the error object, and that’s a privacy problem when the message comes from an Ethereum client:

class InsufficientFundsError extends Error {
  name = 'InsufficientFundsError';
}
const err = new InsufficientFundsError(
  'insufficient funds for gas * price + value: address 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266',
);

const a = tracer.startSpan('send 8453');
a.recordException(err); // everything the error carries
a.end();

const b = tracer.startSpan('send 8453');
b.addEvent('exception', { 'exception.type': err.name }); // only what you choose
b.end();

exception [ 'exception.type', 'exception.message', 'exception.stacktrace' ] insufficient funds for gas * price + value: address 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266
exception [ 'exception.type' ]

The first event exports the sender’s address. viem’s error messages can also include calldata and RPC URLs, which sometimes contain API keys.