Pretty Waterfalls in Honeycomb for an Absinthe app
When first setting up the Absinthe app I work on, someone added OpenCensus Honeycomb package. The way it was set up was very basic: each top-level query had tracing attached.
:object queries do
field :getStuff, :stuff do
meta :trace, true
arg :input, :string
resolve &Resolver.stuff_resolver/3
end
end
This got us information on how long each query took, so we could see which query needed to be optimized and do some debugging when performance issues hit. But it didn’t drill down like I wanted.
Making it better
The first time I tried to solve this, I looked at how telemetry works in the BEAM, and I didn’t want to do a bunch of manual stuff at the start and end of every function. Then I found Telemetry Decorator and thought I had a handy little solution. Nope. Turns out, BEAM telemetry and OpenCensus are different systems that maybe don’t talk to each other. Or at least, they don’t do automagic.
More recently, I dug into it again. This time, I looked at the OpenCensus documentation on tracing and found out about
with_child_span do
...
end
(Sidebar: I love that they give both Erlang and Elixir examples!)
So, I tried adding that wrapper to a bunch of functions. I got traces for them! …but they weren’t attached to their queries. So then I tried adding with_child_span
to resolver functions, and that was a big mistake. That doesn’t compile; don’t do that.
What was I missing?
Diving into the Absinthe-specific documentation, I spotted this:
Until Absinthe merge and publish their telemetry support (see below) and you upgrade, you’ll also need to set :trace in the metadata for any field for which you want tracing to happen
“Any field.” Not “any query.” Field. Let’s try it.
object :stuff do
field :id, non_null(:id)
field :thing, :string
field :percentage, :integer,
resolve: &Resolver.stuff_percentage/3
end
became
object :stuff do
field :id, non_null(:id)
field :thing, :string
field :percentage, :integer,
meta: [trace: true]
resolve: &Resolver.stuff_percentage/3
end
And we got our pretty waterfall graphs showing up in Honeycomb. They still break anywhere you use Async
functions. I suspect getting around that would involve a closure, grabbing the span from the parent function then passing it as an argument to the Async
-wrapped function, which would then manually set up its context.
TL;DR
Three things to remember when using OpenCensus in an Absinthe app:
- add
meta: [trace: true]
to anything that has a resolver - wrap methods in
with_child_span
- don’t wrap the resolver functions