Speed up the implementation of NATS.Jetstream.API.KV.contents/2

[davydog187]

Problem

While testing the NATS KV store, I was surprised to find out that reading the contents of a bucket was an order of magnitude slower than reading an individual key.

Since its an aggregate operation, I did expect it to be slower, but not by this significant of a margin.

iex(1)> Gnat.Jetstream.API.KV.put_value(:cluster_nats, "example", "foo", "bar")
:ok

# Reading a value single digit milliseconds
iex(2)> :timer.tc(fn -> Gnat.Jetstream.API.KV.get_value(:cluster_nats, "example", "foo") end, :millisecond)
{4, "bar"}

# Reading the contents of a bucket is always at least 100ms
iex(3)> :timer.tc(fn -> Gnat.Jetstream.API.KV.contents(:cluster_nats, "example") end, :millisecond)
{134, {:ok, %{"foo" => "bar"}}}

Upon further inspection, it appears that this is a limitation of the client implementation, where it will always wait a minimum of 100ms from the consumer before returning https://github.com/nats-io/nats.ex/blob/v1.9.0/lib/gnat/jetstream/api/kv.ex#L263

Possible Solutions

1. Document this behavior and make the timeout configurable, noting the risk of missing keys if your timeout is too short
2. Explore other protocol-level features for an alternative implementation (I'm not familiar enough yet to make a determination)
3. Investigate implementations in other languages and copy their strategy
4. ???

I am happy to submit a PR but I wanted to start a conversation first

[davydog187]

It looks like the Go client doesn't have this functionality, instead they only support listing the keys

https://github.com/nats-io/nats.go/blob/main/jetstream/kv.go#L1251

[mmmries]

@brandynbennett I think we talked about this back when we first introduced KV functionality when jetstream was a separate library. Do you remember any details about this behavior?

I think we could switch this to use the PullConsumer API so we will know when we've reached the end of the stream and we can eliminate the 100ms wait

[davydog187]

Interesting, let me play around with PullConsumer and see if I can get a PR going.

[brandynbennett]

@mmmries Sorry, I don't remember the details of this 100ms minimum. Looks like we're using a regular Consumer for the value of that contents function. Maybe there's a way to configure that or use the PullConsumer if that works smoother.