Proposed Feature/PR - In-flight request concurrency limiter package

[JimFicarra]

Hi folks,

I recently had a need to limit the number of concurrently executing requests in some of our services instead of implementing ratelimit to protect the services from being over-consumed. In our situation the ratelimit package didn't fit our needs because the nature of the requests we are processing tend to be highly variant in latency and resource consumption.

To address this I created an endpoint.Middleware concurrency limiter package that, like ratelimit, provides capability to:

• deny with error when max request concurrency is reached
• block until in-flight requests are processed and capacity becomes available and can be processed
• be chained with other middleware endpoints (since it's an endpoint.Middlware type)

Go-kit has been very helpful for us and I'd like to contribute this back. Let me know if there's interest and I will submit a PR for review.

Thanks!

-Jim Ficarra

[JimFicarra]

@peterbourgon @ChrisHines @briankassouf @basvanbeek - sorry for the bump - wanted to see if any interest in this PR?

[peterbourgon]

Hey @JimFicarra, sorry for the delay, I had a response typed up some time ago but didn't click Comment 🤦‍♂ Thanks for the idea. Do you have a link to the package?

In general I'm trying to reduce the amount of code in the repo, rather than add to it. I ask: can users implement this feature in their own code? If so, I'd like to avoid bringing it in, and adding to the maintenance burden. Essentially all middlewares can be implemented in user code; changes to e.g. how the HTTP server works cannot.

In the next major version of Go kit (see #843) I expect to remove packages ratelimit, circuitbreaker, sd, and kitgen, at a minimum.

No slight against you, and I hope you'll comment with a link to your package, so anyone who comes by later can find it and use it, if they like. Thanks again!