Make MAX_RETRIES configurable

[JosiahParry]

I noticed that the maximum number of retries is set to 16.

pingora/pingora-proxy/src/lib.rs

Line 68 in bdb13a7

const MAX_RETRIES: usize = 16;

With my proxy a user may be sent to an application that needs to perform a cold start. That time may be a few seconds or longer.

I have set the retry to wait 100ms before retrying.

    fn fail_to_connect(
        &self,
        _session: &mut Session,
        _peer: &HttpPeer,
        _ctx: &mut Self::CTX,
        mut e: Box<Error>,
    ) -> Box<Error> {
        e.set_retry(true);
        std::thread::sleep(std::time::Duration::from_millis(100));
        e
    }

Ideally, I'd like to be able to configure it to poll every x ms for a maximum Duration before finally giving up instead of a fixed number of retries.

[gumpt]

Envoy's retry policies are worth looking at here for inspiration/a good set of tunables. https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-msg-config-route-v3-retrypolicy

[xlfish233]

Is this issue still open? I'd like to work on implementing a more configurable retry strategy.

[JosiahParry]

@xlfish233 not that i am aware of! This would be an awesome feature