feat(exporter): Add endpoint for RQ metrics

[tarunps]

Closes frappe/press#1283

• Adds endpoint to fetch RQ metrics for all benches on server
• Adds endpoint to fetch RQ metrics per bench
• Exempts all benches endpoint from token validation
  • Will be proxying this through nginx metrics endpoints and that has it's own validation setup
  • Force redirect the agent endpoint to nginx one

TODO:
- Add to nginx.conf

		location /agent/benches/metrics {
			return 301 /metrics/rq;
		}

		location /metrics/rq {
			proxy_pass http://127.0.0.1:25052/benches/metrics;
		}

[adityahase]

Looks good. Needs a few changes.

The exemption you added will make /agent/benches/metrics publicly accessible without credentials. Add a forced redirect in NGINX config to /metrics/rq.

Add "rq" in https://github.com/frappe/press/blob/26205eb65e9fd32234114299f17809424e63bfbf/press/api/monitoring.py#L55 So Prometheus scrapes this endpoint.

[ankush]

"asa" ???

[ankush]

Looking at diff I can't quite figure out how will this work with bench workers... is this only intended for Agent RQ workers?

[adityahase]

Drop this stray. I don't know why I added it.

[adityahase]

Remove this hardcoded port. You will have to expose redis_queue ports to 127.0.0.1

Ref:

agent/agent/bench.py

Lines 694 to 705 in 48f4dd3

	ssh_port = self.bench_config.get("ssh_port", self.bench_config["web_port"] + 4000)
	ssh_ip = self.bench_config.get("private_ip", "127.0.0.1")
	bench_directory = "/home/frappe/frappe-bench"
	mounts = self.prepare_mounts_on_host(bench_directory)
	command = (
	"docker run -d --init -u frappe "
	f"--restart always --hostname {self.name} "
	f"-p 127.0.0.1:{self.bench_config['web_port']}:8000 "
	f"-p 127.0.0.1:{self.bench_config['socketio_port']}:9000 "
	f"-p 127.0.0.1:{self.bench_config['codeserver_port']}:8088 "