send txs out

[dvush]

📝 Summary

Allows sending out collected txs.

--tx-receivers-allowed-sources is a list of tx sources to resend
--tx-receivers urls where to send txs (as a octet-stream POST request)

⛱ Motivation and Context

📚 References

---

✅ I have run these commands

• make lint
• make test
• go mod tidy

[metachris]

that place receives duplicates, it should probably be sent at https://github.com/flashbots/mempool-dumpster/blob/main/collector/tx_processor.go#L174

[dvush]

this place in code is after a bunch of operations including writing to a file and request to a node

also duplicates count over all sources but we send only for allowed sources so this can lead to a big loss of txs

I think its better for receiver to do filtering then.

[metachris]

You are mistaken, the transactions are written to a file later, after deduplicating, inside the processTx function.

Your comment about loss of tx because filtering makes sense though. Would be good to add a comment about why this line of code is in this place.

[dvush]

here where writing to a file https://github.com/flashbots/mempool-dumpster/blob/main/collector/tx_processor.go#L123 happens that I refer to
and its before deduplicating

[dvush]

added comment 62915f4

[metachris]

To reuse the HTTP connection, you'll need to read the full body before closing: https://blog.cubieserver.de/2022/http-connection-reuse-in-go-clients/#:~:text=Client%20will%20only%20reuse%20connections,MaxConnsPerHost%20).

// ensure that we close the request so the httpClient can reuse the connection
io.Copy(io.Discard, res.Body)
res.Body.Close()