Skip to content

Latest commit

 

History

History
105 lines (73 loc) · 3.43 KB

TODO.org

File metadata and controls

105 lines (73 loc) · 3.43 KB

katip

Should we remove orphan IO instance and add a LoggingT?

Add a “Maybe LocInfo” using a TH version of logF

Log current PID in Item

Rethink important fields

Maybe add a logF variant that uses the class, but it’s not

mandatory

Another variant that explicitly passes the fields of interest

Add Verbosity to the mix

AppName -> [Text]

  • LogEnv defines the base name-space
  • logF gets a [Text] argument for AppName namespace
  • Whe two are concatted for each message

katip-elasticsearch

Efficient UUID generation (don’t use system rng)

Make RNG once, store it behind an mvar and use it in all uuid generation. Use the rng-utils module for this.

Turns out uuid is already doing some clever stuff in their random instance to make this plenty fast. Benchmarks show we can generate > 1M ids/sec.

Async size-capped buffer for incoming items

Don’t block log requests for server connection, add to buffer and move on. If buffer fills up, drop further logs until there’s space.

We can use a limited chan with a pre-check to make sure that if the size limit is reached, we drop the request instead of blocking until limit is reached.

Be resilient to server disconnects and crashes

The async buffer method described above certainly helps with this. In addition, make sure that nothing out of the ordinary happens if the server is unreachable. Logging should not be an element in any stack that can potentially induce a crash - that would defeat the purpose!

This means:

  • No slowdowns, no crashes even if server is unreachable
  • No file descriptor hogging (tight loop retries, etc.)
  • Gently try once in a while to see if the server has become reachable again

Connection pooling (bloodhound)

Exposing Manager would solve this, as it does internal pooling.

Stop using Strings everywhere (bloodhound)

Seems to be a limitation of http-client that we’ll have to live with for now.

update index settings on setup (when the API becomes available in bloodhound)

by-day or user scheduled index sharding

Consider using templates and write to indexes per-day. So rather than building the direct index the user asks for, set up a template that automatically gets applied to indices matching the prefix, then create an index per day. Apparently this allows more efficient searches. Its yet to be seen when one-big-index breaks down

katip-rotated-log

Create a rotated-log scribe

This works just like the built-in Handle scribe, but pushes the log through the rotated-log packages and therefore rotates files when they get too large

katip-retry

Implement combinator for creating exception Handler that log to katip

Along the lines of the built-in ‘logRetries’ in retry, but instead captures structured information through katip.

Explicitly capture fields to aid in filtering:

  • Retry number
  • The exception
  • The exception type - maybe via Typeable’s typeOf?
  • Decision result: Crashing vs. Retrying