Skip to content

Commit

Permalink
Merge pull request #22 from dbecorp/readme
Browse files Browse the repository at this point in the history
More readme
  • Loading branch information
jakthom authored Oct 22, 2024
2 parents 8293d06 + 99464eb commit 35a1431
Show file tree
Hide file tree
Showing 4 changed files with 33 additions and 17 deletions.
45 changes: 29 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,21 @@
# Hercules

Hercules is a DuckDB-powered Prometheus exporter that makes metrics harvesting go vroom.
![heracles](assets/heracles.png)

Hercules is a DuckDB-powered Prometheus exporter that supercharges metrics.



**Generate** prometheus-compatible metrics from parquet, csv files, json logs, data lakes, databases, http endpoints, and much more.

**Enrich and label** your metrics properly at the source, not in your metrics database.
**Enrich and label** properly at the source, not relabels in your metrics database.

**Supercharge** your metrics harvesting with Prometheus-compatible scrape targets that can run [TPC-H benchmarks](https://www.tpc.org/information/benchmarks5.asp).
**Smash** your metrics harvesting with Prometheus-compatible scrape targets that can tame [TPC-H benchmarks](https://www.tpc.org/information/benchmarks5.asp).


# Getting Started



### Prerequisites

You'll need `go >=1.22` on your machine.
Expand All @@ -29,23 +30,23 @@ make run

### Get Prometheus Metrics

[localhost:9100/metrics](localhost:9100/metric])
[localhost:9100/metrics](http://localhost:9100/metrics)


# Features

### Sources

Hercules materializes metrics from data sources such as:
- Local files (parquet, json, csv, xlsx, etc)
- Object storage (GCS, S3, Azure Blob)
- Data lakes (Iceberg, Delta)
- Databases (PostgreSQL, MySQL, SQLite)
- HTTP endpoints
- Arrow IPC buffers
- **Local files** (parquet, json, csv, xlsx, etc)
- **Object storage** (GCS, S3, Azure Blob)
- **Data lakes** (Iceberg, Delta)
- **Databases** (PostgreSQL, MySQL, SQLite)
- **HTTP endpoints**
- **Arrow IPC buffers**


Hercules sources are represented as either `views` or `tables` - depending on desired performance and specified latency requirements.
Sources are represented as either `views` or `tables` - depending on desired performance and specified latency requirements.

**Example source definition:**

Expand All @@ -62,7 +63,7 @@ sources:

Each Hercules metric is defined using **sql** in a number of supported dialects.

Metric definitions require two fields to be returned in the resultset: a `struct` field of `tags` and a `value` column corresponding to the metric value.
Metric materialization expects two fields in the query resultset: a `struct` field of `tags` and a `value` column corresponding to the metric value.

#### Metric Types

Expand Down Expand Up @@ -107,6 +108,7 @@ metrics:
- 16
```


**Example Summary Metric Definition**
```
summary:
Expand All @@ -125,6 +127,7 @@ metrics:
- 0.99
```


**Example Counter Metric Definition**
```
counter:
Expand All @@ -136,8 +139,18 @@ metrics:
- warehouse
```

### Enrichment

Hercules **sources** and **metrics** can both be joined with *external enrichments*, leading to more ***thorough***, ***accurate*** (or is it precise?) metrics.
### Metric Enrichment

Hercules **sources** and **metrics** can both be joined with external enrichments, leading to more ***thorough***, ***accurate*** (or is it precise?), properly-labeled metrics.

Don't run expensive, time-consuming operations on your centralized Prometheus data store - enrich and label on the edge.


### Macros

Hercules


### Global Prometheus Tags

By enriching your metrics on the edge you no longer have to run *expensive, time-consuming operations on your centralized Prometheus data store.*
Binary file added assets/heracles.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions cmd/hercules/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,5 +91,9 @@ func (d *Hercules) Run() {
defer cancel()
if err := srv.Shutdown(ctx); err != nil {
log.Fatal().Stack().Err(err).Msg("server forced to shutdown")
d.db.Close()
os.Remove(d.config.Db)
}
d.db.Close()
os.Remove(d.config.Db)
}
1 change: 0 additions & 1 deletion hercules.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ version: 1
name: edge
debug: false
port: 9100
db: hercules.db

globalLabels:
- cell: ausw1
Expand Down

0 comments on commit 35a1431

Please sign in to comment.