Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document actor paths configuration #635

Merged
merged 4 commits into from
Mar 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,6 @@ def exampleCommonSettings = Seq(
),
libraryDependencies ++= {
logback ++ Seq(
"io.opentelemetry" % "opentelemetry-sdk-extension-autoconfigure" % OpentelemetryAlphaMinor0Version,
"io.grpc" % "grpc-netty-shaded" % "1.53.0",
"org.wvlet.airframe" %% "airframe-log" % AirframeVersion
)
Expand Down
8 changes: 4 additions & 4 deletions core/src/main/scala/io/scalac/mesmer/core/ActorGrouping.scala
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,12 @@ object ActorGrouping {

final case class InvalidRule(rule: String, message: String) extends Error

def fromRule(rule: String, reporingType: Reporting): Either[Error, ActorGrouping] =
def fromRule(rule: String, reportingType: Reporting): Either[Error, ActorGrouping] =
rule match {
case _ if rule.endsWith("/*") =>
val matcherPath = rule.dropRight(2).pipe(toValidPath)

reporingType match {
reportingType match {
case Reporting.Group =>
PathMatcher
.prefix(matcherPath, false)
Expand Down Expand Up @@ -101,7 +101,7 @@ object ActorGrouping {
case _ if rule.endsWith("/**") =>
val matcherPath = rule.dropRight(3).pipe(toValidPath)

reporingType match {
reportingType match {
case Reporting.Group =>
PathMatcher
.prefix(matcherPath, true)
Expand Down Expand Up @@ -136,7 +136,7 @@ object ActorGrouping {
case x if !x.contains("*") =>
val matcherPath = rule

reporingType match {
reportingType match {
// TODO this should result in Left => there is not sense to group by one actor
case Reporting.Group =>
PathMatcher
Expand Down
34 changes: 0 additions & 34 deletions core/src/main/scala/io/scalac/mesmer/core/PathMatcher.scala
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,6 @@ sealed trait PathMatcher extends (String => Boolean) with Ordered[PathMatcher] {

def matches(path: String): Boolean

/**
* Return Some with a result of map function called with part that passed the matcher
* @param map
* @param path
* @tparam T
* @return
*/
def withPassing[T](map: String => T)(path: String): Option[T]

def compare(that: PathMatcher): Int =
if (sameBase(that)) {
priority - that.priority
Expand Down Expand Up @@ -52,29 +43,13 @@ object PathMatcher {
val pathTrailing = if (path.endsWith("/")) path else s"$path/"
pathTrailing.startsWith(baseTrailing) && (withExactBase || pathTrailing.substring(baseTrailing.length).nonEmpty)
}

def withPassing[T](map: String => T)(path: String): Option[T] = if (matches(path)) {
if (withExactBase) {
Some(map(base))
} else {
val baseTrailing = if (base.endsWith("/")) base else s"$base/"
val next = path.indexOf("/", baseTrailing.length)
if (next < 0) {
Some(map(path))
} else Some(map(path.substring(0, next)))
}
} else None
}

private[core] final case class Exact(base: String) extends PathMatcher {

protected val priority = 2

def matches(path: String): Boolean = path == base || s"$path/" == base

def withPassing[T](map: String => T)(path: String): Option[T] = if (matches(path)) {
Some(map(path))
} else None
}

private[core] final case class SingleVariable(base: String) extends PathMatcher {
Expand All @@ -84,15 +59,6 @@ object PathMatcher {
val trailingSlash = if (base.endsWith("/")) base else s"$base/"
path.startsWith(trailingSlash) && !path.substring(trailingSlash.length).contains('/')
}

def withPassing[T](map: String => T)(path: String): Option[T] = if (matches(path)) {
val baseTrailing = if (base.endsWith("/")) base else s"$base/"
val next = path.indexOf("/", baseTrailing.length)
if (next < 0) {
Some(map(path))
} else Some(map(path.substring(0, next)))
} else None

}

sealed trait Error {
Expand Down
70 changes: 70 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
sidebar_position: 3
---

# Configuration

There are multiple ways to configure the Agent, including system properties, environment variables, and providing the configuration file.

To provide the configuration file to the agent you can either `-Dotel.javaagent.configuration-file=<path-to-file>` or `OTEL_JAVAAGENT_CONFIGURATION_FILE` environment varaible.

## OTEL namespace options

You can find the full list of available configuration options [here](https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/autoconfigure/README.md)

## Mesmer options

### Akka actors grouping
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lgajowy I described the actual behaviour of the current implementation as is, but the implementation it's what bothers me. For most paths definitions there is no distinction between instance and group option behaviour. Similarly different rules can lead to the same results and it's quite confusing which definition is preferred.

I believed the current implementation is the result of incremental evolution of the requirements. I suggest, for some future version of Mesmer, to define and document the exact specification on path matching first and then rewrite the entire matching logic from scratch.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yeah. Couldn't frame that better. Thanks for that input!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#636

I created an issue for fixing that


When the actors' hierarchy grows or there are short-living actors referenced by a unique path name, it can be impractical to give each metric a unique set of attributes. This amplifies the amount of data exported and can affect collector performance. To solve this, Mesmer allows the definition of fine-grained rules on how metrics associated with different actors can be grouped together.

There are three actor attributes grouping options:
- **group** - the metrics, collected for all the actors matching the given path, share the `actor_path` attribute.
- **instance** - the metrics, collected for actors matching this path, have a unique `actor_path` attribute.
- **disabled** - the metrics, collected for all the actors matching the given path, do not get `actor_path` attribute.

By default the path grouping is disabled. You can change this by launching the Mesmer extension with `-Dio.scalac.mesmer.actor.reporting-default=group` configuration option. This will give all the actors' metrics the attribute `actor_path="/"`.

Alternatively, it is possible to override a single path grouping strategy as following
```
java -javaagent:path/to/opentelemetry-javaagent.jar \
-Dotel.javaagent.extensions=path/to/mesmer-otel-extension.jar \
-Dio.scalac.mesmer.actor.reporting-default=ignore \
-Dio.scalac.mesmer.actor.rules."/user/**"=group \
-Dio.scalac.mesmer.actor.rules."/system/**"=group \
-jar your-app.jar
```
This configuration will aggregate metrics for system and user hierarchies separately, grouping them by `actor_path="/system"` and `actor_path="/user"` attributes correspondingly.

The individual rule syntax is `io.scalac.mesmer.actor.rules."<MATCHING_PATH>"=<GROUPING_OPTION>`. The matching path supports limited set of wildcards, aiding to group metrics for variable path segments.

- **/** - matches exactly one actor and therefore can be used only with the `instance` grouping option.
For a topology of */user/my-actor* and */user/my-actor/1* and the rules
```
io.scalac.mesmer.actor.reporting-default=group
io.scalac.mesmer.actor.rules."/user/my-actor"=instance
```
the metrics from */user/my-actor* will get the attribute `actor_path="/user/my-actor"` while the metrics from */user/my-actor/1* will get the attribute `actor_path="/"` and likely will be grouped with metrics from other actors.

- **/\*** - matches a single variable tailing segment.
For a topology of */user/my-actor*, */user/my-actor/1*, */user/my-other-actor* and the rules
```
io.scalac.mesmer.actor.reporting-default=ignore
io.scalac.mesmer.actor.rules."/user/*"=instance
```
the metrics from */user/my-actor* and */user/my-other-actor* will get same values for `actor_path` attribute, while */user/my-actor/1* will be ignored.

- **/\*\*** - matches all tailing segments.
For a topology of */user/my-actor*, */user/my-actor/1*, */user/my-other-actor* and the rules
```
io.scalac.mesmer.actor.reporting-default=ignore
io.scalac.mesmer.actor.rules."/user/**"=instance
```
this will produce a unique metric for each actor with root at */user*. This wildcard also can be applied using with grouping,

```
io.scalac.mesmer.actor.reporting-default=ignore
io.scalac.mesmer.actor.rules."/user/**"=group
```

Resulting in all metrics from actors with root at */user* to be aggregated using common attribute `actor_path="/user"`
87 changes: 8 additions & 79 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,101 +16,30 @@ This guide presents steps for Prometheus and OTEL (native OpenTelemetry protocol

2. Download [mesmer-otel-extension.jar](https://github.com/ScalaConsultants/mesmer/releases/download/v0.8.0.RC1/mesmer-otel-extension.jar) from `mesmer` Releases.

3. Add the dependency on OpenTelemetry autoconfiguration extension to your `build.sbt` file:
```scala
libraryDependencies ++= Seq(
"io.opentelemetry" % "opentelemetry-sdk-extension-autoconfigure" % "1.13.0-alpha"
)
```

4. Add an exporter for your favorite protocol to your `build.sbt` file:

**For OTLP**
```scala
libraryDependencies ++= Seq(
"io.opentelemetry" % "opentelemetry-exporter-otlp" % "1.13.0",
)
```

**For Prometheus**
```scala
libraryDependencies ++= Seq(
"io.opentelemetry" % "opentelemetry-exporter-prometheus" % "1.13.0-alpha".
)
```

5. Run the application with the following options:
3. Run your application with OT Agent `-javaagent` and Mesmer extension `-Dotel.javaagent.extensions` attached.
```sh
-javaagent:path/to/opentelemetry-javaagent.jar
-Dotel.javaagent.extensions=mesmer-otel-extension.jar
# add the following option if you're using Prometheus exporter
-Dotel.metrics.exporter=prometheus
java -javaagent:path/to/opentelemetry-javaagent.jar \
-Dotel.javaagent.extensions=path/to/mesmer-otel-extension.jar \
-jar your-app.jar
```
Eg. if you're running the application with `sbt run` add this to `build.sbt` file:

If you starting your application using `sbt run` you also can update your `build.sbt` with the following settings:
```scala
run / fork := true
run / javaOptions ++= Seq(
"-javaagent:path/to/opentelemetry-javaagent.jar",
"-Dotel.javaagent.extensions=path/to/mesmer-otel-extension.jar",
// add the following option if you're using Prometheus exporter
"-Dotel.metrics.exporter=prometheus"
)
```
or if you're running your application as a `jar` from command line:
```sh
java \
-javaagent:path/to/opentelemetry-javaagent.jar \
-Dotel.javaagent.extensions=path/to/mesmer-otel-extension.jar \
# add the following option if you're using Prometheus exporter
-Dotel.metrics.exporter=prometheus \
-jar your-app.jar
```

6. Test it:

**For OTLP**

You need to have a running OTLP Collector for the metrics to be automatically streamed to it.
4. By default the agent configured to use OTLP exporter which pushes metrics to the [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) running at `http://localhost:4317`. See the collector documentation for the configuration options.

**For Prometheus**
Alternatively you can configure the agent to expose the metrics with Prometheus exporter `-Dotel.metrics.exporter=prometheus`. This will make metrics available for scrapping over HTTP at default port 9464.

Call the metrics endpoint:
```sh
curl -i http://localhost:9464
```

## Akka-specific setup

Add Mesmer Akka extension:

Add the following dependency to your `build.sbt` file:
```scala
libraryDependencies += "io.scalac" %% "mesmer-akka-extension" % "0.8.0.RC1"
```

Add this entry to your `application.conf`:
```
akka.actor.typed.extensions = ["io.scalac.mesmer.extension.AkkaMonitoring"]
```

## ZIO-specific setup

Enable all available ZIO metrics by adding the following layers to your program:
- `Runtime.enableRuntimeMetrics`
- `DefaultJvmMetrics.live.unit`

Eg.
```scala
myProgram.provide(
Runtime.enableRuntimeMetrics,
DefaultJvmMetrics.live.unit,
)
```

For full reference see this ZIO 2.0 SampleApp code:

https://github.com/zio/zio-metrics-connectors/blob/zio/series2.x/core/jvm/src/test/scala/zio/metrics/connectors/SampleApp.scala#L15-L71

**Important for v0.8.0.RC1**

At this moment (v0.8.0.RC1) Mesmer is closely bound with Akka. This is something we're working on, but until then there's a specific step that needs to be made in non-Akka applications. The application needs to be run with these additional parameters (that will turn off Akka instrumentation):
Expand Down
2 changes: 1 addition & 1 deletion docs/supported-metrics.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 3
sidebar_position: 4
---

# Supported metrics
Expand Down