Skip to content
This repository has been archived by the owner on May 6, 2024. It is now read-only.

Documented metrics-probe() function #110

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Documented metrics-probe() function #110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 142 additions & 0 deletions Content/Guides/syslog-ng-guide-admin/metrics-probe-options.htm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
<?xml version="1.0" encoding="utf-8"?>
<html xmlns:MadCap="http://www.madcapsoftware.com/Schemas/MadCap.xsd">
<head>
<link href="../../Resources/TableStyles/RuledTableWithHeading_VerticallyRuled_DoNotEdit.css" rel="stylesheet" MadCap:stylesheetType="table" />
<link href="../../Resources/TableStyles/NoteTable_Yellow_DoNotEdit.css" rel="stylesheet" MadCap:stylesheetType="table" />
<link href="../../Resources/TableStyles/RuledTableWithHeading_DoNotEdit.css" rel="stylesheet" MadCap:stylesheetType="table" />
<meta name="description" content="" />
</head>
<body name="metrics-probe-options">
<h1 name="metrics-probe-options"><a name="metrics-probe-options"></a>Options of metrics-probe</h1>
<div>
<h6 name="increment"><a name="increment"></a>increment()</h6>
<table cellspacing="0" class="TableStyle-RuledTableWithHeading_DoNotEdit" colsep="0" rowsep="0" style="width: 100%;mc-table-style: url('../../Resources/TableStyles/RuledTableWithHeading_DoNotEdit.css');">
<tbody>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyE-Column1-Body1">Type:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyD-Column1-Body1">integer or template</td>
</tr>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyB-Column1-Body1">Default:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyA-Column1-Body1"><span class="Code">1</span></td>
</tr>
</tbody>
<col class="TableStyle-RuledTableWithHeading_DoNotEdit-Column-Column1" style="width: 0.5in;">
</col>
</table>
<p>Available in <MadCap:variable name="General.abbrev"></MadCap:variable> 4.2 and later versions.</p>
<p>This option sets a template, which resolves to a number that defines the increment of the counter. The following example defines a counter called <span class="Code">syslogng_input_event_bytes_total</span>, and increases its value with the size of the incoming message (in bytes).</p>
<div class="Example">
<h6>Example:</h6>
<p><pre>
metrics-probe(
key("input_event_bytes_total")
labels(
"cluster" => "`cluster-name`"
"driver" => "kubernetes"
"id" => "${SOURCE}"
"namespace" => "${`prefix`namespace_name}"
"pod" => "${`prefix`pod_name}"
)
increment("${RAWMSG_SIZE}")
);
</pre>
</p>
</div>
</div>
<div>
<h6 name="key"><a name="key"></a>key()</h6>
<table cellspacing="0" class="TableStyle-RuledTableWithHeading_DoNotEdit" colsep="0" rowsep="0" style="width: 100%;mc-table-style: url('../../Resources/TableStyles/RuledTableWithHeading_DoNotEdit.css');">
<tbody>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyE-Column1-Body1">Type:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyD-Column1-Body1">string</td>
</tr>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyB-Column1-Body1">Default:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyA-Column1-Body1"><span class="Code">classified_events_total</span></td>
</tr>
</tbody>
<col class="TableStyle-RuledTableWithHeading_DoNotEdit-Column-Column1" style="width: 0.5in;">
</col>
</table>
<p>This option defines the name of the counter to create. Note that the value of this option is always prefixed with <span class="Code">syslogng_</span>, for example <span class="Code">key("my-custom-key")</span> becomes <span class="Code">syslogng_my-custom-key</span>.</p>
</div>
<div>
<h6 name="labels"><a name="labels"></a>labels()</h6>
<table cellspacing="0" class="TableStyle-RuledTableWithHeading_DoNotEdit" colsep="0" rowsep="0" style="width: 100%;mc-table-style: url('../../Resources/TableStyles/RuledTableWithHeading_DoNotEdit.css');">
<tbody>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyE-Column1-Body1">Type:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyD-Column1-Body1"></td>
</tr>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyB-Column1-Body1">Default:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyA-Column1-Body1">See the description</td>
</tr>
</tbody>
<col class="TableStyle-RuledTableWithHeading_DoNotEdit-Column-Column1" style="width: 0.5in;">
</col>
</table>
<p>This option defines the labels used to create separate counters, based on the fields of the messages processed by <span class="Code">metrics-probe()</span>. Use the following format:</p>
<p>
<pre>labels(
"name-of-the-label-in-the-output" => "field-of-the-message"
)</pre>
</p>
<p>Default value:</p>
<p>
<pre>labels(
"app" => "${APP}"
"host" => "${HOST}"
"program" => "${PROGRAM}"
"source" => "${SOURCE}"
)</pre>
</p>
<p>This configuration results the following counters:</p>
<p>
<pre>
syslogng_classified_events_total{app="example-app", host="localhost", program="baz", source="s_local_1"} 3
</pre>
</p>
<h6 name="dynamic-labels"><a name="dynamic-labels"></a>Dynamic labels</h6>
<p>Available in <MadCap:variable name="General.abbrev"></MadCap:variable> 4.4 and later versions.</p>
<p>Dynamic labelling makes it possible to use every available <span class="Code">value-pairs()</span> option in the labels, for example, <span class="Code">key()</span>, <span class="Code">rekey()</span>, <span class="Code">pair()</span>, or <span class="Code">scope()</span>.</p>
<div class="Example">
<h6>Example:</h6>
<p><pre>metrics-probe(
key("foo")
labels(
static-label" => "bar"
key(".my_prefix.*" rekey(shift-levels(1)))
)
);</pre>
</p>
<p><pre>syslogng_foo{static_label="bar",my_prefix_baz="anotherlabel",my_prefix_foo="bar",my_prefix_nested_axo="flow"} 4</pre>
</p>
</div>
</div>
<div>
<h6 name="level"><a name="level"></a>level()</h6>
<table cellspacing="0" class="TableStyle-RuledTableWithHeading_DoNotEdit" colsep="0" rowsep="0" style="width: 100%;mc-table-style: url('../../Resources/TableStyles/RuledTableWithHeading_DoNotEdit.css');">
<tbody>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyE-Column1-Body1">Type:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyD-Column1-Body1">integer (0-3)</td>
</tr>
<tr class="TableStyle-RuledTableWithHeading_DoNotEdit-Body-Body1">
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyB-Column1-Body1">Default:</td>
<td class="TableStyle-RuledTableWithHeading_DoNotEdit-BodyA-Column1-Body1">0</td>
</tr>
</tbody>
<col class="TableStyle-RuledTableWithHeading_DoNotEdit-Column-Column1" style="width: 0.5in;">
</col>
</table>
<p>Available in <MadCap:variable name="General.abbrev"></MadCap:variable> 4.2 or later versions.</p>
<p>This option sets the stats level of the generated metrics.</p>
<div class="Note">
<p class="Hyphenation"><span class="AllNoteStyles">NOTE: </span>Drivers configured with <span class="Code">internal(yes)</span> register their metrics on level 3. Due to this, if an SCL is created, the built-in metrics of the driver can be disabled, metrics can be created manually using <span class="Code">metrics-probe()</span>.</p>
</div>
</div>
</body>
</html>
67 changes: 67 additions & 0 deletions Content/Guides/syslog-ng-guide-admin/metrics-probe.htm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<?xml version="1.0" encoding="utf-8"?>
<html xmlns:MadCap="http://www.madcapsoftware.com/Schemas/MadCap.xsd">
<head>
<link href="../../Resources/TableStyles/RuledTableWithHeading_VerticallyRuled_DoNotEdit.css" rel="stylesheet" MadCap:stylesheetType="table" />
<link href="../../Resources/TableStyles/NoteTable_Yellow_DoNotEdit.css" rel="stylesheet" MadCap:stylesheetType="table" />
<link href="../../Resources/TableStyles/RuledTableWithHeading_DoNotEdit.css" rel="stylesheet" MadCap:stylesheetType="table" />
<meta name="description" content="" />
</head>
<body name="metrics-probe">
<h1 name="metrics-probe"><a name="metrics-probe"></a>metrics-probe</h1>

<p>Available in <MadCap:variable name="General.abbrev"></MadCap:variable> 4.1.1 or later versions.</p>
<p><span class="Code">metrics-probe()</span> is a special parser that counts the messages passing through the log path, and creates labeled stat-counters based on the fields of these messages.</p>
<p>The names of the keys and the labels can be configured. The keys are automatically prefixed with the <span class="Code">syslogng_</span> string. Templates can be used in the values of the labels.</p>
<p>The minimal configuration creates counters with the <span class="Code">syslogng_classified_events_total</span> key, labels app, host, program and source.</p>
<div class="Example">
<h6>Example:</h6>
<p><pre>parser p_metrics_probe {
metrics-probe();
# Same as:
#
# metrics-probe(
# key("classified_events_total")
# labels(
# "app" => "${APP}"
# "host" => "${HOST}"
# "program" => "${PROGRAM}"
# "source" => "${SOURCE}"
# )
# );
};</pre>
</p>
</div>
<p>This configuration results the following counters:</p>
<p>
<pre>syslogng_classified_events_total{app="example-app", host="localhost", program="baz", source="s_local_1"} 3
syslogng_classified_events_total{app="example-app", host="localhost", program="bar", source="s_local_1"} 1
syslogng_classified_events_total{app="example-app", host="localhost", program="foo", source="s_local_1"} 1
</pre>
</p>
<p>The metrics can be queried by running the following command:</p>
<p>
<pre>syslog-ng-ctl stats prometheus</pre>
</p>
<p>In the following example, the <span class="Code">metrics-probe()</span> parser creates a counter called <span class="Code">syslogng_custom_key</span> that counts messages that have their <span class="Code">custom_label_name_1</span> field set to <span class="Code">foobar</span>, and for these messages it creates separate counters based on the value of the <span class="Code">custom_label_name_2</span> field.</p>
<div class="Example">
<h6>Example:</h6>
<p><pre>parser p_metrics_probe {
metrics-probe(
key("custom_key") # adds "syslogng_" prefix => "syslogng_custom_key"
labels(
"custom_label_name_1" => "foobar"
"custom_label_name_2" => "${.custom.field}"
)
);
};</pre>
</p>
</div>
<p>This configuration results the following counters:</p>
<p>
<pre>syslogng_custom_key{custom_label_name_1="foobar", custom_label_name_2="bar"} 1
syslogng_custom_key{custom_label_name_1="foobar", custom_label_name_2="foo"} 1
syslogng_custom_key{custom_label_name_1="foobar", custom_label_name_2="baz"} 3</pre>
</p>
<p>In <MadCap:variable name="General.abbrev"></MadCap:variable> 4.4 and later versions, it is possible to create <MadCap:xref href="metrics-probe-options.htm#dynamic-labels"><span style="color: #04aada;" class="mcFormatColor">dynamic labels</span></MadCap:xref> as well.</p>
</body>
</html>
2 changes: 1 addition & 1 deletion Maketargets.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"condition": "General.OSE",
"conditions-to-exclude": "General.PE or General.pe6 or General.SPS or General.SRA or General.ARS",
"version": "3.38",
"techversion": "3.38"
"techversion": "3.38",
"onlinedocumentation": "https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition"
},
"syslog-ng-ose-guide-admin": {
Expand Down
7 changes: 7 additions & 0 deletions Project/TOCs/syslog-ng-ose-guide-admin.fltoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1322,6 +1322,13 @@
<TocEntry
Title="[%=System.LinkedTitle%]"
Link="/Content/Guides/syslog-ng-guide-admin/parser-mariadb-audit.htm"></TocEntry>
<TocEntry
Link="/Content/Guides/syslog-ng-guide-admin/metrics-probe.htm"
Title="[%=System.LinkedTitle%]">
<TocEntry
Link="/Content/Guides/syslog-ng-guide-admin/metrics-probe-options.htm"
Title="[%=System.LinkedTitle%]"></TocEntry>
</TocEntry>
<TocEntry
Title="[%=System.LinkedTitle%]"
Link="/Content/Guides/syslog-ng-guide-admin/parser-websense.htm"></TocEntry>
Expand Down