Skip to content
This repository has been archived by the owner on Oct 23, 2024. It is now read-only.

Latest commit

 

History

History
224 lines (183 loc) · 14.6 KB

collectd-mongodb.md

File metadata and controls

224 lines (183 loc) · 14.6 KB

collectd/mongodb

Monitor Type: collectd/mongodb (Source)

Accepts Endpoints: Yes

Multiple Instances Allowed: Yes

Overview

Monitors an instance of MongoDB using the collectd MongoDB Python plugin. Requires MongoDB 2.6 or later.

This monitor captures the following metrics about MongoDB generally:

  • memory
  • network input/output bytes count
  • heap usage
  • db connections
  • operations count
  • active client connections
  • queued operations

The plugin also captures the following DB-specific metrics:

  • db size
  • db counters

Documentation for MongoDB can be found here.

Creating a MongoDB user for collectd

If you're monitoring a secured MongoDB deployment, it is a good practice to create a MongoDB user with minimal read-only roles, as follows:

If the monitor is configured to send collection level metrics, you need to create and add this role so your user has permission to run collStats and indexStats against system collections and any collection in the db you're monitoring. In the below role, we're adding the appropriate permissions for system collections under the admin db. For more details on how to add all your non-admin collections, check https://www.mongodb.com/docs/manual/reference/resource-document/

use admin
db.createRole( 
  {
    role: "splunkMonitor",
    privileges: [ { resource: { db: 'admin', collection: 'system.roles' }, actions: [ 'collStats', 'indexStats' ] },
                  { resource: { db: 'admin', collection: 'system.users' }, actions: [ 'collStats', 'indexStats' ] },
                  { resource: { db: 'admin', collection: 'system.version' }, actions: [ 'collStats', 'indexStats' ] } ],
    roles: [ { role: "read", db: "admin" } ]
  },
  { w: "majority" , wtimeout: 5000 }
)

use admin
db.createUser( {
  user: "collectd",
  pwd: "collectd",
  roles: [ { role: "readAnyDatabase", db: "admin" }, { role: "clusterMonitor", db: "admin" }, {role: "splunkMonitor", db: "admin"} ]
});

Configuration

To activate this monitor in the Smart Agent, add the following to your agent config:

monitors:  # All monitor config goes under this key
 - type: collectd/mongodb
   ...  # Additional config

For a list of monitor options that are common to all monitors, see Common Configuration.

Config option Required Type Description
pythonBinary no string Path to a python binary that should be used to execute the Python code. If not set, a built-in runtime will be used. Can include arguments to the binary as well.
host yes string Host name/IP address of the Mongo instance
port yes integer Port of the Mongo instance (default: 27017)
databases yes list of strings Name(s) of database(s) that you would like metrics from. Note: the first database in this list must be "admin", as it is used to perform a serverStatus() command.
username no string The MongoDB user to connect as
password no string The password of the above user
useTLS no bool If true, will connect to Mongo using TLS (default: false)
caCerts no string Path to a CA cert that will be used to verify the certificate that Mongo presents (not needed if not using TLS or if Mongo's cert is signed by a globally trusted issuer already installed in the default location on your OS)
tlsClientCert no string Path to a client certificate (not needed unless your Mongo instance requires x509 client verification)
tlsClientKey no string Path to a client certificate key (not needed unless your Mongo instance requires x509 client verification, or if your client cert above has the key included)
tlsClientKeyPassPhrase no string Passphrase for the TLSClientKey above
sendCollectionMetrics no bool Whether to send collection level metrics or not (default: false)
sendCollectionTopMetrics no bool Whether to send collection level top (timing) metrics or not (default: false)

Metrics

These are the metrics available for this monitor. Metrics that are categorized as container/host (default) are in bold and italics in the list below.

  • counter.asserts.regular (cumulative)
    The number of regular assertions raised since the MongoDB process started. Check the log file for more information about these messages.

  • counter.asserts.warning (cumulative)
    In MongoDB 3.x and earlier, the field returns the number of warnings raised since the MongoDB process started. In MongodDB 4, this is always 0.

  • counter.backgroundFlushing.flushes (gauge)
    Number of times the database has been flushed. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)

  • counter.extra_info.page_faults (gauge)
    Mongod page faults

  • counter.lock.Database.acquireCount.intentExclusive (cumulative)

  • counter.lock.Database.acquireCount.intentShared (cumulative)

  • counter.lock.Database.acquireCount.read (cumulative)

  • counter.lock.Database.acquireCount.write (cumulative)

  • counter.lock.Global.acquireCount.intentExclusive (cumulative)

  • counter.lock.Global.acquireCount.intentShared (cumulative)

  • counter.lock.Global.acquireCount.write (cumulative)

  • counter.network.bytesIn (gauge)
    Network bytes received by the database server

  • counter.network.bytesOut (gauge)
    Network bytes sent by the database server

  • counter.network.numRequests (cumulative)
    Requests received by the server

  • counter.opcounters.command (cumulative)
    Number of commands since last restart

  • counter.opcounters.delete (cumulative)
    Number of deletes since last restart

  • counter.opcounters.getmore (cumulative)
    Number of getmore operations since last restart

  • counter.opcounters.insert (cumulative)
    Number of inserts since last restart

  • counter.opcounters.query (cumulative)
    Number of queries since last restart

  • counter.opcounters.update (cumulative)
    Number of updates since last restart

  • counter.opcountersRepl.command (cumulative)
    Number of replicated commands since last restart

  • counter.opcountersRepl.delete (cumulative)
    Number of replicated deletes since last restart

  • counter.opcountersRepl.getmore (cumulative)
    Number of replicated getmore operations since last restart

  • counter.opcountersRepl.insert (cumulative)
    Number of replicated inserts since last restart

  • counter.opcountersRepl.query (cumulative)
    Number of replicated queries since last restart

  • counter.opcountersRepl.update (cumulative)
    Number of replicated updates since last restart

  • gauge.backgroundFlushing.average_ms (gauge)
    Average time (ms) to write data to disk. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)

  • gauge.backgroundFlushing.last_ms (gauge)
    Most recent time (ms) spent writing data to disk. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)

  • gauge.collection.max (gauge)
    Maximum number of documents in a capped collection

  • gauge.collection.maxSize (gauge)
    Maximum disk usage of a capped collection

  • gauge.collections (gauge)
    Number of collections

  • gauge.connections.available (gauge)
    The number of unused incoming connections available. Consider this value in combination with the value of gauge.connections.current to understand the connection load on the database.

  • gauge.connections.current (gauge)
    The number of incoming connections from clients to the database server.

  • gauge.connections.totalCreated (cumulative)
    Count of all incoming connections created to the server. This number includes connections that have since closed.

  • gauge.dataSize (gauge)
    Total size of data, in bytes

  • gauge.extra_info.heap_usage_bytes (gauge)
    Heap size used by the mongod process, in bytes. Deprecated in mongo version > 3.3, use gauge.tcmalloc.generic.heap_size instead.

  • gauge.globalLock.activeClients.readers (gauge)
    Number of active client connections performing reads

  • gauge.globalLock.activeClients.total (gauge)
    Total number of active client connections

  • gauge.globalLock.activeClients.writers (gauge)
    Number of active client connections performing writes

  • gauge.globalLock.currentQueue.readers (gauge)
    Read operations currently in queue

  • gauge.globalLock.currentQueue.total (gauge)
    Total operations currently in queue

  • gauge.globalLock.currentQueue.writers (gauge)
    Write operations currently in queue

  • gauge.indexSize (gauge)
    Total size of indexes, in bytes

  • gauge.indexes (gauge)
    Number of indexes across all collections

  • gauge.mem.mapped (gauge)
    Mongodb mapped memory usage, in MB. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)

  • gauge.mem.resident (gauge)
    Mongodb resident memory usage, in MB

  • gauge.mem.virtual (gauge)
    Mongodb virtual memory usage, in MB

  • gauge.numExtents (gauge)

  • gauge.objects (gauge)
    Number of documents across all collections

  • gauge.repl.active_nodes (gauge)
    Number of healthy members in a replicaset (reporting 1 for health).

  • gauge.repl.is_primary_node (gauge)
    Report 1 when member state of replicaset is PRIMARY and 2 else.

  • gauge.repl.max_lag (gauge)
    Replica lag in seconds calculated from the difference between the timestamp of the last oplog entry of primary and secondary see mongo doc.

  • gauge.storageSize (gauge)
    Total bytes allocated to collections for document storage

  • gauge.tcmalloc.generic.heap_size (gauge)
    Heap size used by the mongod process, in bytes. Same as gauge.extra_info.heap_usage_bytes but supports 64-bit values.

  • gauge.uptime (counter)
    Uptime of this server in milliseconds

Group collection

All of the following metrics are part of the collection metric group. All of the non-default metrics below can be turned on by adding collection to the monitor config option extraGroups:

  • counter.collection.index.accesses.ops (counter)
    Number of times an index has been used (only on Mongo 3.2+)
  • gauge.collection.avgObjSize (gauge)
    Mean object/document size of a collection
  • gauge.collection.count (gauge)
    Number of objects/documents in a collection
  • gauge.collection.indexSize (gauge)
    Size of a particular index on a collection
  • gauge.collection.size (gauge)
    Size of a collection in bytes, not including indexes
  • gauge.collection.storageSize (gauge)
    Size of the collection on disk in bytes, never decreases.

Group collection-top

All of the following metrics are part of the collection-top metric group. All of the non-default metrics below can be turned on by adding collection-top to the monitor config option extraGroups:

  • counter.collection.commandsCount (counter)
    Number of commands issued for a collection
  • counter.collection.commandsTime (counter)
    Time spent in microseconds processing commands issued for a collection
  • counter.collection.getmoreCount (counter)
    Number of getMore requests issued for a collection
  • counter.collection.getmoreTime (counter)
    Time spent in microseconds processing getMore requests for a collection
  • counter.collection.insertCount (counter)
    Number of inserts issued for a collection
  • counter.collection.insertTime (counter)
    Time spent in microseconds processing insert requests for a collection
  • counter.collection.queriesCount (counter)
    Number of queries issued for a collection
  • counter.collection.queriesTime (counter)
    Time spent in microseconds processing query requests for a collection
  • counter.collection.readLockCount (counter)
    Number of read locks issued for a collection
  • counter.collection.readLockTime (counter)
    Time spent in microseconds processing read locks for a collection
  • counter.collection.removeCount (counter)
    Number of remove requests issued for a collection
  • counter.collection.removeTime (counter)
    Time spent in microseconds processing remove requests for a collection
  • counter.collection.totalCount (counter)
    Total number of operations issued for a collection
  • counter.collection.totalTime (counter)
    Time spent in microseconds processing all operations for a collection
  • counter.collection.updateCount (counter)
    Number of update requests issued for a collection
  • counter.collection.updateTime (counter)
    Time spent in microseconds processing update requests for a collection
  • counter.collection.writeLockCount (counter)
    Number of write locks issued for a collection
  • counter.collection.writeLockTime (counter)
    Time spent in microseconds processing write locks for a collection

Non-default metrics (version 4.7.0+)

To emit metrics that are not default, you can add those metrics in the generic monitor-level extraMetrics config option. Metrics that are derived from specific configuration options that do not appear in the above list of metrics do not need to be added to extraMetrics.

To see a list of metrics that will be emitted you can run agent-status monitors after configuring this monitor in a running agent instance.

Dimensions

The following dimensions may occur on metrics emitted by this monitor. Some dimensions may be specific to certain metrics.

Name Description
plugin_instance Port number of the MongoDB instance