From f57d9aa2fac954406f6477338dbb85c7a086b82f Mon Sep 17 00:00:00 2001 From: Hiroshi Hatake Date: Mon, 2 Aug 2021 16:12:22 +0900 Subject: [PATCH 1/2] 1.0: Add metrics plugin descriptions Signed-off-by: Hiroshi Hatake --- SUMMARY.md | 2 ++ buffer/README.md | 3 ++- filter/README.md | 3 ++- formatter/README.md | 3 ++- input/README.md | 3 ++- metrics/README.md | 53 +++++++++++++++++++++++++++++++++++++ metrics/local.md | 29 ++++++++++++++++++++ output/README.md | 3 ++- parser/README.md | 3 ++- service_discovery/README.md | 3 ++- storage/README.md | 3 ++- 11 files changed, 100 insertions(+), 8 deletions(-) create mode 100644 metrics/README.md create mode 100644 metrics/local.md diff --git a/SUMMARY.md b/SUMMARY.md index ed1876f1..b7f046e7 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -122,6 +122,8 @@ * [static](service_discovery/static.md) * [file](service_discovery/file.md) * [srv](service_discovery/srv.md) +* [Metrics Plugins](metrics/README.md) + * [local](metrics/local.md) * [How-to Guides](how-to-guides/README.md) * [Stream Analytics with Materialize](how-to-guides/stream-analytics-with-materialize.md) * [Send Apache Logs to S3](how-to-guides/apache-to-s3.md) diff --git a/buffer/README.md b/buffer/README.md index 9a45d61f..b46fa687 100644 --- a/buffer/README.md +++ b/buffer/README.md @@ -1,6 +1,6 @@ # Buffer Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](../input/) * [Parser](../parser/) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](../storage/) * [Service Discovery](../service_discovery/) * [Buffer](./) +* [Metrics](../metrics/) This article gives an overview of Buffer Plugin. diff --git a/filter/README.md b/filter/README.md index 41d93f26..30a89a3d 100644 --- a/filter/README.md +++ b/filter/README.md @@ -1,6 +1,6 @@ # Filter Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](../input/) * [Parser](../parser/) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](../storage/) * [Service Discovery](../service_discovery/) * [Buffer](../buffer/) +* [Metrics](../metrics/) This article gives an overview of the Filter Plugin. diff --git a/formatter/README.md b/formatter/README.md index 58cd6ca7..04a5998e 100644 --- a/formatter/README.md +++ b/formatter/README.md @@ -1,6 +1,6 @@ # Formatter Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](../input/) * [Parser](../parser/) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](../storage/) * [Service Discovery](../service_discovery/) * [Buffer](../buffer/) +* [Metrics](../metrics/) This article gives an overview of the Formatter Plugin. diff --git a/input/README.md b/input/README.md index 516b8a5b..109f21b2 100644 --- a/input/README.md +++ b/input/README.md @@ -1,6 +1,6 @@ # Input Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](./) * [Parser](../parser/) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](../storage/) * [Service Discovery](../service_discovery/) * [Buffer](../buffer/) +* [Metrics](../metrics/) This article gives an overview of the Input Plugin. diff --git a/metrics/README.md b/metrics/README.md new file mode 100644 index 00000000..be7aded7 --- /dev/null +++ b/metrics/README.md @@ -0,0 +1,53 @@ +# Metrics Plugins + +Fluentd has nine \(9\) types of plugins: + +* [Input](../input/) +* [Parser](../parser/) +* [Filter](../filter/) +* [Output](../output/) +* [Formatter](../formatter/) +* [Storage](../storage) +* [Service Discovery](../service_discovery/) +* [Buffer](../buffer/) +* [Metrics](./) + +This article gives an overview of Metrics Plugin. + +## Overview + +Sometimes, the input/filter/output plugin needs to save its internal metrics in memory, influxdb or prometheus format ready in instances. Fluentd has a pluggable system called Metrics that lets a plugin store and reuse its internal state as metrics instances. + +## How To Use + +On Fluentd core, metrics plugin will handled on `` on `` to set up easily. + +Here is an example with `metrics_local`: + +```text + + + @type local + + +``` + +## List of Built-in Metrics Plugins + +* [`local`](local.md) + +## List of Base Plugin classes with Metrics support + +* `Fluent::Plugin::Input` for Input plugin base class +* `Fluent::Plugin::Output` for most of output plugin base class +* `Fluent::Plugin::Filter` for Filter plugin base class +* `Fluent::Plugin::MultiOutput` for [out_copy](../output/copy.md) plugin base class +* `Fluent::Plugin::BareOutput` for fluent-plugin-forest output plugin base class + +## List of 3rd party metrics plugins + +NOTE: This 3rd party metrics plugin list does not fully covers all of them. + +* [fluent-plugin-metrics-cmetrics](https://github.com/calyptia/fluent-plugin-metrics-cmetrics) + +If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation \(CNCF\)](https://cncf.io/). All components are available under the Apache 2 License. diff --git a/metrics/local.md b/metrics/local.md new file mode 100644 index 00000000..43170414 --- /dev/null +++ b/metrics/local.md @@ -0,0 +1,29 @@ +# local + +The `local` metrics plugin stores the values on memory. + +## Parameters + +### `default_labels` + +Specifies the default labels for the metrics. \(default: `{agent: "Fluentd", hostname: "#{Socket.gethostname}"}`\) + +### `labels` + +Specifies other custom labels for the metrics. \(default: `{}`\) + +## Example + +With this configuration: + +```text + + + @type local + + +``` + +The above configuration will save the internal metrics for plugins on memory. As a result, you can retrive metrics from memory and also you can replace with your custom metrics plugin. + +If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation \(CNCF\)](https://cncf.io/). All components are available under the Apache 2 License. diff --git a/output/README.md b/output/README.md index 88bd60a1..ec2fbb53 100644 --- a/output/README.md +++ b/output/README.md @@ -1,6 +1,6 @@ # Output Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](../input/) * [Parser](../parser/) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](../storage/) * [Service Discovery](../service_discovery/) * [Buffer](../buffer/) +* [Metrics](../metrics/) This article gives an overview of the Output Plugin. diff --git a/parser/README.md b/parser/README.md index 1158d50c..c331d60b 100644 --- a/parser/README.md +++ b/parser/README.md @@ -1,6 +1,6 @@ # Parser Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](../input/) * [Parser](./) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](../storage/) * [Service Discovery](../service_discovery/) * [Buffer](../buffer/) +* [Metrics](../metrics/) This article gives an overview of the Parser Plugin. diff --git a/service_discovery/README.md b/service_discovery/README.md index b3d147b6..bc493bfe 100644 --- a/service_discovery/README.md +++ b/service_discovery/README.md @@ -1,6 +1,6 @@ # Service Discovery Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](../input/) * [Parser](../parser/) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](../storage/) * [Service Discovery](./) * [Buffer](../buffer/) +* [Metrics](../metrics/) This article gives an overview of the Service Discovery Plugin. diff --git a/storage/README.md b/storage/README.md index a8c14e1f..9968cb05 100644 --- a/storage/README.md +++ b/storage/README.md @@ -1,6 +1,6 @@ # Storage Plugins -Fluentd has eight \(8\) types of plugins: +Fluentd has nine \(9\) types of plugins: * [Input](../input/) * [Parser](../parser/) @@ -10,6 +10,7 @@ Fluentd has eight \(8\) types of plugins: * [Storage](./) * [Service Discovery](../service_discovery/) * [Buffer](../buffer/) +* [Metrics](../metrics/) This article gives an overview of Storage Plugin. From db265fa15d5101cea7cb8b36d184eaa062d0afc9 Mon Sep 17 00:00:00 2001 From: Hiroshi Hatake Date: Fri, 6 Aug 2021 23:50:31 +0900 Subject: [PATCH 2/2] 1.0: metrics: Add more metrics local behavior descriptions Signed-off-by: Hiroshi Hatake --- metrics/README.md | 5 +++++ metrics/local.md | 3 +++ 2 files changed, 8 insertions(+) diff --git a/metrics/README.md b/metrics/README.md index be7aded7..40413cb8 100644 --- a/metrics/README.md +++ b/metrics/README.md @@ -32,6 +32,11 @@ Here is an example with `metrics_local`: ``` +`local` type plugin should provide equivalent behavior before Fluentd v1.13. +This metrics type should provide single numeric value storing functionality. + +And this `local` type plugin should be used by default. + ## List of Built-in Metrics Plugins * [`local`](local.md) diff --git a/metrics/local.md b/metrics/local.md index 43170414..fb7ae936 100644 --- a/metrics/local.md +++ b/metrics/local.md @@ -26,4 +26,7 @@ With this configuration: The above configuration will save the internal metrics for plugins on memory. As a result, you can retrive metrics from memory and also you can replace with your custom metrics plugin. +Actually, @type local metrics plugin has equivalent functionality for previous single value based Ruby instance variables. +This behavior will be changed by other 3rd party plugins. + If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation \(CNCF\)](https://cncf.io/). All components are available under the Apache 2 License.