Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(K8s): Added the configuration-guided step to the installation K8… #18479

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ Prometheus OpenMetrics, `nri-prometheus`, transformations are now replaced with

## Self-instrumentation [#self-instrumentation]

Similar to Prometheus OpenMetrics (`nri-prometheus`), the Prometheus agent (`newrelic-prometheus-agent`) self-scrapes internal metrics. These metrics have the `prometheus_` prefix and you can use them to observe the status of the Prometheus instance. By default, only a reduced set of these metrics are sent. You can find the list of metrics on the [default values.yaml file](https://github.com/newrelic/newrelic-prometheus-configurator/blob/main/charts/newrelic-prometheus-agent/values.yaml) of the chart, by searching for `job_name: self-metrics`.
Similar to Prometheus OpenMetrics (`nri-prometheus`), the Prometheus agent (`newrelic-prometheus-agent`) self-scrapes internal metrics. These metrics have the `prometheus_` prefix and you can use them to observe the status of the Prometheus instance. By default, only a reduced set of these metrics are sent. You can find the list of metrics on the [default `values.yaml` file](https://github.com/newrelic/newrelic-prometheus-configurator/blob/main/charts/newrelic-prometheus-agent/values.yaml) of the chart, by searching for `job_name: self-metrics`.

We also give you a curated dashboard for self-metrics with performance and health information, and also metrics volume sent. Install the quickstart containing the [dashboard for the Prometheus agent](https://newrelic.com/instant-observability/prometheus-agent) in your New Relic account.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ By default, Pixie queries withpixie.ai to check if new updates have been pushed
id=""
title="newrelic-manifest.yaml option"
>
in your newrelic-manifest.yaml file under the pl-cluster-config section, add `PL_DISABLE_AUTO_UPDATE: "true"` to the `data` directive. Example:
In your `newrelic-manifest.yaml` file under the pl-cluster-config section, add `PL_DISABLE_AUTO_UPDATE: "true"` to the `data` directive. Example:

```
---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ helm upgrade --install newrelic-bundle newrelic/nri-bundle \
 --set global.fedramp.enabled=true
```

If using a values.yaml file for your installation, you would enable the same parameter.
If using a `values.yaml` file for your installation, you would enable the same parameter.

```yaml
...
Expand All @@ -43,7 +43,7 @@ global:
...
```

The full `nri-bundle` values.yaml file example can be found [here](https://github.com/newrelic/helm-charts/blob/master/charts/nri-bundle/values.yaml).
The full `nri-bundle` `values.yaml` file example can be found [here](https://github.com/newrelic/helm-charts/blob/master/charts/nri-bundle/values.yaml).

### Manifest [#manifest]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ To use this integration, you should:

Create a Yaml file that defines your values. This file defines the different subcharts that Helm will install and then assigns values to them.

An example of a yaml file that you could name `values.yaml` is this:
An example of a YAML file that you could name `values.yaml` is this:

```yaml
global:
Expand Down Expand Up @@ -81,7 +81,7 @@ To use this integration, you should:

Here are some important things to keep in mind:

* Other subcharts may exist in `nri-bundle` that are not represented in this file. If you'd like to add those subcharts, then add their key or value pairs to your yaml file.
* Other subcharts may exist in `nri-bundle` that are not represented in this file. If you'd like to add those subcharts, then add their key or value pairs to your YAML file.
nbaenam marked this conversation as resolved.
Show resolved Hide resolved
* The `newrelic/nri-bundle` chart contains the latest updates for Linux-based nodes.
* You can also configure the `newrelic-logging` chart during this step to install to both Linux and Windows nodes.
* You must configure a `NodeSelector` to avoid scheduling to Windows. If an installation fails on a mixed cluster using default values from guided install, add the `nodeSelector` values.
Expand All @@ -102,7 +102,7 @@ To use this integration, you should:
<Step>
### Define `newrelic-infrastructure` for Windows nodes [#newrelic-infra]

Create a second yaml file to add values for the `newrelic-infrastructure` v2 chart. This is a separate yaml file from the one you created in step one. This is an example yaml file for `newrelic-infrastructure` that you could name `windows-values.yaml`:
Create a second YAML file to add values for the `newrelic-infrastructure` v2 chart. This is a separate YAML file from the one you created in step one. This is an example YAML file for `newrelic-infrastructure` that you could name `windows-values.yaml`:

```yaml
global:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Let's explain each option.

If you're looking to cut costs, you can modify the `global.lowDataMode` parameter value. This will cut down on the number of metrics you send. The modification of this parameter will depend on the method you used to install Kubernetes:

* **Guided install** (New Relic CLI, Helm, manifest). The installation command includes `lowDataMode` by default because it has these parameters: `NRI_CLI_LOW_DATA_MODE`, `NRI_CLI_PROMETHEUS_AGENT_LOW_DATA_MODE`, and `NRI_CLI_LOGGING_LOW_DATA_MODE`.
* **Guided install** (New Relic CLI, Helm, manifest). The installation command includes `lowDataMode` by default because it has these parameters: `NR_CLI_LOW_DATA_MODE`, `NR_CLI_PROMETHEUS_AGENT_LOW_DATA_MODE`, and `NR_CLI_LOGGING_LOW_DATA_MODE`.

* **Helm**. Set the `global.lowDataMode` parameter in the [`nri-bundle` chart](https://github.com/newrelic/helm-charts/tree/master/charts/nri-bundle) to true.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,15 +73,15 @@ To upgrade, follow the [installation instructions](#fluentbit-plugin) or grab th

## Install Fluent Bit output plugin [#fluent-bit]

New Relic has a [Fluent Bit](https://fluentbit.io/) [output plugin](https://github.com/newrelic/newrelic-fluent-bit-output) to forward your logs to New Relic log management. This plugin is also provided in a standalone Docker image that can be installed in a Kubernetes cluster in the form of a DaemonSet, also known as the Kubernetes plugin.
New Relic has a [[Fluent Bit output plugin](https://github.com/newrelic/newrelic-fluent-bit-output) to forward your logs to New Relic log management. This plugin is also provided in a standalone Docker image that can be installed in a Kubernetes cluster in the form of a DaemonSet, also known as the Kubernetes plugin.
nbaenam marked this conversation as resolved.
Show resolved Hide resolved

You can install it in your cluster using our Helm chart in two ways.

### Use our guided install [#helm-guided-install]

Although the [newrelic-logging](https://github.com/newrelic/helm-charts/tree/master/charts/newrelic-logging) chart works as a standalone, we recommend installing it as part of the [nri-bundle](https://github.com/newrelic/helm-charts/tree/master/charts/nri-bundle) chart.
Although the [newrelic-logging](https://github.com/newrelic/helm-charts/tree/master/charts/newrelic-logging) chart works as a standalone, we recommend installing it as part of the [`nri-bundle`](https://github.com/newrelic/helm-charts/tree/master/charts/nri-bundle) chart.

The best way to install this is through our [guided installation](/docs/kubernetes-pixie/kubernetes-integration/installation/kubernetes-integration-install-configure/) process. This guided install can generate the Helm 3 commands required to install it ([see "Helm 3"](/docs/kubernetes-pixie/kubernetes-integration/installation/kubernetes-integration-install-configure/#finish-your-install)).
The best way to install this is through our [guided installation](/docs/kubernetes-pixie/kubernetes-integration/installation/kubernetes-integration-install-configure/) process. This guided install can generate the Helm 3 commands required to install it ([see `Helm 3`](/docs/kubernetes-pixie/kubernetes-integration/installation/kubernetes-integration-install-configure/#finish-your-install)).

### Manual installation [#manual-helm-install]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,44 +18,11 @@ freshnessValidatedDate: never

New Relic's Kubernetes plugin for log forwarding simplifies sending logs from your cluster to New Relic logs. It uses a standalone Docker image and runs as a DaemonSet, seamlessly collecting logs for centralized analysis and troubleshooting. Forwarding your Kubernetes logs to New Relic will give you enhanced <InlinePopover type="logs" /> capabilities to collect, process, explore, query, and alert on your log data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nbaenam We say there's a Kuberentes plugin for log forwarding, but we also say there's a Fluent Bit output plugin. It seems like these are two different names for the same thing, and is "plugin" the best term? Don't Kubenerntes plugins modify the Kubernetes cluster in some way? This Flurent Bit container is just forwarding logs, right?

I wonder if we should just drop "plugin" from the title and switch to something like this: "Forward Kubernetes log." If you did that, you'd have to rearrange the introduction and the Fluent Bit section.

Also, this doc repeats that confusing sentence: "New Relic has a Fluent Bit output plugin to forward your logs to New Relic log management. This plugin is also provided in a standalone Docker image that can be installed in a Kubernetes cluster in the form of a DaemonSet, also known as the Kubernetes plugin."

Again, aren't we just saying you can have two DaemonSets in your cluster?

  • One DaemonSet for the Kubernetes integration
  • Another DaemonSet for the Fluent Bit Docker container

Copy link
Contributor Author

@nbaenam nbaenam Sep 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @mangulonr, can you help us with the above and below comments, please? Thanks!


## Enable Kubernetes for log management [#enable-process]

To forward your Kubernetes logs to New Relic with our plugin:

1. [Install the New Relic Kubernetes integration](/install/kubernetes). This integration includes the Kubernetes plugin for logs.

2. Optionally, you can further tune your installation in [the guided install](/install/kubernetes) using the numerous configuration options available in the [`newrelic-logging` repository](https://github.com/newrelic/helm-charts/tree/master/charts/newrelic-logging#configuration). However, we recommend the standard setup, as it's valid for most users.

<img
title="Guided install from the UI"
alt="Guided install from the UI"
src="/images/kubernetes_screenshot-crop_gather-log-data.webp"
/>

<figcaption>
Go to <DNT>**[one.newrelic.com](https://one.newrelic.com/) > Integrations & Agents**</DNT> and click the <DNT>**Guided install**</DNT> tile. Then select <DNT>**Kubernetes**</DNT> and <DNT>**Guided**</DNT>.
</figcaption>

<Callout variant="important">

If you're [using a Kubernetes secret](https://github.com/newrelic/helm-charts/blob/master/charts/newrelic-logging/values.yaml#L8-L25) to store the New Relic <InlinePopover type="licenseKey" />, the `newrelic-logging` chart defaults to sending logs to the US API endpoint. If the license key belongs to an EU or FedRAMP account, and a secret is used for key storage, you must update the endpoint setting with the appropriate value from the [API reference docs](/docs/logs/log-api/introduction-log-api/#endpoint). Here's an example of how to set this for EU accounts:

```yml
newrelic-logging:
enabled: true
endpoint: https://log-api.eu.newrelic.com/log/v1
```
</Callout>

3. Generate some traffic and wait a few minutes, then [check your account](#find-data) for data.

<InstallFeedback />

## Fluent Bit output plugin [#fluent-bit]

New Relic has a [Fluent Bit](https://fluentbit.io/) [output plugin](https://github.com/newrelic/newrelic-fluent-bit-output) to forward your logs to New Relic log management. This plugin is also provided in a standalone Docker image that can be installed in a Kubernetes cluster in the form of a DaemonSet, also known as the Kubernetes plugin.
New Relic has a [Fluent Bit output plugin](https://github.com/newrelic/newrelic-fluent-bit-output) to forward your logs to New Relic log management. This plugin is also provided in a standalone Docker image that can be installed in a Kubernetes cluster in the form of a DaemonSet, also known as the Kubernetes plugin.

See [Fluent Bit plugin for log forwarding](/docs/logs/forward-logs/fluent-bit-plugin-log-forwarding) for more details about it.
See [Fluent Bit plugin for log forwarding](/docs/logs/forward-logs/fluent-bit-plugin-log-forwarding) doc for more details about it.

### Additional metric details [#addition-metric-details]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nbaenam is all this information about metric data just applicable to the Helm chart version of the New Relic Fluent Bit Docker Container?


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ In a previous procedure, you [created alerts](/docs/new-relic-solutions/build-nr
<Step>
Here, copy `alert-policies/example-alert-policy` directory and its content to `alert-policies` directory at the root level. Rename the directory as `flashdb`.

This directory contains sample YAML files for static and baseline alerts to contribute the corresponding alerts to New Relic I/O. To help you populate your yaml files, you can use New Relic's [NerdGraph API explorer](https://api.newrelic.com/graphiql) to get a JSON representation of each alert condition.
This directory contains sample YAML files for static and baseline alerts to contribute the corresponding alerts to New Relic I/O. To help you populate your YAML files, you can use New Relic's [NerdGraph API explorer](https://api.newrelic.com/graphiql) to get a JSON representation of each alert condition.

<Callout variant="tip">
NerdGraph is New Relic's GraphQL API.
Expand Down
7 changes: 7 additions & 0 deletions src/install/config/kubernetes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,13 @@ steps:
- optionType: dropdown1
options:
- value: 'manifest'
- filePath: 'src/install/kubernetes/guided/configuration-guided.mdx'
nbaenam marked this conversation as resolved.
Show resolved Hide resolved
overrides:
- isConditionalStep: true
selectedOptions:
- optionType: dropdown1
options:
- value: 'guided'
- filePath: 'src/install/kubernetes/helm/configuration-helm.mdx'
overrides:
- isConditionalStep: true
Expand Down
40 changes: 40 additions & 0 deletions src/install/kubernetes/guided/configuration-guided.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
componentType: default
headingText: Gather log data
nbaenam marked this conversation as resolved.
Show resolved Hide resolved
---

This step allows you to forward your Kubernetes logs to New Relic using our plugin. We recommend the default setup, as it's what most users will need.

<img
title="Guided install from the UI"
alt="Guided install from the UI"
src="/images/kubernetes_screenshot-crop_gather-log-data.webp"
/>

Select one of these options:

* **Forward all logs with full enrichment**: It associates all Kubernetes metadata with each log record. This option will increase your ingested data and your cost.

* **Forward all logs with minimal enrichment (low data mode)**: It sets the `NR_CLI_LOW_DATA_MODE` to true. This is the default option for reducing the amount of data ingested. See [Reduce data ingested by setting `global.lowDataMode`](/docs/kubernetes-pixie/kubernetes-integration/installation/reduce-ingest/#reduce-data-ingest) for more info.

<Callout variant="important">

If you're [using a Kubernetes secret](https://github.com/newrelic/helm-charts/blob/master/charts/newrelic-logging/values.yaml#L8-L25) to store the New Relic <InlinePopover type="licenseKey" />, the `newrelic-logging` chart defaults to sending logs to the US API endpoint. If the license key belongs to an EU or FedRAMP account, and a secret is used for key storage, you must update the endpoint setting with the appropriate value from the [API reference docs](/docs/logs/log-api/introduction-log-api/#endpoint). Here's an example of how to set this for EU accounts:

```yml
newrelic-logging:
enabled: true
endpoint: https://log-api.eu.newrelic.com/log/v1
```
</Callout>

At the end, you'll have the command that you need to copy and paste into a terminal.


<img
title="Guided install from the UI"
alt="Guided install from the UI"
src="/images/kubernetes_screenshot-crop_guided-install-command.webp"
/>


13 changes: 2 additions & 11 deletions src/install/kubernetes/guided/install-guided.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Choose one of these instrumentation methods:

* **Guided**: It automates your installation by running a simple script.
* **Helm**: A Helm chart values file with pre-populated required parameters.
* **Manifest**: A manifest yaml file with pre-populated required parameters.
* **Manifest**: A manifest YAML file with pre-populated required parameters.

<img
width="60%;"
Expand All @@ -21,13 +21,4 @@ Choose one of these instrumentation methods:
/>


Whichever option you choose, you'll go through several steps. Fill in the fields that apply to you. Follow the instructions on each screen. At the end, you'll have the command that you need to copy and paste into a terminal.


<img
width="60%;"
title="Guided install from the UI"
alt="Guided install from the UI"
src="/images/kubernetes_screenshot-crop_guided-install-command.webp"
/>

Whichever option you choose, you'll go through several steps. Fill in the fields that apply to you. Follow the instructions on each screen.
2 changes: 1 addition & 1 deletion src/install/kubernetes/manifest/install-manifest.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ headingText: Install the Kubernetes integration

You can manually install Kubernetes integration from a simple manifest with pre-populated required values.

1. Create a file named yaml file, for example `newrelic-manifest.yaml`. Include the configuration you need. The file must follow this structure:
1. Create a YAML configuration file, for example `newrelic-manifest.yaml`. Include the configurations you need, and make sure it follows this structure:

```yaml
global:
Expand Down
Loading