Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added example on CRI parsing longlogs #242

Merged
merged 8 commits into from
Jul 10, 2023
29 changes: 29 additions & 0 deletions logs/fluentd/k8s-helm/http/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,32 @@ By default this chart installs additional dependent chart:
## Coralogix Fluentd Buffer Alert

In order to create an alert on Fluentd buffer in Coralogix, please see [coralogix-alert doc](https://github.com/coralogix/telemetry-shippers/blob/master/logs/fluentd/docs/coralogix-alerts.md)


## Log Logs: containerd / CRI partial logs

If your application is generating logs longer than 16k you should notice that docker dirver is splitting the log in multiple messages.
juan-coralogix marked this conversation as resolved.
Show resolved Hide resolved
To fix this we can use concat to fix this.
juan-coralogix marked this conversation as resolved.
Show resolved Hide resolved

First lets make sure that in the override file, that you use to deploy the helm, has logtag as one of the regex group keys, just like this.
juan-coralogix marked this conversation as resolved.
Show resolved Hide resolved

```yaml
<pattern>
format /^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$/
time_format %Y-%m-%dT%H:%M:%S.%L%z
keep_time_key true
</pattern>
```
If that is not the case please replace the existing one with this one.

Then next to the source we will add the following filter that will concat the logs:
juan-coralogix marked this conversation as resolved.
Show resolved Hide resolved

```yaml
<filter raw.containers.**>
@type concat
key message
use_partial_cri_logtag true
partial_cri_logtag_key logtag
partial_cri_stream_key stream
</filter>
```
Loading