Skip to content

A simple log-aggregator service written in Golang

Notifications You must be signed in to change notification settings

souvikhaldar/loggator

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

loggator

A simple yet fast, extensible and robust log aggregator platform written in golang.

Features

  1. Send logs in JSON to the endpoint and the logs will be carefully stored in database.
  2. Search for logs using various filters.
  3. Generate charts or graphs or visualize functional performance of the service.
  4. Raise alert for anomalous behaviours like too many ERROR logs, too many INFO logs for a particular endpoint signaling possible DDOS, etc.

Dependency

  1. https://github.com/golang-migrate/migrate

Workflow

  1. Send logs to the endpoint The log agent will parse the logs and send to the above endpoint in the following JSON format

    POST /logs/

    {
        "date": "2012-04-23",
        "time": "23:11:12",
        "log": "this is a error log",
        "log_level": "ERROR",
        "tenant_id": 2,
        "service_name": "loggator",
        "package_name": "logs",
        "file_name": "repository.go"
    }
    
  2. Fetch logs based on filters

    Request

    GET /logs?log_level=ERROR&service_name=loggator&package_name=logs&date=2012-04-23&file_name=repository.go&tenant_id=1

    Response

    [
       {
          "tenant_id":1,
          "log_id":4,
          "log":"this is a error log",
          "created_at":"2022-06-21T12:23:29.985752Z",
          "date":"2012-04-23T00:00:00Z",
          "time":"0000-01-01T23:11:12Z",
          "log_level":"ERROR",
          "service_name":"loggator",
          "file_name":"repository.go",
          "package_name":"logs"
       },
       {
          "tenant_id":1,
          "log_id":5,
          "log":"this is a error log",
          "created_at":"2022-06-21T12:29:22.952193Z",
          "date":"2012-04-23T00:00:00Z",
          "time":"0000-01-01T23:11:12Z",
          "log_level":"ERROR",
          "service_name":"loggator",
          "file_name":"repository.go",
          "package_name":"logs"
       },
       {
          "tenant_id":1,
          "log_id":6,
          "log":"this is a error log",
          "created_at":"2022-06-21T12:31:46.73274Z",
          "date":"2012-04-23T00:00:00Z",
          "time":"0000-01-01T23:11:12Z",
          "log_level":"ERROR",
          "service_name":"loggator",
          "file_name":"repository.go",
          "package_name":"logs"
       }
    ]
    

Instructions

  1. To migrate the database:
    i) Install psql
    ii) Access the database psql -U postgres.
    iii) Create the loggator database CREATE TABLE loggator;
    iv) Apply the schema make localmigrateup.

  2. To run the service i) Run go run cmd/server/main.go ii) Run the psql database on port 5432.

API documentation

Run in Postman

About

A simple log-aggregator service written in Golang

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published