Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logstash parsers for cmsweb services #76

Open
vkuznet opened this issue Oct 14, 2019 · 0 comments
Open

Add logstash parsers for cmsweb services #76

vkuznet opened this issue Oct 14, 2019 · 0 comments

Comments

@vkuznet
Copy link
Collaborator

vkuznet commented Oct 14, 2019

This the placeholder ticket to add cmsweb service specific parsers to logstash. For simplicity let's collect all messages collected by filebeats:

  • workqueue
  • reqmon
"message":"[14/Oct/2019:16:39:19] reqmon-df69c6598-tk58z 127.0.0.1 \"GET /wmstatsserver/data/info HTTP/1.1\" 200 OK [data: 346 in 39 out 1982 us ] [auth: OK \"\" \"\" ] [ref: \"\" \"ServerMonitor-reqmon\" ]"
  • reqmgr2
[15/Oct/2019:02:00:39] reqmgr2-89f7df4fd-95g9q 137.138.33.200 "GET /reqmgr2/data/request?name=amaltaro_TaskChain_InclParents_Oct2019_Val_191010_125845_9547 HTTP/1.1" 200 OK [data: 2369 in 1862 out 530492 us ] [auth: OK "/DC=ch/DC=cern/OU=computers/CN=wmagent/vocms0192.cern.ch" "" ] [ref: "" "WMCore.Services.Requests/v002" ]
  • reqmgr2ms
"message":"[14/Oct/2019:16:40:37] reqmgr2ms-6796895484-7vzj6 127.0.0.1 \"GET /ms-transferor/data/status HTTP/1.1\" 200 OK [data: 351 in 158 out 683 us ] [auth: OK \"\" \"\" ] [ref: \"\" \"ServerMonitor-reqmgr2ms\" ]"
  • couchdb
"message":"[Mon, 14 Oct 2019 14:39:59 GMT] [info] [<0.26674.38>] 127.0.0.1 - - GET /workqueue/_design/WorkQueue/index.html 200"
  • phedex
"message":"::1 - - - [14/Oct/2019:16:44:48 +0200] \"GET /phedex/datasvc/doc HTTP/1.1\" 200 15229 \"-\" \"ServerMonitor-phedex\""
  • dbs
"message":"INFO:cherrypy.access:[14/Oct/2019:16:43:52] dbs-phys03-w-5fd9b6ffc-9kqxb 127.0.0.1 \"GET /dbs/ HTTP/1.1\" 200 OK [data: 298 in 468 out 1320 us ] [auth: OK \"\" \"\" ] [ref: \"\" \"ServerMonitor/2.0\" ]"
"message":"127.0.0.1 - - [14/Oct/2019:16:45:47] \"GET / HTTP/1.1\" 200 22 \"\" \"ServerMonitor/2.0\""
  • das
{"DASQuery":{"query":"dataset=/TTToSemiLeptonic_TuneCP5_13TeV-powheg-pythia8/RunIIFall17MiniAODv2-PU2017_12Apr2018_94X_mc2017_realistic_v14-v2/MINIAODSIM","hash":"705c64bd8a82ee31aa9607dada4a6208","spec":{"dataset":"/TTToSemiLeptonic_TuneCP5_13TeV-powheg-pythia8/RunIIFall17MiniAODv2-PU2017_12Apr2018_94X_mc2017_realistic_v14-v2/MINIAODSIM"},"fields":["dataset"],"pipe":"","instance":"prod/global","detail":false,"system":"","filters":{},"aggregators":[],"error":"","tstamp":1570577955},"PID":"705c64bd8a82ee31aa9607dada4a6208","ProcessTime":10.645284837,"Unix":1570577956,"level":"info","msg":"ready","time":"2019-10-09T01:39:16+02:00"}
  • crabserver
"message":"[14/Oct/2019:16:45:22]  RESTSQL:ovUDzLxBPpVb RELEASED cmsweb_analysis_preprod@devdb11 timeout=300 inuse=0 idle=1"
  • crabcache
"message":"[14/Oct/2019:16:46:21] crabcache-6c7f6559d6-5ksbz 127.0.0.1 \"GET /crabcache/info HTTP/1.1\" 200 OK [data: 340 in 69 out 608 us ] [auth: OK \"\" \"\" ] [ref: \"\" \"ServerMonitor-crabcache\" ]"
  • dqmgui
  • t0wmadatasvc
"message":"[14/Oct/2019:16:44:21] t0wmadatasvc-74d87c769b-p8zcx 127.0.0.1 \"GET /t0wmadatasvc/prod/hello HTTP/1.1\" 200 OK [data: 352 in 25 out 64701 us ] [auth: OK \"\" \"\" ] [ref: \"\" \"ServerMonitor-t0wmadatasvc\" ]"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant