Overview
Remediation Components Stats
This release introduces the ability for remediation components to provide metrics about their efficiency, such as the amount of blocked packets/bytes (fw bouncer) or http requests (web based bouncers). Those metrics can be seen in the console (via instance card) and cscli metrics show bouncers. Currently only firewall-bouncer (0.0.30) supports it, we are working on Nginx and HaProxy remediation components.
Instance & Remediation Components details
To help people managing larger setups, machines and remediation components now provide more information to the Local API, such as underlying OS. This shows in cscli machines list and cscli bouncers|machines inspect.
Kube/Docker improvments
Various bugfixes and QoL improvements have been made towards kube and docker support, notably to make AppSec usage easier in those environments.
Hub API
We replacing the existing Hub with an API and 1.6.3 is the first version to rely on it. The goal is to be able to introduce new item types easily (ie. acquisition file, notification templates etc.) as well as be able to merge some massive collections (such as sigma releases) that were held back.
New Features
Improvements
- Allow auto registration of machines in LAPI (#3202) @blotus
- allow .index.json to embed item content (#3145) @mmetc
- cscli hub update: option --with-content to keep embedded items in index; use it in docker (#3192) @mmetc
- implement GetFSType on openbsd with the correct statfs struct member (#3191) @rnagy
- pkg/cwhub: cache control / send etag header from file contents, check with HEAD req (#3187) @mmetc
- cscli: add option --ignore-missing to "bouncers delete", "machines delete" (#3177) @mmetc
- enhancement: Remove useragent set by RC (#3167) @LaurenceJJones
- command "cscli doc --target /path/to/dir" (#3169) @mmetc
- Allow QueryCAPIStatus to return as well enrollment status (#3159) @buixor
- Use the new hub api url (#3132) @sabban
- perf: retrieve unsorted metrics (#3148) @mmetc
- enhance: add crowdsec user agent to cti do request func (#3130) @LaurenceJJones
- enhance: Add default_range_remediation (#3109) @LaurenceJJones
- command cscli [machines|bouncers] inspect (#3103) @mmetc
- lapi detailed metrics: API spec + models (#3100) @mmetc
- lapi detailed metrics: db schema (#3099) @mmetc
- improved tls middleware revocation checks (#3034) @mmetc
- config: expand env variables in local_api_credentials.yaml and .yaml.local (#3093) @mmetc
- enhancement: add deprecation notice to cscli dashboard prerun (#3079) @LaurenceJJones
- enhancement: add other log levels to docker start script (#3072) @LaurenceJJones
- db,lapi: round durations and timestamps to 1 second (#3015) @mmetc
- update vagrant image for fc39, fc40 and ubuntu24.04 (#3042) @sabban
- db: mark immutable columns / remove unused (#3024) @mmetc
Changes
- bats: curl helpers to mock log processors and bouncers (#3141) @mmetc
- docker: symlink all data files to the staging area (#3120) @mmetc
- refact cscli metrics: fix lines between tables, skip wrapper api (#3137) @mmetc
- usage metrics: validate maxLength for some elements (#3131) @mmetc
- remove warning "maxopenconns is 0, default to 100" (#3129) @mmetc
- revert "db: round created, updated... timestamps to 1 second" (#3127) @mmetc
- cscli machines/bouncers: dry helper code and move to cscli (#3123) @mmetc
- func tests: update curl wrapper (#3121) @mmetc
- update vagrant config for opensuse (#3119) @mmetc
- make: remove redundant go version check (#3118) @mmetc
- refact cscli metrics: split stat types to own files (#3107) @mmetc
- cscli refact: extract metrics to own package (#3106) @mmetc
- cscli refact: extract table wrapper to own package (#3105) @mmetc
- db refact: drop column machines.status (#3102) @mmetc
- refactor: prefer logrus.WithField over WithFields with a single param (#3087) @mmetc
- cscli: refactor hubtests / extract methods run(), coverage() (#3086) @mmetc
- refactor pkg/parser: extract processGrok (#3080) @mmetc
- cscli: refactor "cscli setup" to avoid global variable (#3075) @mmetc
- refactor context (cscli, pkg/database) (#3071) @mmetc
- pkg/cwhub: use explicit context for item install, upgrade (#3067) @mmetc
- pkg/cwhub: use explicit context for version check, index update (#3064) @mmetc
- add: missing go reference badge (#3066) @mazzma12
- cscli: use cmp.Or from go 1.22 (#3058) @mmetc
- refactor "cscli decisions" (#3061) @mmetc
- refactor pkg/database, pkg/models (#3022) @mmetc
- tests: log.Fatal -> return err (#3056) @mmetc
- fix tests in 01_crowdsec.bats for packages (#3054) @sabban
- CI/bats: test remediation api via socket (#3052) @mmetc
- pkg/database: refactor & rename "soft delete" to "expire" (#3025) @mmetc
- cscli: log.Fatal -> return err (#3049) @mmetc
- cscli: refactor alerts/console/decisions/setup (#3003) @mmetc
- pkg/dumps.DumpTree: split to reduce complexity (#3001) @mmetc
- pkg/database: simplify flush agents/bouncers (#3026) @mmetc
Bug Fixes
- fix appsec/tls issues by cloning http transport (#3213) @mmetc
- cscli dashboard: exit earlier on docker (#3210) @mmetc
- hide geoip related warnings (#3179) @blotus
- add missing ApiKeyAuthorizer to decisions swagger endpoint (#3178) @a-mnich
- prevent nil deref when loading cti config (#3176) @mmetc
- metrics: avoid nil deref with inactive bouncers or malformed response (#3170) @mmetc
- cscli metrics: explicit message "no bouncer metrics found" (#3155) @mmetc
- cscli/hub: don't return error if some file can't be recognized (#3150) @mmetc
- docker: make sure the sqlite db is present before changing GID (#3140) @mmetc
- Store alert remediations status in DB (#3115) @blotus
- tests: increase delta for flaky float comparison (#3122) @mmetc
- typos (#3104) @mmetc
- fix "cscli [machines|bouncers] delete" autocompletion (#3101) @mmetc
- db: don't set machine heartbeat until first connection (#3019) @mmetc
- db: don't set bouncer last_pull until first connection (#3020) @mmetc
- Typos (#3084) @mmetc
- cscli: fixed some inconsistency in returning errors (#3076) @mmetc
- cscli: fix deprecation message for "context delete" (#3078) @mmetc
- fix test 01_cscli by avoiding discrepancies in cscli explain (#3059) @sabban
- add decently new version of python for centos 7 and 8 for testing (#3053) @sabban
Chore / Deps
- update grokky and deps (#3206) @blotus
- CI: use go 1.22.6 (#3171) @mmetc
- CI: update test dependencies (#3146) @mmetc
- update expr (#3144) @blotus
- CI: update action versions (#3143) @mmetc
- CI: use go 1.22.5 (#3128) @mmetc
- deps: use go-cs-lib 0.13 (#3136) @mmetc
- CI: use go 1.22.4 (#3095) @mmetc
- CI: update golangci-lint to 1.59; enforce canonical HTTP headers (#3074) @mmetc
- deps: use ent 0.13.1 (#3023) @mmetc
- lint: dockerfiles (#3138) @mmetc
- lint: replace "github.com/pkg/errors" in apiserver (#3097) @mmetc
- lint: import statement order (#3085) @mmetc
- lint: replace gocognit with revive (#3094) @mmetc
- lint: github.com/pkg/errors -> errors (#3091) @mmetc
- lint (intrange) (#2970) @mmetc
- lint (copyloopvar) (#2971) @mmetc
- CI/lint: enable some "revive" rules (#3041) @mmetc
- lint: revive/useless-break (#3050) @mmetc
- enable linter: revive(bool-literal-in-expr) (#2983) @mmetc
- enable linter: revive (deep-exit) (#3077) @mmetc
- enable linter: revive (var-declaration) (#3069) @mmetc
- enable linter: revive (indent-error-flow) (#3068) @mmetc
- enable linter "stylecheck" (#3060) @mmetc
- lint revive(deep-exit): refactor cmd/crowdsec (#3063) @mmetc
- enable linter: revive (blank-imports) (#3062) @mmetc
- enable linter: revive (early-return) (#3051) @mmetc
- lint: enable revive/if-return, revive/error-strings (#3057) @mmetc
- lint: disable redundant linters, update revive configuration and code metrics (#3081) @mmetc
- lint: replace cyclop, gocyclo with revive; basic pkg/hubtest helper (#3065) @mmetc
- enable linter: revive (superfluous-else) (#3082) @mmetc
- reduce log verbosity, minor CI fixes, lint (#3157) @mmetc
Geolite2 notice
This product includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com.
Installation
Take a look at the installation instructions.
Contributors
rnagy, blotus, and 6 other contributors