Extremely slow run using Docker #4991

bentcoder · 2024-09-08T09:57:59Z

bentcoder
Sep 8, 2024

Hi,

Do you see any problem with command below and the reason why it runs soooo slow?

Thanks

docker run \
	--rm \
	--tty \
	--workdir /source \
	--volume ${PWD}:/source:ro \
	--volume ${PWD}/tmp/golangci:/root/.cache:rw \
	golangci/golangci-lint:v1.60.3 golangci-lint run --verbose --config golangci.yaml

INFO golangci-lint has version 1.60.3 built with go1.23.0 from c2e095c0 on 2024-08-22T21:51:56Z 
INFO [config_reader] Used config file golangci.yaml 
INFO [lintersdb] Active 50 linters: [asciicheck bidichk bodyclose containedctx decorder dogsled dupl durationcheck errcheck errname errorlint exhaustive forbidigo forcetypeassert funlen gochecknoglobals gochecknoinits gocognit goconst gocritic gocyclo godot goimports gosec gosimple govet ineffassign interfacebloat lll makezero misspell nakedret nestif nilerr nilnil nlreturn noctx nonamedreturns prealloc rowserrcheck sloglint sqlclosecheck stylecheck tenv unconvert unparam unused usestdlibvars wastedassign whitespace] 
INFO [loader] Go packages loading at mode 575 (deps|exports_file|files|imports|compiled_files|name|types_sizes) took 38.010156635s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 155.602521ms 
INFO [linters_context/goanalysis] analyzers took 5m28.949602817s with top 10 stages: buildir: 2m16.073312749s, exhaustive: 40.022216746s, printf: 33.911185s, ctrlflow: 33.010373316s, fact_purity: 23.831024044s, bidichk: 9.075444414s, wastedassign: 7.982315882s, dupl: 5.634203254s, gocritic: 5.360372371s, goimports: 4.725916441s 
INFO [runner] Issues before processing: 214, after processing: 0 
INFO [runner] Processors filtering stat (in/out): skip_files: 214/214, skip_dirs: 214/214, exclude: 85/85, filename_unadjuster: 214/214, exclude-rules: 85/0, invalid_issue: 214/214, path_prettifier: 214/214, identifier_marker: 85/85, cgo: 214/214, autogenerated_exclude: 214/85 
INFO [runner] processing took 440.312744ms with stages: path_prettifier: 315.281196ms, autogenerated_exclude: 116.739199ms, identifier_marker: 5.144875ms, exclude-rules: 2.400753ms, skip_dirs: 591.211µs, invalid_issue: 96.411µs, cgo: 24.371µs, filename_unadjuster: 16.628µs, nolint: 5.708µs, sort_results: 5.537µs, uniq_by_line: 1.245µs, skip_files: 1.035µs, max_same_issues: 949ns, fixer: 932ns, exclude: 766ns, source_code: 719ns, max_from_linter: 417ns, path_shortener: 238ns, diff: 212ns, max_per_file_from_linter: 124ns, path_prefixer: 112ns, severity-rules: 106ns 
INFO [runner] linters took 1m8.075126192s with stages: goanalysis_metalinter: 1m7.631139507s 
INFO File cache stats: 58 entries of total size 103.0KiB 
INFO Memory: 917 samples, avg is 215.8MB, max is 464.4MB 
INFO Execution took 1m46.282639685s

$ tree tmp/
tmp/
├── golangci
│   ├── 00
│   │   ├── 000ab77b0ae7fe5ddc06f61d80dd9c8bc4fcc5a999479869292c216e7bfa2e1e-d
│   │   ├── ....
│   │   └── 00dedcbaf5dd36c49447863279d4bf0649eb1838d56074e8076fc688dff92fca-d
│   ├── 01
│   │   ├── 01140c8cf7100baa02be88890391dc695717b5cc4895d160e9fc29fd84d984b0-d
....

run:
  concurrency: 4
  timeout: 5m
  tests: true
  modules-download-mode: readonly
  allow-parallel-runners: true
  go: "1.23.0"

output:
  sort-results: true

linters-settings:
  decorder:
    dec-order:
      - const
      - var
      - type
      - func
  errcheck:
    check-blank: true
    exclude-functions:
      - (github.com/jackc/pgx/v5.Tx).Rollback(context.Context)
      - (*github.com/me/pkg/xrest.Response).Write
  funlen:
    lines: 100
    statements: 50
  gocognit:
    min-complexity: 20
  gocyclo:
    min-complexity: 20
  misspell:
    locale: UK
  nestif:
    min-complexity: 2
  sloglint:
    context: "all"
    static-msg: true
    args-on-sep-lines: true
    forbidden-keys:
      - time
      - level
      - message
      - log.time
      - log.level
      - log.message
      - log.caller
      - svc.name
      - svc.env
      - svc.ver

linters:
  disable-all: true
  enable:
    - asciicheck
    - bidichk
    - bodyclose
    - containedctx
    - decorder
    - dogsled
    - dupl
    - durationcheck
    - errcheck
    - errname
    - errorlint
    - exhaustive
    - forbidigo
    - forcetypeassert
    - funlen
    - gochecknoglobals
    - gochecknoinits
    - gocognit
    - goconst
    - gocritic
    - gocyclo
    - godot
    - goimports
    - gosimple
    - gosec
    - govet
    - ineffassign
    - interfacebloat
    - lll
    - makezero
    - misspell
    - nakedret
    - nestif
    - nilerr
    - nilnil
    - nlreturn
    - noctx
    - nonamedreturns
    - rowserrcheck
    - prealloc
    - sloglint
    - sqlclosecheck
    - stylecheck
    - tenv
    - typecheck
    - unconvert
    - unparam
    - unused
    - usestdlibvars
    - wastedassign
    - whitespace

issues:
  max-issues-per-linter: 0
  max-same-issues: 0
  exclude-rules:
    - path: main.go
      linters:
        - forbidigo
        - funlen
        - sloglint
    - path: _test.go
      linters:
        - funlen

Answered by ldez

Sep 9, 2024

This element --volume ${PWD}/tmp/golangci:/root/.cache:rw implies this:

${PWD}
├── go.mod
├── go.sum
├── main.go
├── ...
└── tmp <------------
    └── golangci
        └── ...

And because --volume ${PWD}:/source:ro, the ${PWD}/tmp/golangci folder is also 2 times inside the container:

/
├── root
│   └── .cache <----------
│       └── ...
├── source
│  ├── go.mod
│  ├── go.sum
│  ├── main.go
│  ├── ...
│  └── tmp <----------
│      └── golangci
│          └── ...
└── ...

So tmp folder is inside the project, like the other project files.

You should remove this folder and using something like that:

docker run \
	--rm \
	--tty \
	--workdir /source \
	--volume ${PWD}:/source:ro \
	--volume /t…

View full answer

ldez · 2024-09-09T00:43:13Z

ldez
Sep 9, 2024
Maintainer

I can see several elements:

the first run of golangci-lint can be slow because it needs to build the cache
the first run of golangci-lint can be slow because it needs to download project dependencies
you mounted ${PWD}/tmp/golangci but this directory is inside your project, those files will be detected by the analyzer and it will take time even if there are not Go files.

4 replies

bentcoder Sep 9, 2024
Author

For the third one, do you mean we should copy files into the container instead or something like that? An example would be great. Thank you.

ldez Sep 9, 2024
Maintainer

This element --volume ${PWD}/tmp/golangci:/root/.cache:rw implies this:

${PWD}
├── go.mod
├── go.sum
├── main.go
├── ...
└── tmp <------------
    └── golangci
        └── ...

And because --volume ${PWD}:/source:ro, the ${PWD}/tmp/golangci folder is also 2 times inside the container:

/
├── root
│   └── .cache <----------
│       └── ...
├── source
│  ├── go.mod
│  ├── go.sum
│  ├── main.go
│  ├── ...
│  └── tmp <----------
│      └── golangci
│          └── ...
└── ...

So tmp folder is inside the project, like the other project files.

You should remove this folder and using something like that:

docker run \
	--rm \
	--tty \
	--workdir /source \
	--volume ${PWD}:/source:ro \
	--volume /tmp/golangci:/root/.cache:rw \
	golangci/golangci-lint:v1.60.3 golangci-lint run --verbose --config golangci.yaml

Like that, the cache is outside the project files.

Answer selected by bentcoder

bentcoder Sep 9, 2024
Author

Thank you very much. Much appreciated the help 🥇 Nowhere near the previous speed.

INFO golangci-lint has version 1.60.3 built with go1.23.0 from c2e095c0 on 2024-08-22T21:51:56Z 
INFO [config_reader] Used config file golangci.yaml 
INFO [lintersdb] Active 50 linters: [asciicheck bidichk bodyclose containedctx decorder dogsled dupl durationcheck errcheck errname errorlint exhaustive forbidigo forcetypeassert funlen gochecknoglobals gochecknoinits gocognit goconst gocritic gocyclo godot goimports gosec gosimple govet ineffassign interfacebloat lll makezero misspell nakedret nestif nilerr nilnil nlreturn noctx nonamedreturns prealloc rowserrcheck sloglint sqlclosecheck stylecheck tenv unconvert unparam unused usestdlibvars wastedassign whitespace] 
INFO [loader] Go packages loading at mode 575 (compiled_files|exports_file|name|deps|files|imports|types_sizes) took 4.076682783s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 82.608353ms 
INFO [linters_context/goanalysis] analyzers took 0s with no stages 
INFO [runner] Issues before processing: 235, after processing: 0 
INFO [runner] Processors filtering stat (in/out): cgo: 235/235, skip_files: 235/235, exclude-rules: 106/0, path_prettifier: 235/235, skip_dirs: 235/235, identifier_marker: 106/106, filename_unadjuster: 235/235, invalid_issue: 235/235, autogenerated_exclude: 235/106, exclude: 106/106 
INFO [runner] processing took 93.306662ms with stages: autogenerated_exclude: 84.97633ms, identifier_marker: 3.54869ms, exclude-rules: 2.691503ms, path_prettifier: 1.529846ms, skip_dirs: 492.137µs, invalid_issue: 22.983µs, cgo: 20.092µs, filename_unadjuster: 10.489µs, sort_results: 5.168µs, nolint: 4.265µs, max_same_issues: 1.177µs, fixer: 705ns, uniq_by_line: 653ns, skip_files: 472ns, exclude: 412ns, source_code: 401ns, diff: 384ns, max_from_linter: 267ns, severity-rules: 214ns, max_per_file_from_linter: 177ns, path_shortener: 167ns, path_prefixer: 130ns 
INFO [runner] linters took 853.591085ms with stages: goanalysis_metalinter: 760.013858ms 
INFO File cache stats: 0 entries of total size 0B 
INFO Memory: 52 samples, avg is 30.4MB, max is 91.0MB 
INFO Execution took 5.054544491s

ldez Sep 9, 2024
Maintainer

If you appreciate our work, think to give us a tip 😉 :

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extremely slow run using Docker #4991

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 4 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Extremely slow run using Docker #4991

bentcoder Sep 8, 2024

Replies: 1 comment · 4 replies

ldez Sep 9, 2024 Maintainer

bentcoder Sep 9, 2024 Author

ldez Sep 9, 2024 Maintainer

bentcoder Sep 9, 2024 Author

ldez Sep 9, 2024 Maintainer

bentcoder
Sep 8, 2024

Replies: 1 comment 4 replies

ldez
Sep 9, 2024
Maintainer

bentcoder Sep 9, 2024
Author

ldez Sep 9, 2024
Maintainer

bentcoder Sep 9, 2024
Author

ldez Sep 9, 2024
Maintainer