Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question]: checkout self fails on self-hosted agent with System.UnauthorizedAccessException: Access to the path '/proc/53314/oom_score_adj' is denied. #4972

Open
1 of 4 tasks
Timo-Weike opened this issue Sep 2, 2024 · 1 comment

Comments

@Timo-Weike
Copy link

Describe your question

The agent sometimes has issues with the "Get sources" (checkout self). During the tasks execution randome commands fail with an access denied error.

Following is an example log where one commands fails with the error and the following command does not fail

2024-09-02T14:27:32.5394191Z ##[debug]Evaluating condition for step: 'Checkout <repo-name>@<branch-name> to self'
2024-09-02T14:27:32.5435975Z ##[debug]Evaluating: SucceededNode()
2024-09-02T14:27:32.5440824Z ##[debug]Evaluating SucceededNode:
2024-09-02T14:27:32.5472167Z ##[debug]=> True
2024-09-02T14:27:32.5479169Z ##[debug]Result: True
2024-09-02T14:27:32.5532479Z ##[section]Starting: Checkout <repo-name>@<branch-name> to self
2024-09-02T14:27:32.5695593Z ==============================================================================
2024-09-02T14:27:32.5696024Z Task         : Get sources
2024-09-02T14:27:32.5696190Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2024-09-02T14:27:32.5696293Z Version      : 1.0.0
2024-09-02T14:27:32.5696478Z Author       : Microsoft
2024-09-02T14:27:32.5696718Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2024-09-02T14:27:32.5696812Z ==============================================================================
2024-09-02T14:27:32.5814673Z ##[debug]Resource Utilization warnings is disabled, switch "DISABLE_RESOURCE_UTILIZATION_WARNINGS" variable to "false" if you want to enable it
2024-09-02T14:27:33.0243225Z ##[debug]Processed: ##vso[telemetry.publish area=AzurePipelinesAgent;feature=Checkout]{"RepoType":"Git","HostOS":"Linux"}
2024-09-02T14:27:33.0255449Z ##[debug]Processed: ##vso[plugininternal.updaterepositorypath alias=self;]/home/<user>/myagent/_work/1/self
2024-09-02T14:27:33.0257418Z ##[debug]Repository requires to be placed at '/home/<user>/myagent/_work/1/self', current location is '/home/<user>/myagent/_work/1/self'
2024-09-02T14:27:33.0258058Z Syncing repository: <repo-name> (Git)
2024-09-02T14:27:33.0258768Z ##[debug]repository url=https://<org>@dev.azure.com/<org>/<project-name>/_git/<repo-name>
2024-09-02T14:27:33.0259466Z ##[debug]targetPath=/home/<user>/myagent/_work/1/self
2024-09-02T14:27:33.0260096Z ##[debug]sourceBranch=refs/heads/<branch-name>
2024-09-02T14:27:33.0260738Z ##[debug]sourceVersion=2f920210c95f8015b589737251c9120f57af746a
2024-09-02T14:27:33.0261357Z ##[debug]clean=True
2024-09-02T14:27:33.0261883Z ##[debug]checkoutSubmodules=False
2024-09-02T14:27:33.0262472Z ##[debug]checkoutNestedSubmodules=False
2024-09-02T14:27:33.0263041Z ##[debug]exposeCred=True
2024-09-02T14:27:33.0263577Z ##[debug]fetchDepth=1
2024-09-02T14:27:33.0264099Z ##[debug]fetchFilter=
2024-09-02T14:27:33.0264838Z ##[debug]fetchTags=True
2024-09-02T14:27:33.0265452Z ##[debug]gitLfsSupport=False
2024-09-02T14:27:33.0265999Z ##[debug]acceptUntrustedCerts=False
2024-09-02T14:27:33.0266553Z ##[debug]schannelSslBackend=False
2024-09-02T14:27:33.0267091Z ##[debug]Which: 'git'
2024-09-02T14:27:33.0410321Z ##[debug]Location: '{matches.First()}'
2024-09-02T14:27:33.0410985Z ##[debug]Which: 'git-lfs'
2024-09-02T14:27:33.0411515Z ##[debug]Not found.
2024-09-02T14:27:33.0412033Z ##[debug]Get git version.
2024-09-02T14:27:33.0412410Z ##[command]git version
2024-09-02T14:27:33.0412873Z ##[debug]Starting process:
2024-09-02T14:27:33.0413435Z ##[debug]  File name: '/usr/bin/git'
2024-09-02T14:27:33.0414011Z ##[debug]  Arguments: 'version'
2024-09-02T14:27:33.0415092Z ##[debug]  Working directory: '/home/<user>/myagent/_work'
2024-09-02T14:27:33.0415774Z ##[debug]  Require exit code zero: 'False'
2024-09-02T14:27:33.0416441Z ##[debug]  Encoding web name:  ; code page: ''
2024-09-02T14:27:33.0417183Z ##[debug]  Force kill process on cancellation: 'False'
2024-09-02T14:27:33.0417920Z ##[debug]  Redirected STDIN: 'False'
2024-09-02T14:27:33.0418543Z ##[debug]  Persist current code page: 'False'
2024-09-02T14:27:33.0419143Z ##[debug]  Keep redirected STDIN open: 'False'
2024-09-02T14:27:33.0419706Z ##[debug]  High priority process: 'False'
2024-09-02T14:27:33.0420323Z ##[debug]  ContinueAfterCancelProcessTreeKillAttempt: 'False'
2024-09-02T14:27:33.0420980Z ##[debug]  Sigint timeout: '00:00:07.5000000'
2024-09-02T14:27:33.0421558Z ##[debug]  Sigterm timeout: '00:00:02.5000000'
2024-09-02T14:27:33.0422146Z ##[debug]  Try to use graceful shutdown: False
2024-09-02T14:27:33.0430137Z ##[debug]Failed to update oom_score_adj for PID: 53314.
2024-09-02T14:27:33.0435850Z ##[debug]Exited process 53314 with exit code 0
2024-09-02T14:27:33.0438416Z ##[debug]System.UnauthorizedAccessException: Access to the path '/proc/53314/oom_score_adj' is denied.
 ---> System.IO.IOException: Permission denied
   --- End of inner exception stack trace ---
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
   at System.IO.StreamWriter.ValidateArgsAndOpenPath(String path, Boolean append, Encoding encoding, Int32 bufferSize)
   at System.IO.File.WriteAllText(String path, String contents)
   at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.WriteProcessOomScoreAdj(Int32 processId, Int32 oomScoreAdj) in /mnt/vss/_work/1/s/src/Agent.Sdk/ProcessInvoker.MacLinux.cs:line 75
2024-09-02T14:27:33.0441062Z ##[debug]Process started with process id 53314, waiting for process exit.
2024-09-02T14:27:33.0441700Z ##[debug]STDOUT/STDERR stream read finished.
2024-09-02T14:27:33.0442266Z ##[debug]STDOUT/STDERR stream read finished.
2024-09-02T14:27:33.0442948Z ##[debug]Finished process 53314 with exit code 0, and elapsed time 00:00:00.0560151.
2024-09-02T14:27:33.0443415Z git version 2.43.0
2024-09-02T14:27:33.0443887Z ##[debug]Detect git version: 2.43.0.
2024-09-02T14:27:33.0444764Z ##[debug]Set git useragent to: git/2.43.0 (vsts-agent-git/3.243.1).
2024-09-02T14:27:33.0445571Z ##[debug]Checking if the repo on /home/<user>/myagent/_work/1/self matches the expected repository origin URL. expected Url: https://<org>@dev.azure.com/<org>/<project-name>/_git/<repo-name>
2024-09-02T14:27:33.0446437Z ##[debug]Inspect remote.origin.url for repository under /home/<user>/myagent/_work/1/self
2024-09-02T14:27:33.0608751Z ##[command]git config --get remote.origin.url
2024-09-02T14:27:33.0616361Z ##[debug]Starting process:
2024-09-02T14:27:33.0617112Z ##[debug]  File name: '/usr/bin/git'
2024-09-02T14:27:33.0617705Z ##[debug]  Arguments: 'config --get remote.origin.url'
2024-09-02T14:27:33.0618333Z ##[debug]  Working directory: '/home/<user>/myagent/_work/1/self'
2024-09-02T14:27:33.0618932Z ##[debug]  Require exit code zero: 'False'
2024-09-02T14:27:33.0619530Z ##[debug]  Encoding web name:  ; code page: ''
2024-09-02T14:27:33.0620155Z ##[debug]  Force kill process on cancellation: 'False'
2024-09-02T14:27:33.0620824Z ##[debug]  Redirected STDIN: 'False'
2024-09-02T14:27:33.0621408Z ##[debug]  Persist current code page: 'False'
2024-09-02T14:27:33.0622203Z ##[debug]  Keep redirected STDIN open: 'False'
2024-09-02T14:27:33.0622797Z ##[debug]  High priority process: 'False'
2024-09-02T14:27:33.0623395Z ##[debug]  ContinueAfterCancelProcessTreeKillAttempt: 'False'
2024-09-02T14:27:33.0623998Z ##[debug]  Sigint timeout: '00:00:07.5000000'
2024-09-02T14:27:33.0624856Z ##[debug]  Sigterm timeout: '00:00:02.5000000'
2024-09-02T14:27:33.0625451Z ##[debug]  Try to use graceful shutdown: False
2024-09-02T14:27:33.0626066Z ##[debug]Updated oom_score_adj to 500 for PID: 53321.
2024-09-02T14:27:33.0626698Z ##[debug]Process started with process id 53321, waiting for process exit.
2024-09-02T14:27:33.0627363Z ##[debug]STDOUT/STDERR stream read finished.
2024-09-02T14:27:33.0627955Z ##[debug]STDOUT/STDERR stream read finished.
2024-09-02T14:27:33.0628523Z ##[debug]Exited process 53321 with exit code 0
2024-09-02T14:27:33.0629428Z ##[debug]Finished process 53321 with exit code 0, and elapsed time 00:00:00.0048143.
2024-09-02T14:27:33.0630204Z ##[debug]Get remote origin fetch url from git config: https://<org>@dev.azure.com/<org>/<project-name>/_git/<repo-name>
2024-09-02T14:27:33.0630994Z ##[debug]Repository remote origin fetch url is https://<org>@dev.azure.com/<org>/<project-name>/_git/<repo-name>
2024-09-02T14:27:33.0631648Z ##[debug]URLs match.
2024-09-02T14:27:33.0632290Z ##[debug]Show the working tree status for repository at /home/<user>/myagent/_work/1/self

...

Versions

Azure Dev Ops agent 3.243.1 / Ubuntu

Environment type (Please select at least one enviroment where you face this issue)

  • Self-Hosted
  • Microsoft Hosted
  • VMSS Pool
  • Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Operation system

Linux (ubuntu 24.04)

Version controll system

Git

Azure DevOps Server Version (if applicable)

No response

@vmapetr
Copy link
Contributor

vmapetr commented Sep 9, 2024

Hi @Timo-Weike, thank you for reporting this!
Could you please confirm that the issue is also occurs on previous agent versions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants