Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vendoring can lead to corrupt files #23396

Open
mortenmj opened this issue Aug 22, 2024 · 2 comments
Open

Vendoring can lead to corrupt files #23396

mortenmj opened this issue Aug 22, 2024 · 2 comments
Labels
more data needed team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug

Comments

@mortenmj
Copy link

Description of the bug:

Occasionally, when vendoring all dependencies in our repository, file content can be written to the wrong file, leading to corrupt dependencies. Other than stress testing the system by vendoring a large number of dependencies, which is when I generally see this, I am not sure how to reproduce this reliably.
In the specific example I'm seeing now, a partial golang source file has been written to a BUILD.bazel file.

Which category does this issue belong to?

Core

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Run bazel vendor //... in a repository with many external dependencies, potentially several times in order to trigger the issue

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 7.3.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

Running in a GitHub Codespace. May or may not matter w.r.t. the reliability of the underlying platform

@github-actions github-actions bot added the team-Core Skyframe, bazel query, BEP, options parsing, bazelrc label Aug 22, 2024
@meisterT meisterT added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. and removed team-Core Skyframe, bazel query, BEP, options parsing, bazelrc labels Aug 28, 2024
@meisterT
Copy link
Member

cc @meteorcloudy

@meteorcloudy
Copy link
Member

This is so weird, the way a repo is vendored is by moving its source from the external root to the vendor directory, we are not even copying any files around:

// 3. Move the external repo to vendor dir. It's fine if this step fails or is interrupted,
// because the marker file under external is gone anyway.
FileSystemUtils.moveTreesBelow(repoUnderExternal, repoUnderVendor);

It'll be really helpful if you can somehow construct a potential reproducible case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
more data needed team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug
Projects
None yet
Development

No branches or pull requests

6 participants