Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ngtcp2] fix: detect the libressl in vcpkg #40922

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

talregev
Copy link
Contributor

@talregev talregev commented Sep 12, 2024

  • Changes comply with the maintainer guide.
  • SHA512s are updated for each updated download.
  • The "supports" clause reflects platforms that may be fixed by this new version.
  • Any fixed CI baseline entries are removed from that file.
  • Any patches that are no longer applied are deleted from the port's directory.
  • The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
  • Only one version is added to each modified port's versions file.

fix #40915

@dg0yt
Copy link
Contributor

dg0yt commented Sep 12, 2024

Reminder to all: libressl is not built in vcpkg CI. (Reason: conflict with openssl.)

So this must be tested manually, thoroughly.

@FrankXie05 FrankXie05 added the category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist label Sep 12, 2024
@talregev
Copy link
Contributor Author

@FrankXie05 This PR solve the regression when you have external openssl in the system, and ngtcp2[libressl] is detect it instead the vcpkg libressl.

@BillyONeal
Copy link
Member

Has this change been submitted upstream? If they have a hard dependency on the libressl or boringssl fork(s) they should be doing this themselves.

@talregev
Copy link
Contributor Author

talregev commented Sep 13, 2024

Has this change been submitted upstream? If they have a hard dependency on the libressl or boringssl fork(s) they should be doing this themselves.

I didn't submit the change to upstream because this is unique to vcpkg.

The cmake flag of openssl and the find_package(openssl) in upstream is to detect libressl/quictls/boringssl but not openssl itself.

In vcpkg I choose libressl because is the only one that fit.
We don't have quictls port.
boringssl need to update.

Then we need unique find package for libressl that it will not detect other libs outside vcpkg.

@dg0yt
Copy link
Contributor

dg0yt commented Sep 13, 2024

Then we need unique find package for libressl that it will not detect other libs outside vcpkg.

We have no reproducer for the claim, but it is proven wrong for x64-linux with vcpig-ci-libressl from #40924.

Nevertheless this PR fixes a problem: libressl doesn't have a CMake wrapper, and plain find_package(OpenSSL) won't properly deal with transitive usage requirements.

@talregev
Copy link
Contributor Author

talregev commented Sep 13, 2024

We have no reproducer for the claim, but it is proven wrong for x64-linux with vcpig-ci-libressl from #40924.

I reproduce this claim on github ci action. on windows latest.
Try with find_package(OpenSSL)

@talregev
Copy link
Contributor Author

talregev commented Sep 14, 2024

@dg0yt I run your test port on github action windows ci.
https://github.com/talregev/curl/actions/runs/10859242578/job/30138314640?pr=22
and I download the logs.

windows_logs_7696dc6fb70eff5b0a5079d78b9818f4dc486861.zip

I reproduce the problem:
config-x64-windows-out.log

-- Found OpenSSL: optimized;C:/vcpkg/installed/x64-windows/lib/crypto.lib;debug;C:/Program Files/OpenSSL/lib/VC/libcrypto64MDd.lib (found version "2.0.0")
-- OpenSSL::SSL IMPORTED_LOCATION_DEBUG: C:/Program Files/OpenSSL/lib/VC/libssl64MDd.lib
CMake Error at CMakeLists.txt:19 (message):
  OpenSSL::SSL IMPORTED_LOCATION_DEBUG is not from vcpkg.

As I say, find_package(openssl) is detect openssl libs outside vcpkg, in github ci action, and this PR is fix that.

@talregev
Copy link
Contributor Author

@BillyONeal Please review my PR. thank you.

@talregev
Copy link
Contributor Author

talregev commented Sep 14, 2024

@FrankXie05 @jimwang118
Any reason for delay review this PR?
Other PRs that come after this PR already review and merge.

@BillyONeal
Copy link
Member

I didn't submit the change to upstream because this is unique to vcpkg.

The cmake flag of openssl and the find_package(openssl) in upstream is to detect libressl/quictls/boringssl but not openssl itself.

If upstream wants to detect libressl/quictls/boringssl but might get OpenSSL how is that not relevant to upstream? In particular hard coding that the answer is libressl and ignoring what the package says seems questionable.

In vcpkg I choose libressl because is the only one that fit. We don't have quictls port. boringssl need to update.

Then we need unique find package for libressl that it will not detect other libs outside vcpkg.

It seems like this is a bug in the libressl port rather than something that should change in ngtcp2?

@FrankXie05 @jimwang118
Any reason for delay review this PR?

They are on vacation :)

@talregev
Copy link
Contributor Author

talregev commented Sep 16, 2024

If upstream wants to detect libressl/quictls/boringssl but might get OpenSSL how is that not relevant to upstream? In particular hard coding that the answer is libressl and ignoring what the package says seems questionable.

Upstream want to detect libressl/quictls/boringssl. When the not relevant lib is detected, like openssl, it not enable the quic ngtcp2 code. This is how upstream want it.

It seems like this is a bug in the libressl port rather than something that should change in ngtcp2?

It not a bug in libressl, it by design how find_package work. First it detect in vcpkg, then it detect outside. (needed for many ports to detect libs outside vcpkg). The hidden bug maybe, when iterate, it show first the openssl. You can see that in the logs.

They are on vacation :)

Thank you! I didn't know that :)

@talregev
Copy link
Contributor Author

@BillyONeal Can other people in the team that available can review my PR?
Thank you in advance :)

@dg0yt
Copy link
Contributor

dg0yt commented Sep 17, 2024

It seems like this is a bug in the libressl port rather than something that should change in ngtcp2?

It not a bug in libressl, it by design how find_package work. First it detect in vcpkg, then it detect outside. (needed for many ports to detect libs outside vcpkg). The hidden bug maybe, when iterate, it show first the openssl. You can see that in the logs.

  • IMO there is a bug in port libressl with regard to find_package(OpenSSL): The port must take care of platform and config type properties which are unknown to FindOpenSSL.cmake. This is what vcpkg cmake wrappers exist for. In particular, it must take of the binary names for windows debug artifacts:

    -- Found OpenSSL: optimized;C:/vcpkg/installed/x64-windows/lib/crypto.lib;debug;C:/Program Files/OpenSSL/lib/VC/libcrypto64MDd.lib (found version "2.0.0")
    
  • find_package has the right search order for packages. Problems often come from find_library (used in Find modules) when it lacks candidate NAMES or NAMES_PER_DIR. This is to be initialized by the wrapper instead.

@dg0yt
Copy link
Contributor

dg0yt commented Sep 17, 2024

If upstream wants to detect libressl/quictls/boringssl but might get OpenSSL how is that not relevant to upstream? In particular hard coding that the answer is libressl and ignoring what the package says seems questionable.

Upstream want to detect libressl/quictls/boringssl. When the not relevant lib is detected, like openssl, it not enable the quic ngtcp2 code. This is how upstream want it.

Is upstream aware of libressl offering official CMake config? I believe they don't want users to be frustrated about find_package(OpenSSL) working unreliably for the purpose of detecting libressl.

@talregev
Copy link
Contributor Author

It seems like this is a bug in the libressl port rather than something that should change in ngtcp2?

It not a bug in libressl, it by design how find_package work. First it detect in vcpkg, then it detect outside. (needed for many ports to detect libs outside vcpkg). The hidden bug maybe, when iterate, it show first the openssl. You can see that in the logs.

  • IMO there is a bug in port libressl with regard to find_package(OpenSSL): The port must take care of platform and config type properties which are unknown to FindOpenSSL.cmake. This is what vcpkg cmake wrappers exist for. In particular, it must take of the binary names for windows debug artifacts:
    -- Found OpenSSL: optimized;C:/vcpkg/installed/x64-windows/lib/crypto.lib;debug;C:/Program Files/OpenSSL/lib/VC/libcrypto64MDd.lib (found version "2.0.0")
    
  • find_package has the right search order for packages. Problems often come from find_library (used in Find modules) when it lacks candidate NAMES or NAMES_PER_DIR. This is to be initialized by the wrapper instead.

@dg0yt Can you fix it for libressl port?

@dg0yt
Copy link
Contributor

dg0yt commented Sep 17, 2024

Not planned.

@talregev
Copy link
Contributor Author

talregev commented Sep 17, 2024

Not planned.

Then better to merge this fix until we find time to fix libressl.

@talregev
Copy link
Contributor Author

@FrankXie05 Can you put the review label?
This way they actually look on this PR, and review it.

@FrankXie05 FrankXie05 added the info:reviewed Pull Request changes follow basic guidelines label Sep 19, 2024
@BillyONeal
Copy link
Member

@ras0219-msft @vicroms @JavierMatosD @data-queue and I discussed this today.

We agree that it could be acceptable to take this change given that the feature is named "libressl":

    "libressl": {
      "description": "Compile with libressl",
      "dependencies": [
        "libressl"
      ]
    },

However, we believe that is a mistake and thus the overall edit goes in the wrong direction. find_package(OpenSSL should be owned by the port named openssl, and attempting to use libressl like this should require an overlay-port.

We also agree that this output:

-- Found OpenSSL: optimized;C:/vcpkg/installed/x64-windows/lib/crypto.lib;debug;C:/Program Files/OpenSSL/lib/VC/libcrypto64MDd.lib (found version "2.0.0")

suggests a bug in the LibreSSL port and should not be worked around in every downstream port that wants to consume LibreSSL in this way.

Of note, this change breaks boringssl which presumably would work if someone overlayed boringssl -> libressl.

We would be happy to accept a change that fixes LibreSSL to make this work (even though we have no means of testing this in CI, there's a special exception for libressl and boringssl as they were added before the addition of registries to the product) but we are not willing to take this change at this time.

Thanks for your contribution to vcpkg and hope that makes sense!

@BillyONeal BillyONeal removed the info:reviewed Pull Request changes follow basic guidelines label Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ ngtcp2[libressl] ] is not building libressl feature on github action ci
4 participants