Provide an API to unify PATH lookups for dotnet across client extensions

[baronfel]

Is your feature request related to a problem? Please describe.

C#, DevKit, and many other extensions don't have a unified way of locating dotnet. This Extension is in a unique place of mediating access to SDKs across many extensions, and so is a good fit for an API that can be called to unify all of the disparate error handling and lookups that other extensions have to know today.

A clear and concise description of what you want to happen. Include any alternative solutions you've considered.

We should create an API that captures the fast-checking and lookups that our client extensions do today to find dotnet.

Describe the solution you'd like

We create and export the new API.

Additional context

No response

[dibarbet]

For the C# extension, the first place we look for .net is finding a runtime to run the extension on. That code can be found here - https://github.com/dotnet/vscode-csharp/blob/main/src/lsptoolshost/dotnetRuntimeExtensionResolver.ts#L41

1. First, we check the dotnet.dotnetPath VSCode setting to see if the user has specified a path to a runtime in VSCode settings.
2. If no path is specified in dotnet.dotnetPath, we attempt to find a global .NET installation using the $PATH. This essentially shells out to dotnet --info and parses the result. We check that the .NET architecture and extension architecture match and that the installed .NET has a runtime with a minimum version (at the time of writing, 8.0 is the minimum). We allow for example a .NET 9 preview runtime to be used as we have rollForward enabled on the server.
3. If no acceptable runtime is found using the $PATH, we fallback to installing a local runtime using the .NET install tool API. At the time of writing this requests an 8.0 version of the runtime

If DevKit is being used, that's it - the C# extension won't do any SDK resolution. However, if the C# extension is running standalone without DevKit, we have to resolve the SDK in a couple of places.

1. When loading the project, we call out to MSBuildLocator to find the correct SDK to use. We rely on MSBuildLocator to do the heavy lifting to find the appropriate SDK.
2. We also need to run commands against the SDK (e.g. restore), or find the vstest dlls from the SDK. In that scenario, we parse the $PATH to find the SDK on the server side. That code lives here (and likely isn't as well tested as the runtime acquisition part). We don't verify anything about the SDK, we assume that the one on the path is appropriate for the project.

[lifengl]

For C# Dev Kit extension, our logic is slightly different from the C# extension:
1, we didn't take the dotnet.dotnetPath setting (which sounds we should, to match the expectation from customers using C# extension, as $PATH is not always an option in a remote session (like working with a Linux container)

2, we shell out which/where dotnet, which is essentially resolving dotnet from $PATH, when which/where is not found in the system, we enumerate $PATH environment variable. In any case, if dotnet is found through PATH, we resolve it to the physical location by calling realpath, and check whether it is linked to a real dotnet executable. If it is not, we shell dotnet --list-runtimes, and find the dotnet from parent folder chain where the runtime is installed. The location is necessary as most C# Dev Kit is built to be self-host executable, and we need set DOTNET_ROOT to allow them to resolve the runtime.

After this point, we will check whether a runtime installed to the global dotnet location meets our minimal condition to load the extension, which is basically that a non preview version of NET 8 runtime + ASP.NET Core has been installed in the location. If it is false, we will fallback to install a local runtime using .NET install tool API. We depend on this tool to install a release version of NET 8 runtime to load majority of C# Dev Kit processes.

The exactly runtime being used in C# Dev Kit depends:
1, pure tooling processes: including code server, service host to support test explorer, Aspire service and reliability watching processes:
all of them are self-hosting processes and only run with a release version of NET 8 runtime (except Aspire, which uses release version of NET 8 ASP.Core runtime). They will use the runtime from the global installation (when it meets the condition), or one installed locally.

2, pure user domain process: this includes the process we run design time builds. This process is not using self-host process, we always load it with the global installation dotnet (after we resolve it through realPath, or dotnet --list-runtimes). This process requires NET 6 runtime, but we set flag to allow it to rollForward to latest runtime/preview version of runtime installed in the SDK location, and MSBuildLocator is loaded in this process to load msbuild from SDK (based on global.json). MsBuild nodes were spinned off from this process, as we force all design time build to run out of process nodes.

3, the project system hosting process: this process is somewhat hybrid. It requires NET 8 runtime, and rollForward is enabled. If the user has NET 8/9 SDK installed, it will use the global runtime (we need run a higher version of runtime to work with the version of the SDK used by the user code). If the user only has older NET 6/7 SDK, it will run with a local installation.

By default, it is a self-host process, but we switch to load it with global dotnet (the same way as the DT build hosting process) to work with test signed version of SDK on Mac. (self host process cannot load test signed runtime through Mac security layer).

4, C# Dev Kit also uses dotnet command extensively for 'build'. 'restore packages', 'create new project', 'create project item' etc, and all of them will spin off dotnet command directly, which depends on the command to be resolved through $PATH environment. The earlier logic is only used to ensure we load CDK processes, but was not used in those scenarios (except some feature might use resolved dotnet version to know capabilities of the command, like whether certain templates are supported.