System.InvalidOperationException and System.Collections.Generic.KeyNotFoundException errors when running tests

[liamfl]

Description

I am running unit tests on a 2 c# test projects using the lates version of Visual Studio Community 2022

The 2 test project are named EngineTests and ServerCodeTests.

While the test projects compile correctly, I regularly get a series of error messages in the Output / Tests window when running unit tests:

System.InvalidOperationException: TestsForOrigin: Unknown TestCaseRecord.Origin value Unknown
System.Collections.Generic.KeyNotFoundException: The given key 'C:\Users\liamf\Source\Repos\TradingReplay\ServerCodeTests\bin\Debug\net6.0\ServerCodeTests.dll' was not present in the dictionary.
An exception occurred while test discoverer 'MSTestDiscoverer' was loading tests. Exception: Could not load type 'Microsoft.VisualStudio.TestTools.UnitTesting.StringEx' from assembly 'Microsoft.VisualStudio.TestPlatform.TestFramework, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
No test is available in C:\Users\liamf\Source\Repos\TradingReplay\ServerCodeTests\bin\Debug\net6.0\ServerCodeTests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Steps to reproduce

Running any test from Test Explorer.

Reloading Visual Studio allows you to do 2-3 test runs before the issue arises again.

Diagnostic logs

ErrorLog.text attached

Environment

Windows 11 10.0.22621 Build 22621
Visual Studiio Community v17.4.3

Nuget packages:
MSTest.TesAdapter v3
MSTest.TestFramework v3
Microsoft.NET.Test.SDK v17.4
ErrorLog.txt

[Evangelink]

Relates to https://developercommunity.visualstudio.com/t/Unit-test-startup-fails-with-an-exceptio/10226576

[Evangelink]

This is currently being investigated, we will post back when we have more info.

[jeremyregnerus]

I have this same issue and have found this to happen whenever you delete or rename a test method.

[robin-kilpatrick]

I have the same issue - run the same test twice and then I can't run it a third time until I restart vs2022 community

[primalspacesystems]

I'm having the same problem - Visual Studio Community 17.4.3 - but for a mixture of C++ and C# tests.

Most of the time, a machine reboot seems to make the issue go away for a while - this is less than ideal :/

(Debug spew after about info)

Cheers
John

Microsoft Visual Studio Community 2022
Version 17.4.3
VisualStudio.17.Release/17.4.3+33205.214
Microsoft .NET Framework
Version 4.8.04161

Installed Version: Community

Visual C++ 2022 00482-90000-00000-AA725
Microsoft Visual C++ 2022

ADL Tools Service Provider 1.0
This package contains services used by Data Lake tools

ASA Service Provider 1.0

ASP.NET and Web Tools 17.4.326.54890
ASP.NET and Web Tools

ASP.NET Web Frameworks and Tools 2012 17.4.326.54890
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0 17.4.326.54890
Azure App Service Tools v3.0.0

Azure Data Lake Tools for Visual Studio 2.6.5000.0
Microsoft Azure Data Lake Tools for Visual Studio

Azure Functions and Web Jobs Tools 17.4.326.54890
Azure Functions and Web Jobs Tools

Azure Stream Analytics Tools for Visual Studio 2.6.5000.0
Microsoft Azure Stream Analytics Tools for Visual Studio

C# Tools 4.4.0-6.22580.4+d7a61210a88b584ca0827585ec6e871c6b1c5a14
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

CodeLensOopProviderCommands Extension 1.0
CodeLensOopProviderCommands Visual Studio Extension Detailed Info

Common Azure Tools 1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Cookiecutter 17.0.22263.6
Provides tools for finding, instantiating and customizing templates in cookiecutter format.

Linux Core Dump Debugging 1.0.9.33020
Enables debugging of Linux core dumps.

Microsoft Azure Hive Query Language Service 2.6.5000.0
Language service for Hive query

Microsoft Azure Stream Analytics Language Service 2.6.5000.0
Language service for Azure Stream Analytics

Microsoft JVM Debugger 1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

NuGet Package Manager 6.4.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

P4VS - Helix Plugin for Visual Studio 2021.2.235.1465
P4VS is the Helix Plugin for Microsoft Visual Studio. P4VS provides developers working within Visual Studio access to Helix version control features, including check in/check out, view file history, and more.

Python - Django support 17.0.22263.6
Provides templates and integration for the Django web framework.

Python - Profiling support 17.0.22263.6
Profiling support for Python projects.

Python with Pylance 17.0.22263.6
Provides IntelliSense, projects, templates, debugging, interactive windows, and other support for Python developers.

Razor (ASP.NET Core) 17.0.0.2246202+61cc048d36a3fc9246d2f04625988b19a18ab8f0
Provides languages services for ASP.NET Core Razor.

SQL Server Data Tools 17.0.62207.28050
Microsoft SQL Server Data Tools

Test Adapter for Boost.Test 1.0
Enables Visual Studio's testing tools with unit tests written for Boost.Test. The use terms and Third Party Notices are available in the extension installation directory.

Test Adapter for Google Test 1.0
Enables Visual Studio's testing tools with unit tests written for Google Test. The use terms and Third Party Notices are available in the extension installation directory.

ToolWindowHostedEditor 1.0
Hosting json editor into a tool window

TypeScript Tools 17.0.10921.2001
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools 4.4.0-6.22580.4+d7a61210a88b584ca0827585ec6e871c6b1c5a14
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual C++ for Linux Development 1.0.9.33020
Visual C++ for Linux Development

Visual F# Tools 17.4.0-beta.22512.4+525d5109e389341bb90b144c24e2ad1ceec91e7b
Microsoft Visual F# Tools

Visual Studio IntelliCode 2.2
AI-assisted development for Visual Studio.

System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.ToLocalTestResultRecord(TestResultRecord testResult)
at System.Linq.Enumerable.WhereSelectListIterator2.MoveNext() at Microsoft.VisualStudio.TestWindow.Host.TestRunSession.AddTestResultRecords(IEnumerable1 testResultRecords)
at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.AddRemoteTestResultRecords(IEnumerable1 testResults) at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.OnTestRunUpdate(IEnumerable1 testResults, IEnumerable1 activeTests) at Microsoft.VisualStudio.TestWindow.Host.TestPlatformProvider.TestResponseHandler.<>c__DisplayClass16_0.<HandleTestRunStatsChangeAsync>b__0() at Microsoft.VisualStudio.TestWindow.Logging.ILoggerExtensions.CallInternalWithLogging(IInternalLogger log, Action action, Boolean shouldThrow) System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.VisualStudio.TestWindow.Host.TestPlatformProvider.<RunTestsAsync>d__27.MoveNext() System.InvalidOperationException: TestsForOrigin: Unknown TestCaseRecord.Origin value Unknown at Microsoft.VisualStudio.TestStorage.MergedTestGroup.TestsForOrigin(TestCaseOrigin origin) at Microsoft.VisualStudio.TestStorage.MergedTestGroup.MarkAsNotRunningAndNotPending() at Microsoft.VisualStudio.TestStorage.MergedTestIndex.MarkAsNotRunningAndNotPending(TestCaseOriginKind originKind) at Microsoft.VisualStudio.TestStorage.TestStoreIndexSet.MarkTestsAsNotRunningAndNotPending(TestCaseOriginKind originKind) at Microsoft.VisualStudio.TestStorage.TestStore.MarkTestsAsNotRunningAndNotPending(TestCaseOriginKind originKind) at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.EndTestRun() at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.OnTestRunCompleted() at Microsoft.VisualStudio.TestWindow.Utilities.EventPumpExtensions.<>c__DisplayClass0_0.<EnqueueAsync>b__0() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.TestWindow.Host.TestPlatformProvider.<RunTestsAsync>d__27.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.TestWindow.Host.TestPlatformProvider.<RunTestsAsync>d__25.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.TestWindow.Logging.ILoggerExtensions.<CallWithCatchAsync>d__111.MoveNext()

[gigimignon]

I have same issue on
Microsoft Visual Studio Professional 2022 (64 bits) - Current
Version 17.4.3

It seems that explorer fail to refresh tests.
Some times, one test with DataRow appairs two times (one time without DataRow and one time with all DataRow)
Some times, rename a test not refresh test explorer.

[stilldesign]

I'm having the same issue when renaming a test method. New test methods are detected and run correctly. When loading a project after the test window has initialized, then newly discovered tests will just fail to run. Everything worked fine in 2.2.10. I know there have been a lot of changes between this and v3. Hope it can be resolved :) For now, simply restarting Visual Studio gets everything working again.

[MichelZ]

3.0.1 has been released, have you tried those yet?

[Evangelink]

Hey there, we are still investigating the issue but at the moment it doesn't seem that the issue is caused directly by MSTest (it's possible this is linked to an updated dependency to Test Platform - we are looking into it).

[peterwald]

This looks to be an issue with the Test Explorer in Visual Studio. The problem is triggered by some new functionality in MSTest 3, however that is not where the problem lies. We are working on a fix currently and will let you know when we know more.

[gigimignon]

3.0.1 has been released, have you tried those yet?

Ues, I have the last version 3.0.1

[gurkantuna]

Hey there, we are still investigating the issue but at the moment it doesn't seem that the issue is caused directly by MSTest (it's possible this is linked to an updated dependency to Test Platform - we are looking into it).

Hey @Evangelink, I definitely think it's an issue caused directly by MSTest because it still throws exception even though I removed all other dependencies and other TestMethods in my test project.

[Evangelink]

Hi @g-u-r-k-a-n,

Yes issue is caused by MSTest but what I meant and that @peterwald explained is that we have used a new functionality for Test Explorer, through the use of Test Platform Object Model that seems to be causing the issue. So yes it's because you are using MSTest v3 but it's not linked to something we implement but rather either by Test Explorer change of behaviour or by some change on the object model implementation.

Nonetheless, we are investigating the issue and will provide a fix as soon as we can.

[hjrb]

Hi, having the same issue with VS 2022 Enterprise version 17.4.3
For me it works on the same machine (same project) with VS Enterprise Preview Version 17.5.0 Preview 2.0
But after a while it also stops working as described above....

Here is the VS 2022 Preview version info.
Microsoft Visual Studio Enterprise 2022
Version 17.5.0 Preview 2.0
VisualStudio.17.Preview/17.5.0-pre.2.0+33209.295
Microsoft .NET Framework
Version 4.8.04084

Installed Version: Enterprise

ADL Tools Service Provider 1.0
This package contains services used by Data Lake tools

ASA Service Provider 1.0

ASP.NET and Web Tools 17.5.194.32986
ASP.NET and Web Tools

Azure App Service Tools v3.0.0 17.5.194.32986
Azure App Service Tools v3.0.0

Azure Data Lake Tools for Visual Studio 2.6.5000.0
Microsoft Azure Data Lake Tools for Visual Studio

Azure Functions and Web Jobs Tools 17.5.194.32986
Azure Functions and Web Jobs Tools

Azure Stream Analytics Tools for Visual Studio 2.6.5000.0
Microsoft Azure Stream Analytics Tools for Visual Studio

Bundler & Minifier 2.9.3
Adds support for bundling and minifying JavaScript, CSS and HTML files in any project.

C# Tools 4.5.0-2.22580.13+ea0b6351c129c7decd9bf76f4388239509e28e0d
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Clean Solution 1.4.35
Cleans up the solution for the next time you open it. Closes open documents and collapses folders automatically when a solution is being closed.

CleanBinAndObjCommand Extension 1.2.58
CleanBinAndObjCommand Visual Studio Extension Detailed Info

Clear MEF Component Cache 1.4
Clears the Visual Studio MEF component cache to fix issues with cache corruption.

Code alignment 15.1.118.0
Code alignment helps you present your code beautifully, enhancing clarity and readability.

Common Azure Tools 1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Cookiecutter 17.0.22318.1
Provides tools for finding, instantiating and customizing templates in cookiecutter format.

CreateUnitTestBoilerplateCommand Extension 1.0
CreateUnitTestBoilerplateCommand Visual Studio Extension Detailed Info

Entity Framework Core Power Tools 2.5
Adds useful design-time EF Core DbContext features to the Visual Studio Solution Explorer context menu.

Extensibility Message Bus 1.4.1 (main@2ee106a)
Provides common messaging-based MEF services for loosely coupled Visual Studio extension components communication and integration.

File Differ 3.0.43
The easiest way to diff two files directly in Solution Explorer

File Icons 2.7
Adds icons for files that are not recognized by Solution Explorer

IndentRainbowPackage Extension 1.0
IndentRainbowPackage Visual Studio Extension Detailed Info

Markdown Editor v2 2.0.130
A full featured Markdown editor with live preview and syntax highlighting. Supports GitHub flavored Markdown.

Menu Extension 1.0
Menu Visual Studio Extension Detailed Info

Microsoft Azure Hive Query Language Service 2.6.5000.0
Language service for Hive query

Microsoft Azure Stream Analytics Language Service 2.6.5000.0
Language service for Azure Stream Analytics

Microsoft Azure Tools for Visual Studio 2.9
Support for Azure Cloud Services projects

Microsoft JVM Debugger 1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Mono Debugging for Visual Studio 17.5.2 (d15e47f)
Support for debugging Mono processes with Visual Studio.

Node.js Tools 1.5.41130.1 Commit Hash:8e2bd0c47a236f67dd4a99f60aacfa306308387c
Adds support for developing and debugging Node.js apps in Visual Studio

NuGet Package Manager 6.5.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

OpenOnGitHub 1.3
Visual Studio Extension for opening files on GitHub.com

Python - Profiling support 17.0.22318.1
Profiling support for Python projects.

Python with Pylance 17.0.22318.1
Provides IntelliSense, projects, templates, debugging, interactive windows, and other support for Python developers.

Rainbow Braces 1.0.40
Colorizes matching brace pairs to make it easy to identify them and their scope. Works for curly brackets, parentheses, and square brackets.

Razor (ASP.NET Core) 17.0.0.2255201+5530de5c8323cd1ecacb497bbab5637907d0fc02
Provides languages services for ASP.NET Core Razor.

ResX Resource Manager ResXManager
Manage localization of all ResX-based resources in one place. Shows all resources of a solution and lets you edit the strings and their localizations in a well-arranged data grid.

Select Next Occurrence 1.4.182
Selects the next occurrence of the current selection and adds multiple cursors for editing.

SonarLint for Visual Studio 6.9.0.54300
SonarLint is an extension to your favorite IDE that provides on-the-fly feedback to developers on new bugs and quality issues injected into their code.

SQL Server Data Tools 17.2.31208.0
Microsoft SQL Server Data Tools

ToolWindowHostedEditor 1.0
Hosting json editor into a tool window

TypeScript Tools 17.0.11130.2001
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools 4.5.0-2.22580.13+ea0b6351c129c7decd9bf76f4388239509e28e0d
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 17.5.0-beta.22607.4+f5794982b82d308590542bd1e101162dc4f45a13
Microsoft Visual F# Tools

Visual Studio IntelliCode 2.2
AI-assisted development for Visual Studio.

Visual Studio Spell Check Everywhere VSSpellCheckEverywhere
An extension that enables spell checking within any Visual Studio file editor or tool window that uses WPF text boxes.
https://GitHub.com/EWSoftware/VSSpellChecker

Visual Studio Spell Checker VSSpellChecker
An editor extension that checks the spelling of comments, strings, and plain text as you type or interactively with tool windows.
https://GitHub.com/EWSoftware/VSSpellChecker

VisualStudio.DeviceLog 1.0
Information about my package

VisualStudio.Mac 1.0
Mac Extension for Visual Studio

VSColorOutput64 2022.2
Color output for build and debug windows - https://mike-ward.net/vscoloroutput

Word Highlight With Margin Extension 1.0
Highlight all occurrences of a selected word and display carets on the enhanced vertical scrollbar at the right margin.

Xamarin 17.5.0.120 (main@4c08778)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer 17.5.2.64 (remotes/origin/d17-5@627f860876)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin Templates 17.5.3 (3c53880)
Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.

Xamarin.Android SDK 13.1.99.61 (main/2e10309)
Xamarin.Android Reference Assemblies and MSBuild support.
Mono: 6dd9def
Java.Interop: xamarin/java.interop/main@984711a3
SQLite: xamarin/sqlite@fdc1e34
Xamarin.Android Tools: xamarin/xamarin-android-tools/main@fa3711b

Xamarin.iOS and Xamarin.Mac SDK 16.1.1.27 (933c6c2c9)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

[IndigoHealth]

Ok, this is really weird! I added <Nullable>enable</Nullable> to my library project that contains my custom assert routines (e.g., AssertEx.DictionariesAreEqual(...)), and I corrected the resulting warnings and errors. When I tried running the test cases (in VS 2022 Pro v 17.4.3), the Test Explorer panel behaved weirdly. It didn't display the "I'm working on it" icons for my tests, but it seemed to be telling me that everything passed. And it gave me a bunch of errors under the "Errors" notification at the top-right corner of that panel and in the "Tests" output.

Since I read (above) that some folks have resolved the issue by rebooting, I closed VS and rebooted my computer. And... now the problem has magically disappeared.

Don't know if this helps the investigation at all.

[nietras]

I am consistently hitting this issue too in VS 2022 17.5.0 Preview 2.0 with:

    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.4.1" />
    <PackageReference Include="MSTest.TestAdapter" Version="3.0.0" />
    <PackageReference Include="MSTest.TestFramework" Version="3.0.0" />

with .NET SDK 7.0.101 and targetting net7.0. I am not hitting this issue using v2 of MSTest in same VS it seems.

System.InvalidOperationException: TestsForOrigin: Unknown TestCaseRecord.Origin value Unknown
   at Microsoft.VisualStudio.TestStorage.MergedTestGroup.TestsForOrigin(TestCaseOrigin origin)
   at Microsoft.VisualStudio.TestStorage.MergedTestGroup.MarkAsNotRunningAndNotPending()
   at Microsoft.VisualStudio.TestStorage.MergedTestIndex.MarkAsNotRunningAndNotPending(TestCaseOriginKind originKind)
   at Microsoft.VisualStudio.TestStorage.TestStoreIndexSet.MarkTestsAsNotRunningAndNotPending(TestCaseOriginKind originKind)
   at Microsoft.VisualStudio.TestStorage.TestStore.MarkTestsAsNotRunningAndNotPending(TestCaseOriginKind originKind)
   at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.EndTestRun()
   at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.OnTestRunCompleted()
   at Microsoft.VisualStudio.TestWindow.Utilities.EventPumpExtensions.<>c__DisplayClass0_0.<EnqueueAsync>b__1()
   at Microsoft.VisualStudio.TestWindow.Extensibility.TaskCompletionSourceExtensions.RunActionWithCatch[T](TaskCompletionSource`1 taskCompletionSource, Action action)
   at Microsoft.VisualStudio.TestWindow.Logging.ILoggerExtensions.CallInternalWithLogging(IInternalLogger log, Action action, Boolean shouldThrow)

Makes running tests in VS impossible after just a few runs.

[lanwin]

Same problem here. Interestingly the R# Testrunner seems to work.

Restarting VS solves the problem for me. But its annoying.

[Evangelink]

Interestingly the R# Testrunner seems to work.

The error is coming from VS Test Explorer window trying to exploit some new feature enabled in v3 of MSTest. Because this feature is for VS Test Explorer only, it makes sense there is no impact outside of VS.

[bollmanncom]

Same here. Sometimes I can make only one test run before the error occurs.

[mm-ryo]

Same problem. It's only happpend in integration tests. Unit tests are working fine.

Message: 
Test method Test.IntegrationTests._ReturnsExpectedResultAsync threw exception:
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Http.ObjectContent1[[System.Collections.Generic.List1[[, **********, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]'.

Stack Trace: 
HttpContent.CopyToAsync(Stream stream, TransportContext context, CancellationToken cancellationToken)
<b__0>d.MoveNext()
--- End of stack trace from previous location ---
<g__RunRequestAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
ClientHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
CookieContainerHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
**********_ReturnsExpectedResultAsync() line 192

System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.ToLocalTestResultRecord(TestResultRecord testResult)
at System.Linq.Enumerable.WhereSelectListIterator2.MoveNext() at Microsoft.VisualStudio.TestWindow.Host.TestRunSession.AddTestResultRecords(IEnumerable1 testResultRecords)
at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.AddRemoteTestResultRecords(IEnumerable1 testResults) at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.OnTestRunUpdate(IEnumerable1 testResults, IEnumerable`1 activeTests)
at Microsoft.VisualStudio.TestWindow.Host.TestPlatformProvider.TestResponseHandler.<>c__DisplayClass16_0.b__0()
at Microsoft.VisualStudio.TestWindow.Logging.ILoggerExtensions.CallInternalWithLogging(IInternalLogger log, Action action, Boolean shouldThrow)

[Evangelink]

@peterwald @ocallesp Can we have some news on the investigation/fix of this issue?

[KvnSearle]

I started seeing this error when I updated my nuget packages.

MSTest.TestAdapter and MSTest.Framework got up graded to version 3.0.2.

I switched them back to version 3.0.0 and the issue went away.

[LonelyWolf123]

Here's my workaround. Close your solution and close Visual Studio Enterprise, then kill vbcscompiler.exe process.
Once restarted, it works ... but I do this 100 times for day. Very big loose of time, in this way Visual Studio is really unusable.
Hope it will be fixed soon

[peterwald]

@peterwald @ocallesp Can we have some news on the investigation/fix of this issue?

We hear the feedback and are actively working on a fix. It's a bit of a tricky bug/fix and it is taking some time to validate, but we will have a fix shortly.

[LonelyWolf123]

I started seeing this error when I updated my nuget packages.

MSTest.TestAdapter and MSTest.Framework got up graded to version 3.0.2.

I switched them back to version 3.0.0 and the issue went away.

Starting from your suggestion, I made this attempt: use default packages when creating a new test project. But it was not enough to go back to version 3.0.0 of Testframework and Testadapter, I had to do it for all the packages used for testing, including fakes. Then clean solution / projects and rebuild. So far the issue has not recurred.

here's the package versions, the same of a new test project created with VS 17.4.3:

Microsoft.NET.Test.Sdk, Version="17.3.2"
Microsoft.QualityTools.Testing.Fakes Version="16.11.230815"
MSTest.TestAdapter Version="2.2.10"
MSTest.TestFramework Version="2.2.10"
coverlet.collector Version="3.1.2"

[gavinmckay]

I've temporarily switched to using the VSTest console instead (https://learn.microsoft.com/en-us/visualstudio/test/vstest-console-options?view=vs-2022) accessed within VS Enterprise 2022 via:

1. Tools -> Command Line -> Developer Command Prompt
2. vstest.console.exe The.Project.dll /Tests:YOUR_TEST_PREFIX

It's certainly not ideal but better than restarting VS and the debug tools multiple times. Not sure if this will end up with the same issue but reading the above I suspect not as it doesn't use the VS Test dialog at all.

Edit: and I can also still debug the failing unit tests as well via Visual Studio, so it's clunky but I can (a) work out what is failing using the Console and (b) debug to find out why using Visual Studio

[peterwald]

@bvbellomo @Dmytro-Hryshyn-89 @KieranDevvs Can you confirm what version of VS you are running? The 17.4.5 version of VS has not shipped yet. So, unless you have upgraded to a preview version of VS, then it is expected that you would have the same issue.

[KieranDevvs]

@bvbellomo @Dmytro-Hryshyn-89 @KieranDevvs Can you confirm what version of VS you are running? The 17.4.5 version of VS has not shipped yet. So, unless you have upgraded to a preview version of VS, then it is expected that you would have the same issue.

If the fix hasn't shipped yet, then the issue isn't solved thus the ticket should remain open.

[peterwald]

If the fix hasn't shipped yet, then the issue isn't solved thus the ticket should remain open.

There are multiple versions that this fix was shipped in. The 17.5 previews have been shipped for a few weeks now and we've had responses above confirming the fix. As I mentioned earlier, it takes a much more involved process to ship servicing updates to earlier releases and thus there will be a delay until the 17.4 line receives this update.

[StephanMoeller]

I imagine the time loss around the world caused by this bug alone. Do you have any idea when the fix will be part of something non-preview?

[Dmytro-Hryshyn-89]

Hi @peterwald sorry for delay. I am running Microsoft Visual Studio Professional 2022 (64-bit) - Current
Version 17.4.4

[peterwald]

The 17.4.5 release is now publicly available including this fix.

[DrewM-Hax0r]

Thanks, 17.4.5 fixed this issue for me.

[gusarov]

Any intermediate solution for this? I have this from commandline, without VS...

An exception occurred while invoking executor 'executor://mstestadapter/v2': Could not load type 'Microsoft.VisualStudio.TestTools.UnitTesting.StringEx' from assembly 'Microsoft.VisualStudio.TestPlatform.TestFramework, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Stack trace:
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestSettings.PopulateSettings(IDiscoveryContext context)

[TomKuhn]

This is not fixed in Visual Studio 2022 Enterprise V17.5.4

I am using NUNIT, not MSTest

System.InvalidOperationException: TestsForOrigin: Unknown TestCaseRecord.Origin value Unknown
at Microsoft.VisualStudio.TestStorage.MergedTestGroup.TestsForOrigin(TestCaseOrigin origin)
at Microsoft.VisualStudio.TestStorage.MergedTestGroup.get_AllTestCases()
at Microsoft.VisualStudio.TestStorage.MergedTestIndex.AddTestCase(TestCaseRecord testCase, Boolean& isMutationVisibleInHierarchy, IEnumerable1& testsToRemove) at Microsoft.VisualStudio.TestStorage.TestStoreIndexSet.AddToIndexSet(TestBaseRecord record, Boolean isLoadingFromCache) at Microsoft.VisualStudio.TestStorage.TestStore.Add(TestBaseRecord record, Boolean onlyAddIfNew) at Microsoft.VisualStudio.TestWindow.Host.TestRunSession.AddTestCases(ICollection1 testCaseRecords)
at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.AddRemoteTestCaseRecords(IEnumerable1 testCases) at Microsoft.VisualStudio.TestWindow.Host.VsTestRunSession.OnTestsDiscovered(IEnumerable1 testsDiscovered)
at Microsoft.VisualStudio.TestWindow.Logging.ILoggerExtensions.CallInternalWithLogging(IInternalLogger log, Action action, Boolean shouldThrow)

[Evangelink]

@peterwald I will let you handle this.

[peterwald]

This is not fixed in Visual Studio 2022 Enterprise V17.5.4

I am using NUNIT, not MSTest

@TomKuhn, the original issue was due to an interaction between MSTest and the Visual Studio Test Explorer. You must be seeing a different condition that generates a similar error.

Can you help us understand how to reproduce the error with NUnit?

[TomKuhn]

@peterwald It is very similar, in that seemingly randomly, it will fail to execute any tests with the test explorer displaying many errors (exceptions) matching the above post. Only a restart of visual studio will fix it. Sadly I have had to restart VS about 30 times today :( It does not seem to have anything to do with renaming/removing tests, but repeatedly debugging a test and changing source code while debugging seems to make it happen.

Versions:

Visual Studio: 17.5.4
Nunit nuget package: 3.13.3
Nunit3TestAdapter: 4.4.2
.Net Framework 4.7.2

Old-style project format:

... ...

If there's anything else I can provide, I'm happy to do so

Tom

[TomKuhn]

We do use [TestCaseSource(nameof(LanguagesTestData))] a lot which allows us to dynamically create tests based on executing code, which is an Nunit feature that's quite scary!

[peterwald]

We do use [TestCaseSource(nameof(LanguagesTestData))] a lot which allows us to dynamically create tests based on executing code, which is an Nunit feature that's quite scary!

I've been unable to reproduce the problem using data-driven (TestCaseSource) NUnit tests. Is it possible for you to share a skeleton of what your tests look like? Are you using generics in your tests? Unusual or custom test names? What types are the TestCaseSources providing? Any other tips on what operations trigger the error?

[abatishchev]

Shall the issue be split? The original once was about MSTest, the new one is about NUnit, even if similar - it would benefit everyone to track it separately. Agreed?

[Evangelink]

@abatishchev @TomKuhn I agree it would be better to open a separate issue. Test Explorer related issues (TestStore entry in the exception) are usually better tracked at https://developercommunity.visualstudio.com/home but if it's more convenient for you please do open another issue here and we will tag it with IDE.

[phuong-leadsoft]

This issue still here on VS 2022 17.6.2

[nohwnd]

please file that as a feedback to VS, using this little fellow on top of you VS:

[IanKemp]

I'm seeing these exceptions too in 17.6.3, using XUnit. And I haven't renamed anything; literally just opened VS and tried to run tests.

This is a very annoying and disappointing flaw in what is supposed to be a developer productivity product; I've been resisting the switch to Rider that many of my teammates have made, but the fact that JetBrains are at least able to ensure that core functionality doesn't break for no good reason... is becoming more and more difficult to ignore.

[Evangelink]

I have opened the following issue on dev community for you to track and follow - please feel free to add comment to it: https://developercommunity.visualstudio.com/t/SystemInvalidOperationException-and-Sys/10411793.

Closing this ticket as issue is not on Test Platform side.

[olivier-spinelli]

Same issue using (latest version):
Microsoft.NET.Test.Sdk 17.6.3
NUnit3TestAdapter 4.5.0
NUnit 3.13.3

https://developercommunity.visualstudio.com/t/SystemInvalidOperationException-and-Sys/10411793 is closed.
Any change in the context breaks all. The only workaround I found is to:

• close VS
• delete the folder: .vs<proj name>\v17\TestStore.
• restart VS
  Painful (to say the least).
  Didn't find another issue about this. Should I open a new one?

[Evangelink]

@olivier-spinelli the issue is marked as "under investigation" on dev community website (not as closed).

Ping @AbhitejJohn @peterwald

[olivier-spinelli]

Glad to hear that... But why, being logged in, can't I see it? (Subscription "Visual Studio Enterprise (MPN)")

Viewing as ol***[email protected]
We were unable to get this feedback item. It could be because you don't have access to it or it doesn't exist

Error details: ["Request failed with status code 403"]

[Evangelink]

Oh thanks for the info! I logged in with my personal account and get the same error as you but I can see it correctly with work account. I will check with colleagues what's going-on and will get back to you!

[Evangelink]

@olivier-spinelli the issue is now public! I didn't pay attention that I created it as internal.

[olivier-spinelli]

Many thanks!