Incorrect format for TestCaseFilter Error: Invalid Condition - Using = in SetName

[devbcjohn]

NUnit

• 3.12.0

NUnit3TestAdapter

• 3.16.1

Visual Studio

• Professional 2019
• Version 16.4.3

Project under test

• Console Application
• .NET Core 3.1

Project performing tests

• .NET Core 3.1

Application to repo

• NUnitTestAdapterCharacters.zip

When using TestCaseData and SetName. Putting an = in the test causes error in Test Output. This also causes all other tests to not be run using Test Explorer in Visual Studio.

Example:
yield return new TestCaseData("Input B").SetName("Unit To Test Input B = itself");

Full Error Message

An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error: Invalid Condition 'FullyQualifiedName=UnitTests.Tests.Unit To Test Input B = itself'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.

This occurs in other framework setups as well but the repo project attached is using the versions listed above.

This looks like it could be similar to #549

[OsirisTerje]

Does it work if you replace = with == ?

[devbcjohn]

No it doesn't work in that case either. Also it appears ! causes issues as well.

I also tried > and < those do not appear to present a problem.

Using ==

An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error: Invalid Condition 'FullyQualifiedName=UnitTests.Tests.Unit To Test Input B == itself'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.

Using !

An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error: Invalid Condition 'FullyQualifiedName=UnitTests.Tests.Unit To Test Input B !itself'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.

[aolszowka]

I am still able to get this to reproduce, using the above zip file, with the following versions:

Visual Studio 2019 Enterprise 16.5.4
NUnit 3.12.0
NUnit3TestAdapter 3.17.0-beta.1

The error is:

---------- Starting test run ----------
NUnit Adapter 3.17.0.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error: Invalid Condition 'FullyQualifiedName=UnitTests.Tests.Unit To Test Input B = itself'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
========== Test run finished: 0 Tests run in 525.5 ms (0 Passed, 0 Failed, 0 Skipped) ==========

I started to make a minimized test case, however the reproduction of the issue became very flaky, sometimes this snippet of code would cause failures:

    [TestFixture]
    public class TestFilterBug
    {
        [TestCase(TestName = "Invalid = TestNameEquals")]
        public void ReservedNames()
        {
            Assert.True(true);
        }
    }

It seemed like you needed at least 10 characters after the =, but then it would start working.

Any idea where to start digging?

[aolszowka]

Sorry to double post; but this may be a dupe of #691

[4oooo4]

same issue, any updates?

[smildenberger]

I just noticed the same issue with == or != being in SetName(). I could run the individual tests but if they were included in a set of tests to run then no tests were run. Test Adapter 3.17.0 in VS 2019 16.9.5

[OsirisTerje]

It seems this works now, using the latest VS2019, version 16.11.3. I have added the repro to the repro-repo together with the same for #691 and #549 which are similar to this one. All works. See repro solution here: https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue711

Note: This works without Real-Time Test Discovery, but fails when Real-Time Test Discovery is turned on.