Class 65k tests is disproportionally slow

[nohwnd]

Creating a class with 65k tests (limit of max methods per class is just a smidge above that), results in the run being really slow.

Class with 10k empty tests runs in about 4 seconds. (Even if there are 9999 filters).

Class with 65k tests run over 4 minutes.

This does not seem to be related to filtering, it takes this long even without any filters applied.

I did not check if this happens if the total amount of tests are over a given limit. Single class with more than 10k tests is unlikely, but run with 20k tests in total is not as unlikely.

AB#1865125

[nohwnd]

Added to sprint for quick test if someone can see what is going on. We mostly should optimize for test runs of 1-100 tests, and discovery of 10-1000 tests, so this is quite a bit out of that scope, but still might be worth it if obvious problem is found.

[nohwnd]

After the perf changes in mstest this is still not linear, but it improved a lot.

[nohwnd]

The slowness seems to be caused by the lookup of method in a huge class being slow:

When we spread the tests into classes with 20 tests per class, or even 1000 tests per class the perf is normal again. So imho this is just a denegerate case.

[nohwnd]

Measures between the perf before using reflection cache for discovery and run, and after (between 3.3 and 3.5):