Upgrading to Microsoft Test Manager 2012 (MTM)

It’s surprisingly hard to find what you will gain by upgrading to a new version of Microsoft Test Manager.

In this blog post we’ll cover MTM 2012. For MTM 2013 go here.

I’ve managed to track down some blog posts and received a few links from Microsoft. Below is a summary of what I’ve found. Please get in touch if you have any additional info you found interesting / driving in your upgrade adventure.

MTM 2010 to MTM 2012 RTM

• Compatibility of Microsoft Test Manager with Visual Studio 2010
• Manual Testing Windows Store Apps
• Enhanced Action Logs for Windows Store Apps
• Exploratory Testing window
• Manual Test Steps Can Include Multiple Lines
• Manual Tests Includes Rich Text
• Microsoft Test Manager Test Plan Results
• Cloning Test Suites into Other Plans for New Iterations
• MTM Performance improvements

MTM 2012.1

• Pause and Resume in MTM
• Populate test suites using hierarchical queries in MTM
• Edit test cases during execution in MTM
• Automatic updates in MTM
• Other enhancements in MTM

Here’s what I pulled from the KB Article

• Code coverage for web applications in MTM
  Testers can now know how much of the application code is being covered by test cases executed within MTM, and developers can now know whether their code is being accessed by the application and is a valid test case.
• Cross browser testing support for coded UI test
  You can use coded UI tests help automate browser based applications that are built on Internet Explorer, Firefox and Chrome.
• Create an image action log from exploratory tests
• Populate test suites using hierarchical queries in MTM
• Directly edit test case properties from the Test Runner of MTM
• Data collectors populate Trait information for Visual Studio unit test grouping
  You can now organize and filter the test runs based on several conditions in Test Explorer. Traits is a new concept, a common denominator for several underlying terms: Test Category, Test Property, Priority, and Owner.
• The ability to pause and resume manual test sessions in MTM
• Automatic updates for MTM and Microsoft Feedback Client
• Usability improvements for coded UI testing
• No need to manually install SQL Server Express for load testing (By default, use LocalDB)
• Use command-line functionality to deep copy test plans to better support release management
• Publish test results to Team Foundation Server (TFS) from the command line

MTM 2012.2

• Test Plan/Test Suite Cloning in MTM
• Light-weight Browser based Test management & Execution
• Customization of test result fields and marking test results as NA in MTM/Web

Here’s what I pulled from the KB Article

Microsoft Test Manager

• Assume that you run a test that has no expected results, and you file a bug during the test. In this situation, the test is auto-computed to pass.
• A shared test step may be deleted unexpectedly when you delete a selected test step.
• The following error occurs when you try to create a new environment in MTM Lab Center:

Error in querying the running virtual machines.
Specified argument was out of the range of valid values.

Unit testing

• Test methods with TestInitialize attributes may be called in an incorrect sequence.

Load Testing

• Unable to run an x86-targeted test DLL from MTM against a Test Controller that is running on an x64 operating system.

Test and Lab Manager

• Assume that you have applied Visual Studio 2012 Update 1 on a computer that has TFS 2012 and Visual Studio 2012 installed. You try to run a build that contains tests on the computer. In this situation, the build fails together with a MethodNotFoundException 
• Merged code coverage files display incorrect coverage data.

MTM 2012.3

Here’s what I pulled from the KB Article

Web test

• Microsoft Visual Studio 2010 crashes when you create a Web test on a computer that has both Visual Studio 2010 Service Pack 1 (SP1) and Visual Studio 2012 Update 2 installed.

Coded UI test

• You cannot run coded UI together with the cross browser testing feature by using Selenium components.

Load test

• When you analyze a load test run, parts of the results are missing from the graph in the default view.
• A fix that is described in the following KB article is included:

2769120 FIX: IP address of a virtual user changes when you run a load test if IP switching is enabled in Visual Studio 2010

Unit test

• 783438: Mstest.exe fails to execute unit because of missing assemblies
• Assume that you use theExpectedSqlException attribute in the unit test code in SQL Server Data Tools for Visual Studio 2012. When you run the unit test, the ExpectedSqlException attribute does not work correctly.
• When you install Visual Studio 2012 Update 2 on a build computer that has the RTM version of build agent installed, tests cannot run, and the build operation fails with the following error message:

‘Method not found: ‘Void Microsoft.VisualStudio.TestPlatform.Utilities.ClientUtilities.InitializeTestPlatform(Microsoft.VisualStudio.TestPlatform.ObjectModel.Client.ITestPlatform, System.String, System.Collections.Generic.List`1)’.’

Test Case Manager (Web Access)

• 783802: You cannot access test plans by using Web access if custom work item fields are configured

Microsoft Test Manager

• Memory leak occurs when you perform Exploratory Testing by using Microsoft Test Manager.

MTM 2012.4

Here’s what I pulled from the KB Article

Test and Lab Manager

• When you set up lab management in Team Foundation Server 2012 by configuring a connection to SCVMM 2012 R2 server, the setup is not successful, and you receive the following error message:

“Get-MachineConfig” command let doesn’t exist

• When you try to clone an unfenced environment and deploy it, you receive the following error message:

To use this environment, you must install a compatible test agent in all the machines of the environment. Click ‘install agents’ to complete this task.

• In a Build-Deploy-Test (BDT) workflow, a large number of web-service calls are made for refreshing the test run on a hosted service. This causes many unnecessary web-service calls.

Unit Test

• Unit test explorer in Visual Studio 2012 cannot discover windows store tests when .NET Framework 4.5.1 is installed.
• Users that have special characters (such as “-” in the beginning of the name) in the machine name are not able to discover and run tests.
• Visual Studio intermittently crashes when you try to discover tests by Test Explorer.

Web Test

• Issue 1:
  You cannot record or playback actions involving navigation on a page on Internet Explorer 10 or on Internet Explorer 11 together with the latest IE GDR updates installed.Issue 2:
  You cannot playback the scenario on a page together with AJAX as expected, the playback does not complete. For example, when you clicks attach file, select file and playback recorded steps, the file upload starts but never completes.
• You cannot record a web performance test in Internet Explorer 11 because the Enhanced Protection Mode is enabled by default in Internet Explorer 11.

Coded UI Test and Action Recordings

• You cannot record or playback actions that involve navigation on a page on Internet Explorer 10 or on Internet Explorer 11 when Internet Explorer updateKB 2870699 is installed.
• You cannot play back tests on a web page that uses AJAX, because the playback does not complete. For example, when you click attach file,select a file to upload, or click upload in a test, the file upload starts but never completes during playback.

Test Case Manager

• Error occurs when you publish the test result in TFS 2012 environment by using MSTest 2010.
• Assume you associate a build with a test plan. When you run a test case from web access, the test runs in Analyze test run tab does not show the build number.
• You could not select custom long text fields in Microsoft Test Manager (MTM) 2012 and Test Case Management (TCM) web grids.

Microsoft Test Manager

• When you create a new SCVMM-based lab environment in Microsoft Test Manager, a product key is not required during Sysprep for some operating systems (such as Windows Server 2012 R2). However, you is still prompted for a key, and therefore cannot proceed.

That’s it… up next is MTM 2013.

Mike