xUnit.net and running multiple test assemblies

xUnit.net is a unit testing framework for the .Net platform designed with inspired by some nice ideas like reducing custom attributes, one instance per method , etc. The Internet is full of blog posts and articles about xUnit. In this link you can find a comparison of xUnit and other popular unit testing frameworks.

For the CI I use MSBuild script that compiles, runs test, generates documentations, etc.  For test projects I have the habit of postfixing them with "Tests”  to simplify fetching all test dlls in MSBuild scripts using wild cards (Who dares to claim that Convention over Configuration is new in the .Net world? ). Running all NUnit tests in some project would look like :

<CreateItem Include="tests\*\bin\Debug\*.Tests.dll">
  <Output TaskParameter="Include" ItemName="TestAssembly" />
</CreateItem>
<NUnit Assemblies="@(TestAssembly)" ToolPath="$(NUnit-ToolPath)" />

Adding new test project will require me to add nothing to the build script. I have just to follow the convention.

Unfortunately this is not possible with xUnit task because it accepts only one assembly. Thus I have to specify each assembly separately. The xml result will be also split. @bradwilson nicely wrote me via  Twitter, that I can use a project file to specify multiple assembly.  This, while solving the second problem, doesn’t enable you to use wild cards. You have to add each assembly manually to the project.

Gallio is a tool  or, may better a framework, for running many test frameworks in the same way and provides as unique results format. It integrates also very well with some build servers.

To integrate Gallio in my MSBuild script I downloaded the bits from here and extracted them to the tools folder inside my project root folder.

Running the tests was as simple as

<UsingTask AssemblyFile="$(ToolPath)\gallio\Gallio.MSBuildTasks.dll"
	TaskName="Gallio" />

<Target Name="Test" DependsOnTargets="Build">
	<Message Text="==== Starting Gallio to run tests ===" />
	
	<CreateItem Include="tests\*\bin\Debug\*Tests.dll">
      <Output TaskParameter="Include" ItemName="TestAssemblies" />
    </CreateItem>
    <Gallio RunnerExtensions="TeamCityExtension,Gallio.TeamCityIntegration"
        Assemblies="@(TestAssemblies)"  /> 
</Target>

RunnerExtensions is an extension for Gallio to provide TeamCity with status messages. Integrating Gallio with TeamCity will be discussed in a following post.

NUnit   and running multiple assemblies from the console

Even though NUnit MSBuild task supports multiple assemblies, if you want to use NCover with NUnit  you have to do some manual work to enable NCover running NUnit console tool. The @() operator of MSBuild join all paths with a semicolon. On the other hand NUnit console runner expect all assemblies to be separated with a white space. Fortunately you can tell @() how it should concatenate the paths.

<NCover ToolPath="$(NCoverPath)"
        CommandLineExe="$(NUnit-ToolPath)\nunit-console.exe"
        WorkingDirectory="."
        CommandLineArgs="@(TestDll->'%(Identity)', ' ')"
        CoverageFile="$(CoverageFile)"
        LogFile="$(CoveragePath)\Coverage.log"
        AssemblyList="@(CoverageDll)" />

In the fourth line @ concatenate the relative path of each file using a white space. You may need to surround each path with a quote if it contains white spaces.