logo1

logo

Comparing Microsoft Moles in VS2010 to Microsoft Fakes in VS11

0 Comments
Posted in Unit Testing

In my previous post: Using Stubs and Shims to Test with Microsoft Fakes in Visual Studio 11 I went over Microsoft Fakes (a mocking framework built into Visual Studio 2011) in detail.  During my research I ran into Microsoft’s Moles Isolation Framework, which is the prior version of Microsoft Fakes that’s been in Microsoft Research for a few years. 

Moles is a lightweight framework for test stubs and detours in .NET that is based on delegates. Moles may be used to detour any .NET method, including non-virtual/static methods in sealed types. Moles is freely available on Visual Studio Gallery or bundled with Pex.

Just as with Microsoft Fakes, Moles provides a way to detour any .NET method as well as generally accepted basic Mocking functionality. The real interesting part about Moles is that it happens to be compatible with both Visual Studio 2010 and 2008. As I mentioned in the previous post there are only a handful of products that can detour .NET methods in VS 2010: TypeMock’s Isolator, Telerik’s JustMock and Microsoft’s Moles, costing: $799, $299 and free respectively.

I've been looking into “future proofing” some of our new testing initiatives and perhaps using Moles to bridge the gap between today’s testing and tomorrow’s tools. The Moles roadmap points out that Microsoft Fakes is the next generation of Moles and that a Moles to Fakes conversion will require a few modifications.

The Fakes Framework in Visual Studio 11 is the next generation of Moles & Stubs, and will eventually replace it. Fakes is different from Moles, however, so moving from Moles to Fakes will require some modifications to your code. A guide for this migration will be available at a later date.

That raises a few questions:

  • How different are the two versions?
  • How many modifications will be required for conversion?
  • Is it “safe” to use Moles in existing projects?

Under the covers things appear to have been refactored significantly for inclusion in VS 11. It’s hard to get a feel of how much movement there’s been since compare tools tend to breakdown when namespaces and class names are changed dramatically. After looking through the structures of both Moles and Fakes with ILSpy it’s pretty obviousthat the dev teams on this have been quite busy.

Luckily enough though, after a decent amount of usage it looks as though Moles is nearly identical to Microsoft Fakes in terms of general usage. With that in mind, the migration, even if it is a manual one, doesn’t look to be too bad and appears relatively straight forward. Despite the simplicity of conversion I’ll be holding out hope for an automated conversion out of the box or as a separate tool. 

The main user facing changes from Moles to Fakes are:

  • Fakes refers to a detour as Shim, instead of as a Mole
  • Generated classes that were prefixed with “M” for Mole  or “S” for Stub are instead prefixed with “Shim” and “Stub” respectively
  • The HostType attribute is no longer necessary
  • Some assembly attributes may no longer be necessary
  • .moles configuration files are now .fakes configuration files
  • Some configuration file options have been removed

With that said here’s a quick overview of installation, usage and comparisons detailing the similarities, reusing examples from my previous post on Microsoft Fakes.

Note: this post is a little lighter on the details of what Microsoft Fakes and Moles are and why you’d want to use it. If you find yourself wanting a little more background information and/or haven’t read my previous post on the subject, then you may want to take some time and check it out first

Installing Moles

Moles is available through Visual Studio’s Extension Manager. This eventually leads you to the product page in the Visual Studio Gallery. 

ExtensionManager

Search for Moles.

ExtensionManager2

Download and start the installer. Note: not all steps are shown here; the install is rather straight forward.

 MolesInstall1

Pick the typical setup type as custom doesn’t really offer anything you’d want to change anyway.

MolesInstall1b

Quick, easy, done.

MolesInstall1c

Now we’re ready for some “Moling” action!  Hmmm… perhaps it was for the better that they renamed this in Visual Studio 11.

Moling Example & Comparison

First off we have a little bit of boilerplate code to setup. Aside from the different namespaces and 1 different class name, this is essentially the same trivial cart example as the previous post, so feel free to skip ahead if your interested in the more “juicy” bits.

In Visual Studio 2010 create a C# Class Library project called MolesExample.  Rename the default class file to CartToMole and modify the code to be as follows:

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace MolesExample
{
    public class CartToMole
    {
        public int CartId { get; private set; }
        public int UserId { get; private set; }
        private List<CartItem> _cartItems = new List<CartItem>();
        public ReadOnlyCollection<CartItem> CartItems { get; private set; }
        public DateTime CreateDateTime { get; private set; }

        public CartToMole(int cartId, int userId)
        {
            CartId = cartId;
            UserId = userId;
            CreateDateTime = DateTime.Now;
            CartItems = new ReadOnlyCollection<CartItem>(_cartItems);
        }
 
        public void AddCartItem(int productId)
        {
            var cartItemId = DataAccessLayer.SaveCartItem(CartId, productId);
            _cartItems.Add(new CartItem(cartItemId, productId));
        }
    }
}

Add the CartItem class.

namespace MolesExample
{
    public class CartItem
    {
        public int CartItemId { get; private set; }
        public int ProductId { get; private set; }

        public CartItem(int cartItemId, int productId)
        {
            CartItemId = cartItemId;
            ProductId = productId;
        }
    }
}

Add the DataAccessLayer class. Again, don’t worry about the connection string, we’ll never end up hitting it anyway.

using System.Data;
using System.Data.SqlClient;

namespace MolesExample
{
    public static class DataAccessLayer
    {
        public static int SaveCartItem(int cartId, int productId)
        {
            using (var conn = new SqlConnection("RandomSqlConnectionString"))
            {
                var cmd = new SqlCommand("InsCartItem", conn);
                cmd.CommandType = CommandType.StoredProcedure;
                cmd.Parameters.AddWithValue("@CartId", cartId);
                cmd.Parameters.AddWithValue("@ProductId", productId);

                conn.Open();
                return (int)cmd.ExecuteScalar();
            }
        }
    }
}

Next add a Unit Test Project called MolesExample.Tests and rename the default unit test class file to CartToMoleTests.

In the MolesExample.Tests project add a reference to the MolesExample project.  You’re solution should look as follows:

InitialSolutionConfig

Right click on the MolesExample reference in MolesExample.Tests references and select the “Add Moles Assembly” option.  Note: Screen captures of the Fakes version are positioned either to the right or bottom depending on your available screen width.

AddFakesAssembly  AddFakesAssembly

This adds a couple of new items to the project. You may need to refresh to see the new items as they didn’t come up right away for me.

AfterFakes  AfterFakes

The new references add the following types. Also take note of how the namespacing between Moles and Fakes has changed.

NewTypes  NewTypes2

The generated .Moles/.Fakes assembly type naming has only changed slightly between versions, which is what we’ll be working with for the most part. 

VS 10VS 11
MCartItem ShimCartItem
MCartItem.AllInstances ShimCartItem.AllInstances
MCartToMole ShimCartToShim
MCartToMole.AllInstances ShimCartToShim.AllInstances
MDataAccessLayer ShimDataAccessLayer
SCartItem StubCartItem
SCartToMole StubCartToShim

The .moles configuration file has been appropriately renamed to .fakes and the internals renamed accordingly.  I’ll expand out the structure of the .moles file later on in the post and compare that to the expanded .fakes configuration file.  For now here are the differences between them in terms of our two projects:

<Moles xmlns="http://schemas.microsoft.com/moles/2010/">
  <Assembly Name="MolesExample" />
</Moles>
<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/">
  <Assembly Name="FakingExample"/>
</Fakes>

Now, let’s create our first test method using a Mole.  Just as before we’re going to unit test the AddCartItem method. AddCartItem calls SaveCartItem on DataAccessLayer, which happens to be static.  We’re going to mock out that database call to isolate the logic in AddCartItem in our unit test. Add the code below to CartToMoleTests.

using Microsoft.Moles.Framework;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace MolesExample.Tests
{
    [TestClass]
    public class CartToMoleTests
    {
        [TestMethod, HostType("Moles")]
        public void AddCartItem_GivenCartAndProduct_ThenProductShouldBeAddedToCart()
        {
            //Create a context to scope and cleanup moles
            using (MolesContext.Create())
            {
                int cartItemId = 42, cartId = 1, userId = 33, productId = 777;

                //Mole SaveCartItem rerouting it to a delegate which 
                //always returns cartItemId
                Moles.MDataAccessLayer.SaveCartItemInt32Int32 = (c, p) => cartItemId;

                var cart = new CartToMole(cartId, userId);
                cart.AddCartItem(productId);

                Assert.AreEqual(cartId, cart.CartItems.Count);
                var cartItem = cart.CartItems[0];
                Assert.AreEqual(cartItemId, cartItem.CartItemId);
                Assert.AreEqual(productId, cartItem.ProductId);
            }
        }
    }
}

Here’s the class file we used in the Microsoft Fakes example for reference.  The only real differences aside from namespacing are on lines 9 (Moles HostType is no longer needed), 13 (context name change) and 19 (actual detour) and all are quite minor.

using Microsoft.QualityTools.Testing.Fakes;
using Microsoft.VisualStudio.TestTools.UnitTesting;
 
namespace FakingExample.Tests
{
    [TestClass]
    public class CartToShimTests
    {
        [TestMethod]
        public void AddCartItem_GivenCartAndProduct_ThenProductShouldBeAddedToCart()
        {
            //Create a context to scope and cleanup shims
            using (ShimsContext.Create())
            {
                int cartItemId = 42, cartId = 1, userId = 33, productId = 777;
 
                //Shim SaveCartItem rerouting it to a delegate which 
                //always returns cartItemId
                Fakes.ShimDataAccessLayer.SaveCartItemInt32Int32 = (c, p) => cartItemId;
 
                var cart = new CartToShim(cartId, userId);
                cart.AddCartItem(productId);
 
                Assert.AreEqual(cartId, cart.CartItems.Count);
                var cartItem = cart.CartItems[0];
                Assert.AreEqual(cartItemId, cartItem.CartItemId);
                Assert.AreEqual(productId, cartItem.ProductId);
            }
        }
    }
}

Now let’s run it in Unit Test Explorer and see it pass.

TestrunAfterMole

Just as before, our mole/detour completely skipped over the SaveCartItem method in DataAccessLayer avoiding the exception due to “RandomSqlConnectionString” clearly not being a valid connection string. Start to finish, Moles ends up being just as easy to setup and use as Fakes for our simple cart example.

Stubs Example and Comparison

As with the Mole example above, there’s some boilerplate we need to setup before we get to the stub test.  Again, there are very few changes here, so if you’ve seen these before feel free to skip ahead.

We’ll need to create an ICartSaver interface and inject/pass it into the CartToStub object.  At which point we can stub out ICartSaver and have it return 42 just as we did for our Moling example. 

Add a new interface ICartSaver to the MolesExample project.

namespace MolesExample
{
    public interface ICartSaver
    {
        int SaveCartItem(int cartId, int productId);
    }
}

Next up, for completeness (although not necessary, since we’ll be stubbing around it), add a CartSaver class to the MolesExample project so we can replicate the implementation of DataAccessLayer.

using System.Data;
using System.Data.SqlClient;

namespace MolesExample
{
    public class CartSaver : ICartSaver
    {
        public int SaveCartItem(int cartId, int productId)
        {
            using (var conn = new SqlConnection("RandomSqlConnectionString"))
            {
                var cmd = new SqlCommand("InsCartItem", conn);
                cmd.CommandType = CommandType.StoredProcedure;
                cmd.Parameters.AddWithValue("@CartId", cartId);
                cmd.Parameters.AddWithValue("@ProductId", productId);

                conn.Open();
                return (int)cmd.ExecuteScalar();
            }
        }
    }
}

Create a new CartToStub class as follows similar to CartToMole but allowing for manual dependency injection of ICartSaver in the constructor:

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace MolesExample
{
    public class CartToStub
    {
        public int CartId { get; private set; }
        public int UserId { get; private set; }
        private List<CartItem> _cartItems = new List<CartItem>();
        public ReadOnlyCollection<CartItem> CartItems { get; private set; }
        public DateTime CreateDateTime { get; private set; }
        private ICartSaver _cartSaver;

        public CartToStub(int cartId, int userId, ICartSaver cartSaver)
        {
            CartId = cartId;
            UserId = userId;
            CreateDateTime = DateTime.Now;
            _cartSaver = cartSaver;
            CartItems = new ReadOnlyCollection<CartItem>(_cartItems);
        }

        public void AddCartItem(int productId)
        {
            var cartItemId = _cartSaver.SaveCartItem(CartId, productId);
            _cartItems.Add(new CartItem(cartItemId, productId));
        }
    }
}

Moving back over to the MolesExample.Tests project, add a new unit test file named CartToStubTests.  Create/Modify the AddCartItem_GivenCartAndProduct_ThenProductShouldBeAddedToCart method to take into account the new changes.

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace MolesExample.Tests
{
    [TestClass]
    public class CartToStubTests
    {
        [TestMethod, HostType("Moles")]
        public void AddCartItem_GivenCartAndProduct_ThenProductShouldBeAddedToCart()
        {
            int cartItemId = 42, cartId = 1, userId = 33, productId = 777;
 
            //Stub ICartSaver and customize the behavior via a 
            //delegate, to return cartItemId
            var cartSaver = new Moles.SICartSaver();
            cartSaver.SaveCartItemInt32Int32 = (c, p) => cartItemId;
 
            var cart = new CartToStub(cartId, userId, cartSaver);
            cart.AddCartItem(productId);
 
            Assert.AreEqual(cartId, cart.CartItems.Count);
            var cartItem = cart.CartItems[0];
            Assert.AreEqual(cartItemId, cartItem.CartItemId);
            Assert.AreEqual(productId, cartItem.ProductId);
        }
    }
}

Here’s the class file we used in the Microsoft Fakes example for reference.  Differences are on lines 8 (Moles HostType) and 15 (generated Stub type name) and are quite minimal.

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace FakingExample.Tests
{
    [TestClass]
    public class CartToStubTests
    {
        [TestMethod]
        public void AddCartItem_GivenCartAndProduct_ThenProductShouldBeAddedToCart()
        {
            int cartItemId = 42, cartId = 1, userId = 33, productId = 777;

            //Stub ICartSaver and customize the behavior via a 
            //delegate, ro return cartItemId
            var cartSaver = new Fakes.StubICartSaver();
            cartSaver.SaveCartItemInt32Int32 = (c, p) => cartItemId;

            var cart = new CartToStub(cartId, userId, cartSaver);
            cart.AddCartItem(productId);

            Assert.AreEqual(cartId, cart.CartItems.Count);
            var cartItem = cart.CartItems[0];
            Assert.AreEqual(cartItemId, cartItem.CartItemId);
            Assert.AreEqual(productId, cartItem.ProductId);
        }
    }
}

Run the unit tests once more to see that everything’s passing.

TestrunAfterStub

Just as with the Moling example, the code is very similar to what would be used in Microsoft Fakes.

.Moles and .Fakes Xml Configuration File Comparison

The changes in the .fakes xml configuration file may require more effort to convert depending on what features were used. Basic usage as in generated but never modified shouldn’t be a problem. However, there are a handful of attributes which could be problematic to convert, since they’ve been removed. I’ve included both files expanded out via intellisense, with a description of the removed elements.

<Moles xmlns="http://schemas.microsoft.com/moles/2010/">
  <Assembly Name="" ExportedTypes="" Location="" ReflectionOnly="" x86="" />

  <CodeStyle AssemblyAlias="" DisableUniqueAlias="" MaxIdentifierLength="">
    <Copyright></Copyright>
    <FileHeader></FileHeader>
  </CodeStyle>

  <StubGeneration Disable="" SkipVirtualIndexers="" SkipVirtualMethods="">
    <Types>
      <Clear />
      <Add AbstractClasses="" Classes="" FullName="" Interfaces="" Namespace="" TypeName=""/>
      <Remove AbstractClasses="" Classes="" FullName="" Interfaces="" Namespace="" Obsolete="" TypeName=""/>
    </Types>
    <TypeFilter /> <!-- Deprecated -->
  </StubGeneration>

  <MoleGeneration Disable="">
    <Types>
      <Clear />
      <Add FullName="" Namespace="" TypeName="" />
      <Remove FullName="" Namespace="" Obsolete="" TypeName=""/>
    </Types>
    <TypeFilter /> <!-- Deprecated -->
  </MoleGeneration>

  <Compilation AssemblyName="" Debug="" DisableCodeContracts="" KeyFile="" ProjectTemplate="">
    <COMReference Guid="" VersionMajor="" VersionMinor="" WrapperTool="" />
    <Property Condition="" Name="" />
  </Compilation>
</Moles>

Here’s the .fakes file intellisense expansion for reference. 

<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/">
  <Assembly Name="" Location="" Version="" x86=""/>

  <StubGeneration Disable="" SkipVirtualIndexers="" SkipVirtualMethods="">
    <Clear />
    <Add AbstractClasses="" FullName="" Interfaces="" Namespace="" TypeName=""/>
    <Remove AbstractClasses="" FullName="" Interfaces="" Namespace="" Obsolete="" TypeName=""/>
  </StubGeneration>

  <ShimGeneration Disable="">
    <Clear />
    <Add FullName="" Namespace="" TypeName=""/>
    <Remove FullName="" Namespace="" Obsolete="" TypeName=""/>
  </ShimGeneration>

  <Compilation Debug="" DisableCodeContracts="" KeyFile="" ProjectTemplate="">
    <COMReference Guid="" VersionMajor="" VersionMinor="" WrapperTool=""/>
    <Property Condition="" Name="" />
  </Compilation>
</Fakes>

Here’s an overview of the changes from Moles to Fakes:

  • The ExportedTypes attribute (.moles line 2) was removed from the Assembly element.

    Per the schema ExportedTypes is described as “Reflect over exported (visible outside the assembly) types only”. The only examples I can find for its usage are in moling the System assembly here and here. We’ll have to wait for the migration guide to see what is a suitable replacement for this behavior.
  • The CodeStyle element (.moles line 4) was removed.

    Per the schema CodeStyle is described as “Code generation settings”. Aside from the Copyright and FileHeader elements the child attributes generally looked like debug switches.  From the docs available at Microsoft Research there isn’t a whole lot of detail on when to use these settings or what purpose they serve.  Nobody’s going to miss these.
  • The Types elements (.moles lines 7 & 16) have been removed and everything below has shifted up one level. Sounds good to me.
  • The Classes attributes (.moles lines 9 & 10) were removed from the Add & Remove elements in StubGeneration.

    Per the schema Classes is described as “An optional filter to filter non-abstract classes”. Why this one was removed but not the inverse (AbstractClasses) is anyone’s guess. I imagine you can achieve the desired results with the other filters, so this could require a little more effort to update if you make use of this it.
  • The AssemblyName attribute (.moles line 24) was removed from the Compilation element.

    Per the Schema AssemblyName is described as “Specifies the assembly name of the generated assembly”. This attribute only applies when compilation is disabled”. Again, no documentation available, no examples of usage, probably not going to miss it.

Additional Considerations for System & Mscorlib

Previously in Moles when adding a Moles assembly for mscorlib or System, Moles required a little bit of jury rigging.  Assembly attributes may be required in some cases.  See this post at social.msdn.microsoft.com (code below quoted from post) and this one from StackOverflow for more info.

using System;
using System.Moles;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Microsoft.Moles.Framework;
[assembly: MoledType(typeof(System.DateTime))]

namespace Y2kMoles
{
    [TestClass]
    public class Y2kTest
    {
        [TestMethod]
        [HostType("Moles")] // add this attribute
        public void Y2k()
        {
            MDateTime.NowGet = () => new DateTime(2000, 1, 1);

            if (DateTime.Now == new DateTime(2000, 1, 1))
                throw new Exception("y2k bug!!!!");
        }
    }
}

Even with these changes I was still getting build errors. Supposedly there are issues related to the installation of SP1. See this post on StackOverflow for more info.

BuildErrors

I ended up modifying the mscorlib.moles file using the type filters to only include DateTime and this fixed everything up.

<Moles xmlns="http://schemas.microsoft.com/moles/2010/">
  <Assembly Name="mscorlib" />
  <StubGeneration Disable="true" />
  <MoleGeneration>
    <Types>
      <Clear />
      <Add TypeName="DateTime" />
    </Types>
  </MoleGeneration>
</Moles>

That’s a slightly suboptimal user experience for such a common case. Microsoft Fakes seems to have fixed these issues. Adding a Fakes assembly to System also adds one for mscorlib and everything works just fine out of the box, which is the expected user experience. So during conversion to Fakes there may be some “dehacking” involved.

Conclusion

It would seem that Microsoft Moles is quite capable and so similar in terms of usage to Microsoft Fakes that using it in the interim until Visual Studio 11 is released is a simple decision. The functionality gained as well as future proofing ongoing development is well worth any drawbacks related to migration or some of Moles’ current issues. There will be some conversion effort involved (as the only conversion plans mentioned so far, are a guide for manual conversion steps), but it will be relatively simple and straight forward. Learn it, use it, love it!

 

The code for this post is available on GitHub.

  • 0 Comments

Pingbacks and trackbacks (1)+

Comments are closed