I am currently working on a project with a pluggable application (load plugins at runtime and execute them) that loads assemblies at runtime, using the Assembly.LoadFile() method. I wanted to test it using NUnit (feel free to read our 3 Ways to Run NUnit from Visual Studio post). Although, the code was functioning very well it always failed during the NUnit tests, always! It took me several days to understand what went wrong and I want to share with you my findings.

First, lets take a look at this very simple plugin code:

   1: // The plugin interface as defined by the pluggable application.
   2:  public interface ITinyPlugin
   3:  { 
   4:     int Increment(int number); 
   5:  } 
   6:  // The implementation of the plugin. 
   7:  public class IncPlugin : ITinyPlugin
   8:  {
   9:      public int Increment(int number) { return ++number; } 
  10:  }

The pluggable application defines the interface for its plugins: ITinyPlugin. In another assembly, the IncPlugin class implements ITinyPlugin. Here is the test code:

   1: [Test] 
   2: public void FooTest() 
   3: { 
   4:     // False test. 
   5:     int a = 4; 
   6:     int b = 5; 
   7:     Assert.AreNotEqual(a, b); 
   8:     // Load plugin, use it and test again. 
   9:     a = IncrementUsingPlugin(a); 
  10:     Assert.AreEqual(a, b); 
  11: }

There is a false test where 4 should be different from 5 and then using the plugin, 4 is incremented and the result is being compared to 5. You would expect the test to pass, but like me, you are wrong! Before explaining the flow, we need to know how to load an assembly (plugin) at runtime and execute its code:

   1: private static int IncrementUsingPlugin(int number) 
   2: { 
   3:     try 
   4:     { 
   5:         // Get all of the plugins pathes. 
   6:         string executionPath = 
   7:         Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
   8:         string pluginsDir = String.Format(@"{0}\Plugins", executionPath); 
   9:         string[] files = Directory.GetFiles(pluginsDir, "*.dll"); 
  10:         if (files.Length > 0) 
  11:         { 
  12:             // Load the first plugin and execute it. 
  13:             Assembly assembly = Assembly.LoadFile(files[0]); 
  14:             foreach (Type type in assembly.GetTypes()) 
  15:             { 
  16:                 if (!type.IsClass || type.IsNotPublic) 
  17:                 {
  18:                     continue; 
  19:                 }
  20:                 Type[] interfaces = type.GetInterfaces(); 
  21:                 if (((IList)interfaces).Contains(typeof(ITinyPlugin))) 
  22:                 { 
  23:                     object obj = Activator.CreateInstance(type); 
  24:                     ITinyPlugin plugin = (ITinyPlugin)obj; 
  25:                     return plugin.Increment(number); 
  26:                 } 
  27:             } 
  28:         } 
  29:         return number; 
  30:     } 
  31:     catch (Exception) { return number; } 
  32: }

image Usually, the plugins are located inside a ‘”Plugins” directory in the execution path. The code above, get the plugin directory path and read all of the assemblies located in it. It then, loads the first assembly, take the class that implements ITinyPlugin and use its Increment method. Everything looks fine, and believe me it also works fine in a standalone application (i.e not through NUnit), so why on earth does this test fails??

Debugging the code, I found out that executionPath variable was different from what I expected and different from its value when this code runs outside NUnit. Due to that fact, the Assembly.LoadFile() method failed because its input which is the assembly full name was wrong. After some more investigation, I realized (using the Modules window – Ctrl + D + M during runtime) that all of my assemblies are loaded from a temporary location:

“C:\Documents and Settings\shahar\Local Settings\Temp\nunit20\ShadowCopyCache\3880_633459266024531250\Tests\

Well, it turns out that NUnit, by default, uses a shallow copy option to run its tests. All of the assemblies needed for the execution of the test are copied to a temporary folder so that the original assemblies won’t be locked. Using this approach, the user can run his tests and work with Visual Studio at the same time. The problem here is that our Plugins folder wasn’t copied and hence wasn’t found during the test.


If you want to disable the shadow copy option (version 2.4.6), just go to NUnit GUI, Tools->Test Loader->Advanced and check the “Disable shadow copy” checkbox. Close NUnit and re-run your tests – Everything is fine now, the tests are green!!!

Hope you found this post helpful, enjoy…

Tags :

5 Responses to “Why Pluggable Applications Fail in NUnit?”

  1. Victor

    Said on May 17, 2008 :

    Wow, so it may be a good idea to store the plugins path in an environment variable…
    But on the other hand, I don’t like the idea of changing my code just because of a problem in my testing framework…
    Good article, Thanks.

  2. Ngu Soon Hui

    Said on May 18, 2008 :

    There is an easy way out for this. Instead of
    string executionPath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);

    Just use:
    string dllPath=Assembly.GetExecutingAssembly().CodeBase..Substring(8);
    string executionPath =Path.GetDirectoryName(dllPath);

    You can now get the physical path of your dll, not the shadow copy path.

    It’s ugly, but it works on my project

  3. Dave

    Said on June 12, 2008 :

    If you don’t like the .Substring(8) above, you can use a Uri to remove the schema:

    // Show that we have to use codebase to get the original directory rather than the shadow copy:
    // .Codebase is original location, .Location is the shadow copy
    Console.WriteLine(“Raw codebase=” + Assembly.GetExecutingAssembly().CodeBase);
    Console.WriteLine(“Raw location=” + Assembly.GetExecutingAssembly().Location);

    // Starts “file:///C:/” and ends with the unit test DLL
    string codeBase = Assembly.GetExecutingAssembly().CodeBase;
    Console.WriteLine(“codeBase=” + codeBase);

    // Starts with “file:\C:\” and ends at directory containing the unit test DLL
    string codeBaseDir = Path.GetDirectoryName(codeBase);
    Console.WriteLine(“codeBaseDir=” + codeBaseDir);

    // Removes scheme – now starts “C:\”
    Uri uriToCodeBaseDir = new Uri(codeBaseDir);
    string localPathNoScheme = uriToCodeBaseDir.LocalPath;
    Console.WriteLine(“localPathNoScheme=” + localPathNoScheme);

    // Add directory containing test files
    string testFilesDir = Path.Combine(localPathNoScheme, TEST_FILES_DIR);
    Console.WriteLine(“testFilesDir=” + testFilesDir);

    // Add test file itself
    string testFile = Path.Combine(testFilesDir, TESTFILE);
    Console.WriteLine(“testFile=” + testFile);

    // Check it!
    Console.WriteLine(“Goal=” + File.Exists(testFile));

    // Or, you can do most of it in one go…
    localPathNoScheme = new Uri(Path.GetDirectoryName(Assembly.GetExecutingAssembly().CodeBase)).LocalPath;
    testFilesDir = Path.Combine(localPathNoScheme, TEST_FILES_DIR);
    testFile = Path.Combine(testFilesDir, TESTFILE);
    Console.WriteLine(“Goal=” + File.Exists(testFile));

  4. Chris

    Said on November 27, 2008 :

    Even easier:

    string dllPath=Directory.GetCurrentDirectory() + ‘\\’ + Path.GetFileName(Assembly.GetExecutingAssembly().Location));

    Cheers ;)

1 Trackback(s)

  1. May 19, 2008: Reflective Perspective - Chris Alcock » The Morning Brew #96

Post a Comment