Need support for DLL Silverlight projects

Jul 27, 2011 at 4:43 PM

When running unit tests for the first time with AgUnit I received a FileNotFoundException when the StatLightConfigurationFactory.AssertFileExists is looking for a XAP file.  My Silverlight project is not creating a XAP file but a DLL.  This was working fine with the old version of AgUnit and Resharper 5.

Please enable DLL support so that non-xap Silverlight applications can be tested.  Thanks.

You can see what I am trying to unit test at http://nmock3.codeplex.com

Jul 27, 2011 at 5:47 PM

SDeKock - FYI: I have some basic support in StatLight for dll's only. (I say basic, because I threw it together a while back, but haven't heavily used/tested it). The support it has is to use some reflection to determine the dll's referenced (and their referenced (and...so on)) assemblies and package them up into a xap that is then used within StatLight.

If you look here https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestRunner/Silverlight/SilverlightUnitTestTaskRunner.cs#L105 and made the call to .GetStatLightConfigurationForDll(...) instead it could potentially do what you need.

Let me know if you have any questions.

Jul 27, 2011 at 6:49 PM
Edited Jul 27, 2011 at 6:52 PM

OK, I'll have a look. I'll have try to have something ready by Friday.
The biggest problem is that ReSharper doesn't really provide me with a lot of info about the project.

The easiest way to fix this right now would be to change your Silverlight class library to a Silverlight application project.

Aug 11, 2011 at 2:48 PM

Good afternoon!

I don't want to rush you but have you got any news on the silverlight .dll support?

Changing all our existing tests from a class library to a application project is very cumbersome unless i am overlooking that there is an automatic way to convert this in VS2010 - a tip here would be welcome!

 

Best regards,

Stefan

Aug 11, 2011 at 3:07 PM
Edited Aug 11, 2011 at 3:07 PM

I'll see what I can do to hack something together, but I won't have much time until Monday.

You can do it yourself if you want to. Follow the instructions on http://agunit.codeplex.com to get and build the source code.

The problem lies in these files, depending on which framework you're using (I guess MSTest):

https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestProvider/MSTest/MsTestAssemblyTaskProvider.cs
https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestProvider/XUnit/XUnitAssemblyTaskProvider.cs
https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestProvider/nUnit/NUnitAssemblyTaskProvider.cs 

See the TODOs ...

I was planning to get the XapFileName property from the project. But I'm not sure I can get at it through the ReSharper API.

I guess in your situation, for now we can use a File.Exists() on the XAP file, and if that fails, use the DLL.

If you use the DLL, as staxmanade has pointed out, you also need to change the call to GetStatLightConfigurationForXap to GetStatLightConfigurationForDll here:

https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestRunner/Silverlight/SilverlightUnitTestTaskRunner.cs

Aug 12, 2011 at 11:18 AM

Hi!

I tried to get this up and running for myself but seem to be stuck. I have incorporated the following changes (please note that i did'nt really go for clean code but rather for a rapid prototyping here, please bear with me):

https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestProvider/nUnit/NUnitAssemblyTaskProvider.cs

public string GetXapPath(RemoteTask task)
        {
            var assemblyTask = GetTask(task);

            var xapPath = assemblyTask.AssemblyLocation.Replace(".dll", ".xap");

            if (File.Exists(xapPath))
                return xapPath;
            else
                return assemblyTask.AssemblyLocation;
           
            // return assemblyTask.AssemblyLocation.Replace(".dll", ".xap"); // TODO: Find a way to get this from the project settings.
        }

then as staxmanade suggested in:  https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestRunner/Silverlight/SilverlightUnitTestTaskRunner.cs

private static StatLightConfiguration CreateStatLightConfiguration(IEnumerable<MethodTask> testMethods, ILogger logger, string xapPath)
        {
            var configurationFactory = new StatLightConfigurationFactory(logger);

            if (xapPath.Contains(".dll"))
            {
                return configurationFactory.GetStatLightConfigurationForDll(
                    unitTestProviderType: UnitTestProviderType.Undefined, // Let StatLight figure it out
                    dllPath: xapPath,
                    microsoftTestingFrameworkVersion: null, // Let StatLight figure it out
                    methodsToTest: new Collection<string>(testMethods.Select(m => m.GetFullMethodName()).ToList()),
                    tagFilters: null,
                    numberOfBrowserHosts: 1, // Maybe you spin up 3 or 4 here if you know you're running a ton of tests
                    isRemoteRun: false,
                    queryString: "", // This is passed to the browser host page (say your test need some configuration - could be passed here - probably not a use case in ReSharper runner)
                    webBrowserType: WebBrowserType.SelfHosted,
                    forceBrowserStart: false,
                    showTestingBrowserHost: false // If you need UI support this needs to be true
                );
            }


            ....
           
        }

 

The changes compile and can be started, but running a  nunit for silverlight dll test gives me the following error:

"Unit Test Runner failed to load test assembly" - FileNotFoundException "StatLight.Core, V1.4.4241.15421 or a depency was not found.  Full error (please excuse the german version): http://imageshack.us/photo/my-images/228/agunitnunitdlltesterror.jpg/

 

I have tried messing around with StatLight - specifically the AssemblyResolver.cs - to resolve this but cant seem to nail the problem..  i am afraid my .Net knowledge fails at this point.

 

Best regards,

Stefan

Aug 12, 2011 at 12:02 PM

Looks indeed like a problem in StatLight.
Apparently it's not adding the StatLight.Core assembly to the test xap when you're using the GetStatLightconfigurationForDll method.

As staxmanade pointed out, he never tested this functionality.

I don't have time to look at it until Monday.
I'll help you search for a solution then.

Aug 15, 2011 at 6:24 PM

Any updates on the DLL-runner? I lost the option to run my NUnit-based Silverlight tests when I updated Resharper from 5.x to 6. AgUnit looks really promising, but unfortunately I'm in the same boat as the original poster, meaning all my unit tests are in DLLs. 

Aug 15, 2011 at 11:58 PM
Edited Aug 16, 2011 at 12:56 AM

Our tests are in DLLs as well, and so I'm also looking forward to using AgUnit.

Aug 16, 2011 at 9:55 AM

No updates yet sorry, I'll try working on it this evening (GMT+2).
I understand this is high priority, it's on the top of my TODO list. 

Aug 16, 2011 at 11:44 PM

Making progress, if all goes well, I'll have something ready later this week.

Aug 16, 2011 at 11:44 PM

Let me know if there is anything I can help with.

Aug 17, 2011 at 10:44 PM

Ok, If you get the latest source, it will work.

There could be one small problem:

StatLight detects the test framework you're using depending on the assemblies your project has referenced. But it gets this from your project's assemby. 
If for example you're using "Microsoft.Silverlight.Testing", but you're using none of the types from it (like the TagAttribute), it will fail, because the compiler will think that the reference is not in use and optimize it away.

To fix this, put a [Tag] attribute on one of your tests, that should temporarily fix the problem.

Me and staxmanade are looking into a problem. 

Aug 18, 2011 at 12:07 AM

Hey All,

I the latest commit in statlight https://github.com/staxmanade/StatLight/commit/b0f756f13d699f1caace1c66e8f3fe1919090dac fixes the issue of not having a reference to the Microsoft.Silverlight.Testing.

Go ahead and update AgUnit to this version and it should work nicely. Let me know if you see any issues.

Happy testing.

Aug 18, 2011 at 7:55 AM

Thank you for your hard work! I did some testings with the latest commit and it seems we're almost there. My tests are now loaded into the ReSharper Unit Test Runner and it shows all of them correctly, though for some reason it ignores all the tests when executed. It just reports "Ignored" and doesn't provide any other info. Do you happen to know if there's some way to get more details out of the runner? Now there isn't any output, just the yellow color and "Ignored".

My tests are in a project created with Silverlight NUnit Project Template (http://www.testdriven.net/downloads/SilverlightNUnitProject.zip). As a test, I created a new project with that template, built it and executed the default test. And it worked! But when I added a new Assert to the test, the execution failed. So, this works:

        [Test]
        public void SystemVersion()
        {
            Assembly systemAssembly = typeof(Uri).Assembly;
            StringAssert.Contains("=2.0.5.0,", systemAssembly.FullName,
                "Check we're testing a Silverlight 2.0 assembly");
        }

But this doesn't:

        [Test]
        public void SystemVersion()
        {
            Assembly systemAssembly = typeof(Uri).Assembly;
            //StringAssert.Contains("=2.0.5.0,", systemAssembly.FullName,
            //    "Check we're testing a Silverlight 2.0 assembly");

            Assert.IsTrue(true);

        }

When executed, the runner shows the following exception:

System.MethodAccessException: Attempt by method 'NUnit.Framework.Constraints.EqualConstraint.Matches(System.Object)' to access method 'System.Collections.ArrayList..ctor()' failed.
   at NUnit.Framework.Constraints.EqualConstraint.Matches(Object actual)
   at NUnit.Framework.Assert.That(Object actual, Constraint constraint, String message, Object[] args)
   at NUnit.Framework.Assert.IsTrue(Boolean condition, String message, Object[] args)
   at SilverlightNUnitProject2.SilverlightTests.SystemVersion(): Attempt by method 'NUnit.Framework.Constraints.EqualConstraint.Matches(System.Object)' to access method 'System.Collections.ArrayList..ctor()' failed.
at NUnit.Framework.Constraints.EqualConstraint.Matches(Object actual)
at NUnit.Framework.Assert.That(Object actual, Constraint constraint, String message, Object[] args)
at NUnit.Framework.Assert.IsTrue(Boolean condition, String message, Object[] args)
at SilverlightNUnitProject2.SilverlightTests.SystemVersion() 

 

Aug 18, 2011 at 9:36 AM

AgUnit reports all tests it has not received results for from StatLight as ignored when the test run completes.
This is probably due to an exception being thrown somewhere, but not being displayed to you.

Apparently I forgot to comment the "Debugger.Break()" line again in https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestRunner/Silverlight/SilverlightUnitTestTaskRunner.cs 
This might be the cause of your ignored tests problem. I've done a new commit with it commented out now. 

It can also be an exception being thrown by StatLight. You can try these:

- Debugging instead of running your unit tests, you might see the exception show up. Enable "Thrown" for CLR-exceptions in Debug > Exceptions.
- Debugging AgUnit: Follow the instructions here: http://agunit.codeplex.com/discussions/268216#post654862 
- Thrown an exception in the handler method for TraceClientEvent here: https://github.com/sdekock/AgUnit/blob/master/src/AgUnit.Runner.Resharper60.TaskRunner/UnitTestRunner/Silverlight/SilverlightResultsHandler.cs

Aug 18, 2011 at 8:40 PM

Miksu, Please let me know if you make any progress.

Feel free to contact me if you need assistance:

twitter: @sdekock
skype: sdekock
e-mail / gtalk / live messenger: sdekock87@gmail.com

As soon as we've resolved this issue, I'm creating a new release of AgUnit.

Aug 18, 2011 at 9:11 PM

Miksu, I've been able to reproduce your error.

It looks to me like a bug in the NUnit version you're using. Looks like a missing method.
Or something using reflection to non-public types (Silverlight is very strict with that).
Looks like it was written to be used to run against the desktop .NET runtime.

I suggest using this port or NUnit for Silverlight, I've tested it and it works:

http://code.google.com/p/nunit-silverlight/ 

I'm creating a 0.4.2 release now !