AgUnit - Silverlight unit testing with ReSharper

Commented Issue: AgUnit 0.7 not running under R# 7.1 [6454]

harryhazza — Wed, 08 May 2013 15:42:20 GMT

AgUnit is installed and loaded, but Silverlight tests do not run using the Silverlight runtime, as if the plugin was simply not there.

Have attached screenshots of Options > Plugins, Unit Testing, Unit Testing > MSTest, and the ReSharper log file for loading Visual Studio and performing a test run.

All DLLs were unblocked before installing plugin.
Comments: Apologies, this does work. If you read this, you simply need to follow the instructions elsewhere! Do not use the windows native unzip program, use 7-zip to extract to the resharper plugins folder. After extract, right click on any of the extracted DLLs and if you should __NOT__ get an option to "Unblock".

Commented Issue: AgUnit 0.7 not running under R# 7.1 [6454]

harryhazza — Wed, 08 May 2013 15:03:20 GMT

AgUnit is installed and loaded, but Silverlight tests do not run using the Silverlight runtime, as if the plugin was simply not there.

Have attached screenshots of Options > Plugins, Unit Testing, Unit Testing > MSTest, and the ReSharper log file for loading Visual Studio and performing a test run.

All DLLs were unblocked before installing plugin.
Comments: I have uninstalled dotCover and restarted Visual Studio. The resharper runner lists the tests in the test runner so it can 'see' them but it does not run them.

New Post: Modify source to skip tests and add a reason

JoanComasFdz — Thu, 28 Mar 2013 08:12:11 GMT

Aha, thanks for that info, I'll wait for that example :)

Meanwhile I am debugging it and it seems that some libraries are not found.

From the resharper.log:

[ExternalSourcesProjectFilesManager] Execute Changes  *A(NAMESPACE1)
[Caches] <CacheManager> AssemblyChanges: 1
[PMM] Modify: NAMESPACE12, Version=1.0.0.0, Culture=neutral, PublicKeyToken=485c7b508f2591d7
[PMM] change begin
[PMM] change end
[ExternalSourcesProjectFilesManager] Execute Changes  *A(NAMESPACE2)
[Caches] <CacheManager> AssemblyChanges: 1
Can't resolve reference NAMESPACE3, Version=1.5.2.0, Culture=neutral, PublicKeyToken=ebb7bb93ac082f1e from PATH/TO/LIBRARY.dll
SessionStateChanged

Then starts all this lines:

VSDocumentOperationsHandler::ConnectToStream, id 41397582 text stream:37033924

And these are the final lines:

TaskRunner output end
OnSessionStateChanged
OnSessionStateChanged
SessionStateChanged
Counters not valid
SessionStateChanged
ExecSessionChange
SessionStateChanged
PreviewUpdate

SilverlightUnitTestTaskRunner is not reached while debugging and the SL unit test is ignored. Can this resolving exceptions be the problem?

Edit: If I execute againt he same test after a while, that is the output:

09:28:08.921: Thread:1: OnSessionContentsChanged
09:28:08.936: Thread:1: OnSessionStateChanged
09:28:08.936: Thread:1: OnSessionStateChanged
09:28:08.936: Thread:1: OnSessionStateChanged
09:28:16.019: Thread:1: SessionStateChanged
09:28:16.019: Thread:5: TaskRunner output:
09:28:16.034: Thread:1: SessionStateChanged
09:28:16.050: Thread:1: SessionStateChanged
09:28:16.362: Thread:1: PreviewUpdate
09:29:00.384: Thread:5: 
09:29:00.384: Thread:5: TaskRunner output end
09:29:00.384: Thread:5: OnSessionStateChanged
09:29:00.384: Thread:5: OnSessionStateChanged
09:29:00.385: Thread:1: SessionStateChanged
09:29:00.385: Thread:1: Counters not valid
09:29:00.396: Thread:1: SessionStateChanged
09:29:00.552: Thread:1: ExecSessionChange
09:29:00.552: Thread:1: SessionStateChanged
09:29:00.858: Thread:1: PreviewUpdate

New Post: Modify source to skip tests and add a reason

SDeKock — Wed, 27 Mar 2013 17:59:43 GMT

In you're example you're only doing NotifyFinished() on the root task nodes (your solution).
They are recursive: each node has child nodes (projects, namespaces, testclasses, tests, ...).

I'll post a more detailed example later.

New Post: Modify source to skip tests and add a reason

JoanComasFdz — Wed, 27 Mar 2013 15:24:38 GMT

Thanks for the reply,

Unfortunately I can't get it working, this is the code I am using:

public class SilverlightUnitTestTaskRunner : RecursiveRemoteTaskRunner
{
[...]
        private void ExecuteSilverlightTasks(TaskExecutionNode node)
        {
            var assemblyTaskProviders = UnitTestTaskProviderFactory.GetAssemblyTaskProviders();
            var classTaskProviders = UnitTestTaskProviderFactory.GetClassTaskProviders();
            var methodTaskProviders = UnitTestTaskProviderFactory.GetMethodTaskProviders();

            var taskEnvironment = new TaskEnvironment(Server, assemblyTaskProviders, classTaskProviders, methodTaskProviders);
            var taskNode = new TaskNode(node, taskEnvironment);

            foreach (var silverlightTask in taskNode.GetSilverlightTasks())
            {
                silverlightTask.Node.NotifyStarting();
                silverlightTask.Node.NotifyFinished("This is a reason.", TaskResult.Skipped);
            }
        }
}

That's how I am doing it:

Buidl the plugin
Close VS
Copy the files to it's folder in resharper's installation folder
Start VS
Check for the plugin to be correctly installed.
Open my solution.
Run one Silverlight unit test.

It says always "Ignored" with no reason.

This is my machine setup:

.NET Framework 4.0.303019
Visual Studio 2010
AgUnit 0.7 source code builded in VS2010.

What am I doing wrong?

New Post: Modify source to skip tests and add a reason

SDeKock — Fri, 15 Mar 2013 17:50:59 GMT

Yes, you can use:

silverlightTask.Node.NotifyFinished("Some reason here", TaskResult.Skipped);

This will also fix the status of the items above the test (the test class / namespace / assembly).

What's the reason you're not just using the [Ignore] attribute on the tests ?

New Post: Modify source to skip tests and add a reason

JoanComasFdz — Fri, 15 Mar 2013 13:41:44 GMT

Hi,

We found this plugin very usfeull and we want to add some custom logic to it.

We have downloaded the source code of AgUnit 0.7 and we want to skip some tests. We are doing so by editting SilverlightUnitTestTaskRunner.ExecuteSilverlightTasks() as:

(...)
private void ExecuteSilverlightTasks(TaskExecutionNode node)
{
    var assemblyTaskProviders = UnitTestTaskProviderFactory.GetAssemblyTaskProviders();
    var classTaskProviders = UnitTestTaskProviderFactory.GetClassTaskProviders();
    var methodTaskProviders = UnitTestTaskProviderFactory.GetMethodTaskProviders();

    var taskEnvironment = new TaskEnvironment(Server, assemblyTaskProviders, classTaskProviders, methodTaskProviders);
    var taskNode = new TaskNode(node, taskEnvironment);

    foreach (var silverlightTaskNode in taskNode.GetSilverlightTasks())
    {
        silverlightTaskNode.Execute(SkipIfNecessary);
    }
}

/// <summary>
/// Skips the silverlight unit test.
/// </summary>
/// <param name="silverlightTask">The silverlight task.</param>
private static void SkipIfNecessary(SilverlightTask silverlightTask)
{
    var message = string.Format("Skiping {0}...", silverlightTask.Node.Task.Id);
    var logger = new TraceLogger(LogChatterLevels.Full);
    logger.Information(message);
}

This is very basic, so we would like to specify that those tests are being ignored for a reason.

Is it any way to do so?

Thanks

New Post: FileLoadException when Copy Local is set to true for Microsoft.Silverlight.Testing

lukeguy — Wed, 13 Mar 2013 17:55:55 GMT

Of course! The ones that I have referenced in my SL test project are from SL Toolkit (dec11)
Microsoft.Silverlight.Testing.dll is 5.0.60818.0 and
Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight.dll is 5.0.60818.0 also.

Reviewed: AgUnit 0.7 (Mar 13, 2013)

akilhoffer — Wed, 13 Mar 2013 16:15:17 GMT

Rated 5 Stars (out of 5) - Worked perfectly for me right out of the box. Thank you!!

New Post: FileLoadException when Copy Local is set to true for Microsoft.Silverlight.Testing

SDeKock — Wed, 13 Mar 2013 15:38:59 GMT

Can you give me the file versions (Not the assembly version, because that will be either 2.0.5.0 or 5.0.5.0 and completely meaningless) of Microsoft.Silverlight.Testing and Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight.dll ?

You can find those by going to the file properties in windows explorer and looking at the details tab.

New Post: Configurable WindowGeometry

SDeKock — Wed, 13 Mar 2013 15:18:59 GMT

Thanks for the compliment :)

As for the configuration options, I've been putting that off very long, but it's something that a lot of people have asked.
AgUnit currently has no implementation for it, but I was planning to integrate it with the R# options. Giving all the goodness of R#'s shared settings ..
This should be quite straightforward, see: http://confluence.jetbrains.com/display/ReSharper/2.9+Settings+%28R7%2
It would be great if you want to submit a pull request, less work for me :)

As for the .NET requirement, I'm planning to lower it again to .NET 4 by using reflection to detect if we need to support MSTest11 (= Visual Studio 2012).
In that case I can load the needed assemblies safely. Now AgUnit has direct references to them causing the .NET 4.5 requirement.

R# 8 support is also coming soon (I hope).

And dotCover is something I've gotten a lot of requests for lately too.

New Post: Configurable WindowGeometry

GaryJohnson — Wed, 13 Mar 2013 14:21:04 GMT

Hello,

I run a lot of UI tests that require BrowserWindowState.Normal, forcing me to create custom builds of AgUnit. Is this something that could be made configurable? Would you be open to a pull request if I attempted to implement this? Finally, if I did try to implement this, does AgUnit have an existing vehicle for configuration options that I should use?

Also, will it ever be possible to get the newer builds of AgUnit working in VS2010, or is this technically unfeasible due to the .NET requirement? I'd like to be able to keep using the latest R# in VS2010.

Finally, thanks for your hard work -- AgUnit rocks.

New Post: FileLoadException when Copy Local is set to true for Microsoft.Silverlight.Testing

lukeguy — Wed, 13 Mar 2013 13:58:36 GMT

Thanks for the reply!

I experimented a bit and did something similar. I first made a backup of all StatLight.Client XAPs and removed them from the StatLight install folder. Then I copied back the newest one (StatLight.Client.For.MSTest2011December.xap) and renamed it to StatLight.Client.For.MSTest2010May.xap (as this if the XAP that StatLight uses to load my tests). This solved the problem from the command line (no more assembly version conflict messages).

Then I did the same in AgUnit install folder, I removed all other XAPs except the new one and renamed it to 2010May. (I did this with the XAPs already present in Plugins/AgUnit, I didn't copy the one from StatLight). And this solved the problem with running the tests from Resharper's Test runner from VS.

I see that in AgUnit install folder I also have every MSTest XAP with "-SL5" suffix. I didn't have the time to experiment with them so I'm not sure if AgUnit's StatLight doesn't use them at all or there is another issue.

I would really like to know if you can reproduce this and how to properly fix the problem. My solution works but I'm not very comfortable with voodoo solutions as I have no idea if I broke something else now. :) Thanks!

New Post: After installing AgUnit in vs 2012 and R7 Silverlight unit tests are still recognized as MsTest tests and do not run

SDeKock — Wed, 13 Mar 2013 12:45:24 GMT

For those interested, this was a problem with Code Contracts.
The checkbox for runtime checks needed to be enabled for the test project.

New Post: FileLoadException when Copy Local is set to true for Microsoft.Silverlight.Testing

SDeKock — Wed, 13 Mar 2013 12:38:28 GMT

This is probably because there's an older version of StatLight included with AgUnit. (which is still loading the SL4 version of Microsoft.Silverlight.Testing.dll).

You can try replacing the StatLight.dll and all the .xap files in the AgUnit plugin folder with those from the latest StatLight.

I'll try reproduce this issue and create a new release.

New Post: FileLoadException when Copy Local is set to true for Microsoft.Silverlight.Testing

lukeguy — Wed, 13 Mar 2013 09:59:58 GMT

Hi! I'm using AgUnit with Resharper 7.1 in VS2012 with SL5 and nunit-silverlight. I'm trying to figure out how to make it possible for my tests to run both from AgUnit and from the browser.

The problem I'm having is that if I set Copy Local to false for Microsoft.Silverlight.Testing and Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight, my tests run in AgUnit fine. But when running them in browser, the browser crashes because these DLLs are missing. If I include them in the XAP, then I can run the test from the browser, but the AgUnit shows a message box saying that assembly version conflicts with the one bound in the app domain.

After I click OK, according to AgUnit the tests is still running and it stays like that indefinitely. I have to manually stop it from Unit Test Sessions view.

I understand this exception comes from StatLight but strangely, if I run my test from StatLight command line, I only get this as a warning (yellow) but all tests run successfully.

********************* ReflectionTypeLoadException *********************
***** Begin Loader Exception Messages *****
   -  Could not load file or assembly 'Microsoft.Silverlight.Testing, Version=5.
0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependenc
ies. The requested assembly version conflicts with what is already bound in the
app domain or specified in the manifest. (Exception from HRESULT: 0x80131053)
   -  Could not load file or assembly 'Microsoft.Silverlight.Testing, Version=5.
0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependenc
ies. The requested assembly version conflicts with what is already bound in the
app domain or specified in the manifest. (Exception from HRESULT: 0x80131053)
   -  Could not load file or assembly 'Microsoft.Silverlight.Testing, Version=5.
0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependenc
ies. The requested assembly version conflicts with what is already bound in the
app domain or specified in the manifest. (Exception from HRESULT: 0x80131053)
   -  Could not load file or assembly 'Microsoft.Silverlight.Testing, Version=5.
0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependenc
ies. The requested assembly version conflicts with what is already bound in the
app domain or specified in the manifest. (Exception from HRESULT: 0x80131053)
   -  Could not load file or assembly 'Microsoft.Silverlight.Testing, Version=5.
0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependenc
ies. The requested assembly version conflicts with what is already bound in the
app domain or specified in the manifest. (Exception from HRESULT: 0x80131053)

***** End Loader Exception Messages *****

...
*************** Summary ********************
********************************************
Total          : 3
Successful     : 3
Failed         : 0
Ignored        : 0
Completion End : 13.3.2013. 10:43:55
Duration       : 00:00:01.3826295
********************************************

Does anyone know what's the issue here? Or a way to work around this? Thank you!

New Post: After installing AgUnit in vs 2012 and R7 Silverlight unit tests are still recognized as MsTest tests and do not run

SDeKock — Mon, 11 Mar 2013 17:00:48 GMT

Sorry, I don't have access to a headset ...

If you want I can connect through teamviewer and remotely assist.
Just let me know the time (+timezone) and teamviewer info.

You can reach me on Live Messenger or Google Talk with sdekock87@gmail.com. Or on Skype with sdekock, but again: no headset ... so chat only.

New Post: After installing AgUnit in vs 2012 and R7 Silverlight unit tests are still recognized as MsTest tests and do not run

ValentinKuzub — Thu, 07 Mar 2013 17:23:30 GMT

Hm ok well then actual version I am currently using is from Toolkit 4.0 so it should be supposedly working just fine.

I will provide you details you asked for before about resharper log.. I would like to fix this asap though may I connect with you by skype maybe? my sn is:qteeth

Thanks in advance.

New Post: After installing AgUnit in vs 2012 and R7 Silverlight unit tests are still recognized as MsTest tests and do not run

SDeKock — Thu, 07 Mar 2013 15:53:49 GMT

You have to look at the FileVersion, not the AssemblyVersion. All Silverlight assemblies (SL2, SL3 and SL4) use 2.0.5.0 as AssemblyVersion.
SL5 changed this to 5.0.5.0.
You can find this version number by right clicking on the file in windows explorer > properties > details tab.
An example of this is 4.0.40413.2011.

StatLight is the framework AgUnit uses under the covers to run the actual unit tests.

Every version from Silverlight 3 and up is supported by StatLight, just not the Silverlight 2 version anymore.
U can get these Silverlight unit testing assemblies from NuGet or from:
http://silverlight.codeplex.com/releases/view/43528 (this is the one for SL4).

Integration with TeamCity should be done with StatLight directly: http://statlight.codeplex.com/wikipage?title=TeamCity%20Integration&referringTitle=Documentation

New Post: After installing AgUnit in vs 2012 and R7 Silverlight unit tests are still recognized as MsTest tests and do not run

ValentinKuzub — Thu, 07 Mar 2013 12:28:08 GMT

Hey SDeKock

I am using Testing assemblies from SL 4.0 ToolKit.

Apparently they are those 2.0 versions you are talking about??

// Assembly Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight, Version 2.0.5.0
[assembly: AssemblyVersion("2.0.5.0")]

I tried to find something more fresh, but not SL5.0 cause as I say my application is SL4.0 and with testing assemblies from 5 toolkit it wouldnt build (i checked).

Are you saying that Silverlight 4.0 is now not supported? Or there is some workaround.. I am not yet ready to move to SL5.0

And also while another question, how would you integrate AgUnit with teamcity , if that is possible.
Thanks in advance