I recently came across a new tool called AutoTest.Net. This is a great tool if you are a TDD/BDD practitioner. Basically AutoTest.Net can sit in the background and watch your source tree. Whenever you make changes to a file and save it, AutoTest will build your solution and run all unit tests that it finds. When it has run the tests it will display the failed tests in the AutoTest window (there is also a console runner if you are a hardcore console user).
This takes away some of the friction that comes with unit testing because now you don’t have to do the step of build and run tests during development. Simply write a test and save. AutoTest runs your tests and you see a failed test. Now implement the code that should make the test and save again. AutoTest again detects the change and does a build and test run. This cycle becomes automatic quite quickly and I’m finding I really like not having to explicitly build and run tests.
AutoTest.Net sports a nice set of features. You can see the full list of features on the main GitHub repo page (scroll down) but here’s a few. It supports multiple test frameworks including MSTest, NUnit, MSpec, and XUnit.net (I believe other frameworks can be supported by writing a plugin). It can also monitor your project in two different ways. One is by watching for changes to all files in your project (ie. watching the source files). The other is by watching only assemblies. Using this second method it would only run tests when you recompile your code manually. I’m not sure where this would be preferred but I might find a use for it yet.
Edit: Svein (the author of AutoTest.Net) commented that one other compelling feature is Growl/Snarl support. If you have either one installed you can get test pass/fail notifications through Growl/Snarl, which means you don’t have to flip back to the AutoTest window to check that status of your code. Nice!
This leads me to one other feature that I missed and that is that AutoTest.Net is cross-platform. It currently supports both the Microsoft .NET framework as well as Mono. This means you can develop on OS X or Linux.
So, to get started, download AutoTest.Net from GitHub. The easiest way is to click the “Downloads” link on the main AutoTest.Net project page and click one of the AutoTest zipfile links (at this time AutoTest.Net-v1.1.zip is the latest).
Once downloaded, unzip to a directory that you’ll run it from. I’ve put it in my “utilities” directory. At this point you can start using AutoTest.Net by running AutoTest.WinForms.exe or AutoTest.Console.exe. If you run the WinForms version, AutoTest will start by asking you what directory you’d like to monitor for changes. Typically you would select the root directory of your .Net solution (where your .sln file resides). Once you’ve selected a directory and clicked OK, you’re ready to start development.
The screenshot above is the main AutoTest.Net window. You can see that it ran 1 build and executed a total of 13 unit tests. The neat thing is that now as I continue to work, all I need to do to cause AutoTest.Net to recompile and re-run all tests is to make a code change and save the file. This sounds like a small change in the regular flow of code, save, build, run tests. However, once I worked with AutoTest.Net for a while it started to feel very natural and going back to the old flow will feel like adding friction.
Alright, so we have AutoTest.Net running and that’s good. As with most tools, it comes with a config file, AutoTest.config (which is shared between the WinForms and Console apps). This config file is well-documented so for the most part you can just open it up and figure out what knobs you can adjust.
One nice feature that AutoTest.Net has is that you can have a base configuration in the directory where the AutoTest binaries reside but then override it by dropping an AutoTest.config file in the root directory that you are monitoring. This allows you to keep a sensible base config in your AutoTest application directory and then override per-project as needed. Typically I’ve been setting the “watch” directory as the directory that my .sln file is in and so you’d drop the AutoTest.config file in the same directory as the .sln file.
In the above screenshot I have the AutoTest.config file which overrides a few settings, one of which is the ignore file: _ignorefile.txt.
One option that is useful is called the IgnoreFile. This is well-documented in the config file but it didn’t click with me initially. The IgnoreFile option specifies a file that contains a list of files and folders that AutoTest.Net should ignore when monitoring the configured directories for changes. The config file mentions the .gitignore file as an similar example so if you are familiar with git, it should make sense. The one piece that took me a while to figure out was where this ignore file should go. I finally figured out that if you put it in the root of the monitored directory (ie. beside your .sln file), it will pick it up (see above screenshot).
AutoTest.Net is a great tool to help with the Red, Green, Refactor flow. It removes some of the friction in my day-to-day work by eliminating the need to manually invoke a compile and test run.
Lastly, the observant readers will have noticed that I said it builds and runs all tests any time it notices a file change. Greg Young is working on a tool called Mighty Moose that builds on top of AutoTest.Net as an add-in to Visual Studio (I think there’s a standalone version too). Mighty Moose ups the ante by figuring out what code changed and what tests would be affected by that change. It then only runs the affected tests, significantly cutting down the test run time.