December 30, 2003

Security Testing with csUnit

Recently I have been wrestling with the idea of providing unit testing within our codebase. Wim has put up some good info pointers in his Miniwiki about automated unit testing, which included a link to csUnit, a C# unit testing framework.

Nick Smith wrote a good article on how to Write Unit Tests for C#, although he uses dotUnit, which is a .NET branch of JUnit, a unit testing framework for Java. It seems clean enough, but for some reason I like csUnit better. No real intellectual basis for this... just a feeling. Not sure why.

This DOES have me thinking about security tests in my .NET code though. There is no reason that these test harnesses cannot be retrofitted to supply security testing per method. I just am not sure I like how the 'extreme programming' community approach unit testing in this regard. I don't know how practical it is to write the tests BEFORE the code. I think it would be easier to write the initial security tests for each method once the method is written, so you know what to test for, and more importantly what the resiliency reaction should be in the face parameter validation, tainted data injection, state manipulation etc.

Another important tangent to this is trying to make this integrate across the project scope and build scripts. Unlike C, C# did not appear to have clear preprocessor directives, to ensure the tests are not compiled into the IL for production release builds. Scott originally thought using ConditionalAttributes would work out, and later found that there are tonnes of #if#else conditions and other C# preprocessor directives. Seems no one knows about them... but they ARE there. I love learning something new every day. Now if/when I implement this I can go write tonnes of C style DEBUG blocks around the testing framework.

What are other peoples experience with this? Anyone out there doing C# unit testing? Any focused on security testing?

Posted by SilverStr at December 30, 2003 08:14 AM | TrackBack
Comments

I haven't used C#, but I have programmed test first for several years and find test-driven design as described in Kent Beck's books to be the most powerful of all the XP practices. Writing the unit tests first not only helps you design the interfaces to your classes, but also forces you to consider the possible types of dangerous inputs before writing your code.

However, it's worth noting that in XP you don't write all your tests first, then write your code. Instead, you write tests for a feature, implement the feature, check your tests, then realize you want to make some changes and so write more tests for your changes in an iterative process. If you realize that you need more tests after writing the code, XP practices readily accepts that fact too, but by requiring you to test first, XP gets you to think in depth about the problems your code might have before you write it. While you hopefully already think like this during the design process, putting that thought into test code makes it more concrete and precise, and thus better able to detect problems in your code. Also, by integrating testing so closely with coding, XP ensures that you have your tests early in the development process when they're most useful to you.

Posted by: jwalden at December 31, 2003 08:07 AM