Software Best Practices

We are considering getting a bit more advanced during our unit testing phase and regarding the use of static and/or dynamic code analysis tools.  While we might argue this is a coding phase activity, I think the lines between coding and unit testing are pretty blurred these days (even reversed if you believe in test first techniques) and group unit testing (automated and manual), coverage analysis, peer code reviews and static/dynamic code analysis all together.  Might even be other activities you throw in here (continuous integration or builds, smoke tests, etc.).

Anyway, turning just to code analysis tools, there are tons of them out there.  We are a combined embedded/real-time + Windows UI shop where most code is written in C++ and C# these days.  Any suggestions of static and/or dynamic code analysis tools to use or not use.  I guess I'm just as interested in the ones to avoid, as I think the biggest issue with code analysis is the false positives you often get.  Based on our past experience, the embedded side of our code often creates havoc with dynamic analysis tools (the minute you instrument the code, it basically stops working as the software/hardware timings become a problem).

Thanks in advance for any wisdoms.

We've found that several forms of code analysis are useful.

Metrics analyzers (such as Understand for C++ or SourceMonitor) are helpful for understanding the overall complexity, comment level, function sizing, etc. metrics. These are helpful if you want to provide team members on guidance about how code should be written to improve maintainability. Steve McConnell's Code Complete book gives guidance on desirable values for these metrics.

Controlling complexity (keeping functions and methods strongly cohesive, reducing coupling to global variables, keeping the cyclomatic complexity low) are key to improving unit testing results.

Lint is very useful for finding all kinds of subtle errors in C and C++ programs. Tools like FxCop (this link will move to MSDN in July) and Visual Studio 2005 perform code analysis tool for C# that is quite comprehensive and can be extended with custom rules. As you say, these tools can sometimes produce lots of "noise", but ignore them at your peril: they are a very inexpensive form of code inspection. In general, I want all the help I can get, before even running unit tests, by turning on all warnings from the compiler and from these analysis tools, and I want to have a very good reason to ignore any warnings.

Once you've actually written the code, run the static analyzers, and written the unit tests you can do coverage analysis if you think it's worth it. Most people think of "code coverage" (i.e., how much of the code has been executed) as the applicable metric. There are other metrics too: branch coverage (execute each leg of each conditional), condition coverage (test each condition within each branch), input domain (test all "interesting" input values to a function or method) are a few others.

For embedded systems testing for code coverage can be tricky: Inserting trace instructions disrupts timing.  You might first consider first how to do unit tests on a development workstation rather than the target system: create mock interfaces for hardware devices and communications interfaces. If you need system test code coverage analysis with no timing degradation, you will need to use auxiliary hardware to capture and buffer instruction fetch addresses, along with software to map these back to the source code.

Hope this helps,

-- Dave