Review: Free C# code analysis tools

Over the upcoming days, I'll be reviewing the C# code I'm working on for my Bachelor's thesis. It consists of nearly 9.000 lines of code (over 300 kB), so I felt somewhat reluctant to read it all through. I decided to try and identify the most obvious problems using automated code analysis tools first. Because I use Visual Studio 2005 under Windows for C# development, the reviews will mainly be targeted at this environment.

I tried the following programs:

Potential problem detection

FxCop, Code Analyzer, Gendarme, devAdvantage

Quality metrics

devMetrics, NDepend, SourceMonitor, vil

Code coverage

NCover

Similarity detection

Simian

None of the following reviews are very comprehensive, because I only played around with the tools for a little while, but these reviews can give you a good indication on what to use and what to ignore.

FxCop

The FxCop program by Microsoft themselves checks assemblies for compliance with the .NET design guidelines, and identifies potential problems within the code.

Features

FxCop checks a lot of issues with Microsoft guidelines. It gives a certainty percentage for each issue found, and also clearly explains what the problem is, and how to fix it. Some of the more interesting issues that are checked for:

• Are exceptions raised that should not be raised by user code? For example, System.Exception should not be thrown directly.
• Is an IFormatProvider supplied when converting strings to numbers? Very important if your code is to behave correctly under other locales.
• Are reference parameters of methods checked to be non-null before use?
• Are fields initialized to default values that are already assigned by the runtime, like null for reference fields? This would result in an unnecessary extra assignment.
• Does the string argument to ArgumentOutOfRangeException contain the name of the argument?
• Are there any unused local variables?
• Are you using public nested classes? These are considered harmful by the guidelines.
• Do abstract types have a public constructor? This should be made protected.
• Are there any unused methods?
• Are variables like fileName capitalized correctly? “filename” is wrong since “file” and “name” are (apparently) separate words.
• Are you using a derived type as a method parameter where a base type would suffice?

This list goes on and on. FxCop found 557 issues in my code from dozens of different rules.

You can jump directly from an issue to the corresponding source line(s) in Visual Studio or another application of your choice.

Judgement

I'm very impressed by the comprehensive list of flaws that this program detects. It is definitely very useful for anything larger than a toy application.

Code Analyzer

At first glance, the Code Analyzer tool seems to do similar things to FxCop. And indeed the website provides us with a useful list of the differences, broken English included:

FxCop advantages (comparing to Code Analyzer):

• Extensive set of rules available out of box. Code Analyzer provides just limited set of sample rules.
• Since it works with assembly metadata works with code created in any .NET language. Code Analyzer works now just with C# sources.

Code Analyzer advantages (comparing to FxCop):

• FxCop is limited to assembly metadata, Code Analyzer works with source code and provides more functionality like comments, position in source code and more.
• FxCop has flat rules structure, which makes orientation in policy more difficult for larger policies. Code analyzer has hierarchical structure, based on logical rules categories.
• FxCop provides only one type of report, Code Analyzer is flexible and provides more report types and users can create their own report types.

Especially the first advantage of Code Analyzer, source code inspection (as opposed to assembly inspection) seems worthwhile. Unfortunately, the program crashed on startup so I am unable to test it.

Gendarme

Powered by the Cecil code inspection library, Gendarme tries to identify points of improvement in your code based on a certain set of rules. There is no binary version yet; you'll have to build it yourself from an SVN checkout.

Features

There is no GUI or IDE plugin, so you're stuck with the command line. Gendarme is run on assemblies, so it does not inspect the actual source code. On my program, it identified the following problems at multiple points in my code:

• You should use String.Empty instead of the literal "", because it gives better performance.
• A static field is written to by an instance method. (This was intentional: each object gets a unique ID, and I increment the “next ID” field in the constructor.)
• Newline literals (\r\n or \n) in strings are not portable; use Environment.NewLine instead.

All in all, useful, but nothing spectacular.

Judgement

This could become a very useful tool, if the rule set is expanded. At the moment it will not identify very much. The lack of a decent user interface limits its practical use.

devAdvantage and devMetrics

devAdvantage is a Visual Studio add-in that helps you identify areas that might use refactoring. devMetrics is an add-in to compute code complexity metrics. The Community Editions can be downloaded for free. The programs look interesting, but do not work on Visual Studio 2005. Bummer.

NDepend

NDepend is a very feature-rich quality measurement tool, also powered by Cecil. It operates on .NET assemblies, but because it also extracts debug information it can link this back to the original source code. A free one-month version can be downloaded for trial, academic and open-source use. You can view the getting-started animation to get an idea of the possibilities.

Features

NDepend uses CQL, the Code Query Language, to extract information about the code. It allows you to construct your own queries if you're willing to invest the time to learn this. CQL is similar to SQL; take a look at this demo (3 minutes 30 seconds, Flash). For example, you can find all methods with over 200 intermediate language instructions, and sort them by descending number of instructions, using the following query:

SELECT METHODS WHERE NbILInstructions > 200 ORDER BY NbILInstructions DESC

NDepend comes with a few dozen built-in CQL queries that measure certain aspects of your code and can be used to quickly spot potential problems.

NDepend will spit out an HTML file like this one with humongous amounts of information on your project, most of which is just detailed factual information that is almost entirely useless. In my situation, NDepend failed to include the CQL results in the HTML file for some unknown reason.

The HTML file does, however, contain some useful information. It provides you with a table where the worst statistics are highlighted per method. It also lists warnings that, as far as I could tell, are not produced with CQL queries. In my case these were mainly “method so-and-so is protected and could be made private” warnings.

Other interesting features are the TypeRank and MethodRank, computed like the proven Google PageRank. It shows which types and methods are the most important in your program. On my program it did indeed give a very good indication.

The main part of the program is the VisualNDepend. This produces a two-dimensional chart much like the disk-space charts from SequoiaView (among others). The area of each rectangle indicates the value of some metric. by default this is the number of lines of code of the respective class or method, but you can also select metrics like the MethodRank or the cyclomatic complexity.

Unfortunately, there is no easy way to ignore certain source files or methods, e.g. designer-generated code. You'll want to ignore these while scanning the results, because generated code usually makes for terrible metrics. You can use CQL to do this, but you'll have to modify each of the predefined CQL quality metrics.

Judgement

NDepend is a difficult tool to work with at first. It can give you a wealth of useful information once you get the hang of it, but for a quick inspection it is less practical.

SourceMonitor

SourceMonitor is a simple free program to compute quality metrics on your code. Apart from C#, it can also be used for C, C++, Java, Visual Basic, VB.NET, Delphi and (strangely) HTML.

Features

SourceMonitor produces a table view of some quality metrics of your code, organized per source file. In the table view, you can double-click on any source file to get more detailed information about this file. This produces, among others, a chart showing how many statements are at a particular “block depth”, the number of brace pairs surrounding it.

The program creates a checkpoint for each measurement, so you can easily track the (hopefully) downward slope of your program's complexity while you are refactoring.

Judgement

A simple, yet useful tool. Very easy to use and understand.

Vil

Everything that Vil does, according to its web site, is done better by NDepend. Also, Vil has no GUI yet and gives a very discontinued impression. I won't bother.

NCover

NCover is a code coverage tool. Its main purpose is to determine how much of your code is covered by your unit tests. It does this by simply running the program or tests and looking which lines are actually executed.

Features

NCover is a simple command line tool without many bells and whistles. Simply tell it which program to run. It generates a large XML file with the output data (445 kB already in my relatively small program). The XML can be viewed with an accompanying XSLT style sheet, which you have to copy over to the right directory yourself.

The resulting view gives you a percentage bar for each class, showing the amount of code executed in that class. Clicking the class name expands it, breaking it down into methods. Clicking a method name breaks it down further into its individual lines.

There are ways to run NCover periodically and monitor the coverage of your tests. I haven't tried this.

Judgement

A simple tool, but more useful than I thought at first glance. It can give you a good indication which parts (especially, which if branches) your unit tests have missed. (Then again, this turns unit testing more into a white-box test when it was intended to be black-box.)

Simian

Simian identifies regions of code that are similar. It is a little Java-oriented, but also supports many other popular languages. Simian is free for non-commercial projects and for trial purposes.

Features

There is no GUI or Visual Studio plugin, you'll have to work from the command line. This hugely diminishes the ease of use, especially because the Windows command prompt is so clunky. Simian produces a list with entries like the following:

Found 11 duplicate lines in the following files: Between lines 30 and 63 in maths\MatrixAlgebra.cs Between lines 28 and 61 in maths\Vector.cs

Correct, MatrixAlgebra.cs was split up into Matrix.cs and Vector.cs, and should be removed entirely.

Judgement

A duplicate code finder sounds very useful, but it's use is very limited. It found nothing useful on my project. The results may vary for other coders. In any case, the lack of IDE integration for .NET analysis makes using this tool more effort than it's worth.

Conclusion

If you care about the details, don't look any further than FxCop. It's very comprehensive and easy to use. Code Analyzer may complement FxCop nicely, if you can get it to run.

For a more general view on things, NDepend can be very useful, if you're willing to invest some hours to get acquainted with it. For a quick overview, SourceMonitor can be a better alternative.

If there's any free program that I've overlooked, please let me know so I can include it!