Parallelism at PDC: Day 3 - finding Heisenbugs

The first parallelism session on Day 3 here at PDC is 'Research: Concurrency Analysis Platform and Tools for Finding Concurrency Bugs'

This was the one time I had to pick between two parallelism sessions - I choose this one because many Microsoft developers internally talked about using the CHESS research in some of their development work.

Parallel programs require proper data synchronization, and getting it wrong can lead to dead locks and race conditions.  These parallel programming bugs are notoriously difficult to debug because they are non-deterministic. A program will pass or fail intermittently.

This gives rise to the term ‘Heisenbugs’ because observing the bug (e.g., in a debugger) can "fix" it!

The presenters were Madan Musuvathi and Thomas Ball, both from Microsoft Research.

Madan explained that debugging is often a huge productivity problem for parallel programmers. Microsoft Research has developed what they call the ‘Concurrency Analysis Platform’ (CAP) to enable concurrency bug-finding tools. Inside Microsoft, a tool called CHESS build on CAP has become popular.

CAP allows a program to be run deterministically. An attendee asked if CAP could just be put in the system scheduler to force determinism, and be there all the time. Tom said it an interesting idea but with drawbacks he did not elaborate on.

Present a tool called "CHESS" - can find Heisenbugs in Win32/.NET in native and managed code. Maden and Tom announced that this tool will be available externally soon at http://msdn.microsoft.com/devlabs. There is information about CHESS today at http://research.microsoft.com/projects/CHESS

CHESS/CAP uses an interleaving approach to increase test coverage.  This runs a program multiple times, and hopes that changing the interleaving will expose a failure.  The advantage is CHESS on CAP is its ability to reproduce the sequential debugging experience to debug a parallel bug. The disadvantage is it can miss possible failures in code is not exercised and the number of interleavings needed to fix this is never realistic (expediential explosion). Tom used the phrases ‘go get some coffee’ and ‘in a day or two’ to describe use on real programs. Tom said CHESS does not scale to large programs today, and Madan promised future work is looking at heuristics to try to reduce the search space intelligently to address this.

Competing approaches to CHESS’s interleaving approach include:

Tom explained that future work using CAP includes a data-race detection engine and a tool for finding memory-model errors.  In general he said you can build many more tools on top of CAP – including debug and visualization tools.  Future work also includes porting to 64 bits.

Madan showed off a tool called 'Sober' - a tool for finding memory-model errors. Madan spun the fact the tool is hard to use by saying 'this tools is for very very smart people.  

CHESS is currently integrated into Visual Studio Team Test only for managed code in a single process (no services).  Tom said native code testing is done only from the command line right now. Both require the Team version of Visual Studio.

We’re really seeing an explosion now of tools to help with parallelism.  CHESS is an interesting twist on helping find Heisenbugs which tries to force a program into failing to detect an error, and allow it to be debugged in reproducible fashion.