The first parallelism session on Day 3 here at PDC is 'Research: Concurrency Analysis Platform and Tools for Finding Concurrency Bugs'
This was the one time I had to pick between two parallelism sessions - I choose this one because many Microsoft developers internally talked about using the CHESS research in some of their development work.
Parallel programs require proper data synchronization, and getting it wrong can lead to dead locks and race conditions. These parallel programming bugs are notoriously difficult to debug because they are non-deterministic. A program will pass or fail intermittently.
This gives rise to the term ‘Heisenbugs’ because observing the bug (e.g., in a debugger) can "fix" it!
The presenters were Madan Musuvathi and Thomas Ball, both from Microsoft Research.
Madan explained that debugging is often a huge productivity problem for parallel programmers. Microsoft Research has developed what they call the ‘Concurrency Analysis Platform’ (CAP) to enable concurrency bug-finding tools. Inside Microsoft, a tool called CHESS build on CAP has become popular.
CAP allows a program to be run deterministically. An attendee asked if CAP could just be put in the system scheduler to force determinism, and be there all the time. Tom said it an interesting idea but with drawbacks he did not elaborate on.
Present a tool called "CHESS" - can find Heisenbugs in Win32/.NET in native and managed code. Maden and Tom announced that this tool will be available externally soon at http://msdn.microsoft.com/devlabs. There is information about CHESS today at http://research.microsoft.com/projects/CHESS
CHESS/CAP uses an interleaving approach to increase test coverage. This runs a program multiple times, and hopes that changing the interleaving will expose a failure. The advantage is CHESS on CAP is its ability to reproduce the sequential debugging experience to debug a parallel bug. The disadvantage is it can miss possible failures in code is not exercised and the number of interleavings needed to fix this is never realistic (expediential explosion). Tom used the phrases ‘go get some coffee’ and ‘in a day or two’ to describe use on real programs. Tom said CHESS does not scale to large programs today, and Madan promised future work is looking at heuristics to try to reduce the search space intelligently to address this.
Competing approaches to CHESS’s interleaving approach include:
- Design a language with defined behaviors and constraints that allow a special tool. Cilk Arts does this. The advantage is it is FAST. Disadvantages include that it will not catch errors created outside of using Cilk Art’s languages extensions.
- Try to detect bugs statically. Deadlock detection can be okay this way, but data races yield too many false positives (so users always dial it back so it will miss real issues) and miss interactions outside the source code you have. Therefore, a static checking can help eliminate some bugs, but cannot be your only solution. Compilers are increasingly offering some static checking to help, but not solve this challenge of ridding a program of Heisenbugs.
- Do a comprehensive analysis tool to find all race conditions and dead lock possibilities by observing the correctness or incorrectness of the program’s runtime behavior. This is what the Intel Thread Checker does. Advantages: in practice find the bugs that matter with few false positives, finds bugs even if the program does not fail, can pinpoint the failure itself, and unlike interleaving only requires one run of a program. Disadvantages: not as fast as static solutions, can miss possible failures if code is not exercised, and does not facilitate debugging in a debugger (maybe CAP can be used in the future to help?).
Tom explained that future work using CAP includes a data-race detection engine and a tool for finding memory-model errors. In general he said you can build many more tools on top of CAP – including debug and visualization tools. Future work also includes porting to 64 bits.
Madan showed off a tool called 'Sober' - a tool for finding memory-model errors. Madan spun the fact the tool is hard to use by saying 'this tools is for very very smart people.
CHESS is currently integrated into Visual Studio Team Test only for managed code in a single process (no services). Tom said native code testing is done only from the command line right now. Both require the Team version of Visual Studio.
We’re really seeing an explosion now of tools to help with parallelism. CHESS is an interesting twist on helping find Heisenbugs which tries to force a program into failing to detect an error, and allow it to be debugged in reproducible fashion.
