Together with several other interested individuals, I am organizing a birds-of-a-feather discussion on automated grading systems at SIGCSE'06 this March.

The primary topic of this BOF is identifying and discussing the major roadblocks to fielding a generally-adoptable automated grading platform that is flexible enough for many institutions to use. We're also hoping to bring together individuals who are interested in solving these problems (or describing why they are infeasible). We've already started the discussion via a wiki page, so if you are interested in contributing, feel free to join us on this wiki page describing obstacles to a common automated grading system.