A Summary of Previous Grand Challenge Proposals for Cognitive Systems

The notion of a Grand Challenge (GC) in computational cognition is not new. It has been addressed both specifically and in the context of GCs in computing as a whole. One well-known example, DARPA's Autonomous Vehicle GC (AVGC), has captured the imagination of the media and the public. The AVGC is much more than a compelling research goal or a way to make DARPA's work relevant to the average layperson; it is a measurable test that can tell us where to focus our work and how much we have accomplished. The AVGC has raised the bar for what it means for a GC to set the agenda for a field of research. There have been previous efforts to develop GCs for computer science, but none of these efforts has addressed directly the needs of DARPA IPTO, in particular, demonstrations of cognitive capabilities with a dimension in learning. To gain insight into why no proposal has yet to become an IPTO GC, the authors performed a historical review and analysis of several sources of GCs in cognitive systems and artificial intelligence. This document summarizes and characterizes these previous GC explorations and evaluates categories of proposals against the DARPA IPTO criteria for selecting a GC. The authors chose to focus their analysis on task-based GCs as the most appropriate for IPTO. Task-based GCs are more likely to be organized around a goal whose achievement can be measured and is decomposable and diagnostic, and whose usefulness and relevance is clear. An example of one such task-based GCs is Lead an Orienteering Team to Victory. For purposes of discussion, they have clustered all GC proposals into categories. They then evaluated each proposal against the criteria for selecting an IPTO GC and summarized these evaluations, by category. Version 1.4.