This paper argues the position that an essential approach to the advancement of the state of the art in cognitive systems is to focus on systems that deeply integrate knowledge representations, cognitive capabilities, and knowledge content. Integration is the path to aggregating constraints in ways that improve the science of cognitive systems. However, evaluating the role of knowledge among these constraints has largely been ignored, in part because it is difficult to build and evaluate systems that incorporate large amounts of knowledge. We provide suggestions for evaluating such systems and argue that such evaluations will become easier as we come closer to applying usefully new, integrated learning mechanisms that are capable of acquiring large and effective knowledge bases.
Reference:
Jones, R. M., & Wray, R. E. (2011). Evaluating integrated, knowledge-rich cognitive systems. Proceedings of the AAAI Symposium on Advanced Cognitive Systems.