Papers

Despite the surprising power of many modern AI systems that often learn their own representations, there is significant discontent about their inscrutability and the attendant problems in their ability to interact with humans. While alternatives such as neuro-symbolic approaches have been proposed, there is a lack of consensus on what they are about. There are often two independent motivations (i) symbols as a lingua franca for human-AI interaction and (ii) symbols as (system-produced) abstractions use in its internal reasoning. The jury is still out on whether AI systems will need to use symbols in their internal reasoning to achieve general intelligence capabilities. Whatever the answer there is, the need for (human-understandable) symbols in human-AI interaction seems quite compelling. Symbols, like emotions, may well not be sine qua non for intelligence per se, but they will be crucial for AI systems to interact with us humans--as we can neither turn off our emotions not get by without our symbols. In particular, in many human-designed domains, humans would be interested in providing explicit (symbolic) knowledge and advice--and expect machine explanations in kind. This alone requires AI systems to at least do their I/O in symbolic terms. In this blue sky paper, we argue this point of view, and discuss research directions that need to be pursued to allow for this type of human-AI interaction.

poster session 4 @ blue 1, poster session 11 @ blue 1, oral session 4 @ blue 1, poster session 4, poster session 11, oral session 4

In this paper, I pose a major challenge for AI researchers: to develop systems that learn in a human-like manner. I briefly review the history of machine learning, noting that early work made close contact with results from cognitive psychology but that this is no longer the case. I identify seven characteristics of human behavior that, if reproduced, would offer better ways to acquire expertise than statistical induction over massive training sets. I illustrate these points with two domains - mathematics and driving - where people are effective learners and review systems that address them. In closing, I suggest ways to encourage more research on human-like learning.

poster session 4 @ blue 1, poster session 11 @ blue 1, oral session 4 @ blue 1, poster session 4, poster session 11, oral session 4

Many present and future problems associated with artificial intelligence are not due to its limitations, but to our poor assessment of its behaviour. Our evaluation procedures produce aggregated performance metrics that lack detail and quantified uncertainty about the following question: how will an AI system, with a particular profile \pi, behave for a new problem, characterised by a particular situation \mu? Instead of just aggregating test results, we can use machine learning methods to fully capitalise on this evaluation information. In this paper, we introduce the concept of an assessor model, \hat{R}(r|\pi,\mu), a conditional probability estimator trained on test data. We discuss how these assessors can be built by using information of the full system-problem space and illustrate a broad range of applications that derive from varied inferences and aggregations from \hat{R}. Building good assessor models will change the predictive and explanatory power of AI evaluation and will lead to new research directions for building and using them. We propose accompanying every deployed AI system with its own assessor.

poster session 4 @ blue 1, poster session 11 @ blue 1, oral session 4 @ blue 1, poster session 4, poster session 11, oral session 4