There are currently few datasets appropriate for training and evaluating models for non-goal-oriented dialogue systems (chatbots); and equally problematic, there is currently no standard procedure for evaluating such models beyond the classic Turing test.
The aim of our competition is therefore to establish a concrete scenario for testing chatbots that aim to engage humans, and become a standard evaluation tool in order to make such systems directly comparable.
J. Yamagishi, T. Nose, H. Zen, T. Toda, и K. Tokuda. Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), стр. 3957-3960. Las Vegas, NV, USA, (марта 2008)
R. Jäschke, A. Hotho, F. Mitzlaff, и G. Stumme. Recommender Systems for the Social Web, том 32 из Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)