There are currently few datasets appropriate for training and evaluating models for non-goal-oriented dialogue systems (chatbots); and equally problematic, there is currently no standard procedure for evaluating such models beyond the classic Turing test.
The aim of our competition is therefore to establish a concrete scenario for testing chatbots that aim to engage humans, and become a standard evaluation tool in order to make such systems directly comparable.
J. Yamagishi, T. Nose, H. Zen, T. Toda, and K. Tokuda. Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), page 3957-3960. Las Vegas, NV, USA, (March 2008)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
A. Hotho, D. Benz, R. Jäschke, and B. Krause (Eds.) Workshop at 18th Europ. Conf. on Machine Learning (ECML'08) / 11th Europ. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD'08), (2008)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
A. Hotho, D. Benz, R. Jäschke, and B. Krause (Eds.) Workshop at 18th Europ. Conf. on Machine Learning (ECML'08) / 11th Europ. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD'08), (2008)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
L. Ratinov, and D. Roth. Proceedings of the Thirteenth Conference on Computational Natural Language Learning, page 147--155. Stroudsburg, PA, USA, Association for Computational Linguistics, (2009)
A. Hogan, E. Mu\ noz, and J. Umbrich. Proceedings of the Billion Triple Challenge 2012 (co-located with ISWC 2012, Boston, US), November 13, Boston, US, (November 2012)
S. Dlugolinsky, M. Ciglan, and M. Laclavik. Proceedings of the 17th International Conference on Intelligent Engineering Systems, page 197--202. IEEE, (2013)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)