There are currently few datasets appropriate for training and evaluating models for non-goal-oriented dialogue systems (chatbots); and equally problematic, there is currently no standard procedure for evaluating such models beyond the classic Turing test.
The aim of our competition is therefore to establish a concrete scenario for testing chatbots that aim to engage humans, and become a standard evaluation tool in order to make such systems directly comparable.
The Natural Language Decathlon (decaNLP) is a new benchmark for studying general NLP models that can perform a variety of complex, natural language tasks.
This year's discovery challenge presents two tasks in the new area
of social bookmarking. One task covers spam detection and
the other covers tag recommendations. As we are hosting the social bookmark and
publication sharing system BibSonomy, we are able to provide a dataset
of BibSonomy for the challenge. A training dataset for both tasks is provided at the beginning of the competition.
The test dataset will be released 48 hours before the final deadline. Due to a very tight schedule we cannot grant any deadline
extension.
The presentation of the results will take place at the ECML/PKDD workshop where the top teams are
invited to present their approaches and results.
mendation service which can be called via HTTP by BibSonomy's recommender when a user posts a bookmark or publication. All participating recommenders are called on each posting process, one of them is choosen to actually deliver the results to the user. We can then measure
J. Yamagishi, T. Nose, H. Zen, T. Toda, and K. Tokuda. Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), page 3957-3960. Las Vegas, NV, USA, (March 2008)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
A. Hotho, D. Benz, R. Jäschke, and B. Krause (Eds.) Workshop at 18th Europ. Conf. on Machine Learning (ECML'08) / 11th Europ. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD'08), (2008)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
A. Hotho, D. Benz, R. Jäschke, and B. Krause (Eds.) Workshop at 18th Europ. Conf. on Machine Learning (ECML'08) / 11th Europ. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD'08), (2008)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
L. Ratinov, and D. Roth. Proceedings of the Thirteenth Conference on Computational Natural Language Learning, page 147--155. Stroudsburg, PA, USA, Association for Computational Linguistics, (2009)
A. Hogan, E. Mu\ noz, and J. Umbrich. Proceedings of the Billion Triple Challenge 2012 (co-located with ISWC 2012, Boston, US), November 13, Boston, US, (November 2012)
S. Dlugolinsky, M. Ciglan, and M. Laclavik. Proceedings of the 17th International Conference on Intelligent Engineering Systems, page 197--202. IEEE, (2013)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
R. Jäschke, A. Hotho, F. Mitzlaff, and G. Stumme. Recommender Systems for the Social Web, volume 32 of Intelligent Systems Reference Library, Springer, Berlin/Heidelberg, (2012)
A. Hotho, D. Benz, R. Jäschke, and B. Krause (Eds.) Workshop at 18th Europ. Conf. on Machine Learning (ECML'08) / 11th Europ. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD'08), (2008)
S. Bleul, T. Weise, and K. Geihs. Service-Oriented Computing (SOC'2009), The European Association of Software Science and Technology, (March 2009)Collocated with KiVS'09 in Kassel, Germany. Appeared in: Electronic
Communications of the EASST (ECASST), ISSN 1863-2122, volume 17, part
Global Sensor Networks (GSN 2009), The European Association of Software
Science and Technology\\
The work is online available at
http://www.it-weise.de/documents/index.html\#BWG2009TWSCAROSWSC.\\
The paper can be downloaded at
http://www.it-weise.de/documents/files/BWG2009TWSCAROSWSC.pdf.\\
The presentation can be downloaded at
http://www.it-weise.de/documents/files/BWG2009TWSCAROSWSC\_pres.pdf.\\
Contact Thomas Weise at tweise@gmx.de or http://www.it-weise.de/..
A. Hotho, D. Benz, R. Jäschke, and B. Krause (Eds.) Workshop at 18th Europ. Conf. on Machine Learning (ECML'08) / 11th Europ. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD'08), (2008)
I. Zaihrayeu, L. Sun, F. Giunchiglia, W. Pan, Q. Ju, M. Chi, and X. Huang. Proceedings of the 6th International Semantic Web Conference and 2nd Asian Semantic Web Conference (ISWC/ASWC2007), Busan, South Korea, volume 4825 of LNCS, page 617--630. Berlin, Heidelberg, Springer Verlag, (November 2007)