'Amazon Mechanical Turk provides a web services API for computers to integrate "artificial artificial intelligence" directly into their processing by making requests of humans. Developers use the Amazon Mechanical Turk web service to submit tasks to the A
Bossa is an open-source software framework for distributed thinking - the use of volunteers on the Internet to perform tasks that use human cognition, knowledge, or intelligence.
"Amazon Mechanical Turk is a marketplace for work that requires human intelligence. The Mechanical Turk web service enables companies to programmatically access this marketplace and a diverse, on-demand workforce. Developers can leverage this service to build human intelligence directly into their applications." Sample Business Cases: http://aws.amazon.com/mturk/#bus-case "We had the wrong idea about computers managing without human beings. Computers have realised that the real way to make money is to employ millions of people doing jobs computers cannot do, and for pennies." (Pierre Lazuly) http://www.readwriteweb.com/archives/how_to_use_mechanical_turk_to_rock_conference_blogging.php
Performing human-subjects experiments on Amazon Mechanical Turk offers many benefits, including very low experiment costs, quick turn-around rates, and relatively simple approvals from human subjects boards. But you have to be careful to avoid bias and error.
PsiTurk is an open platform for conducting custom behvioral experiments on Amazon's Mechanical Turk.
It is intended to provide most of the backend machinery necessary to run your experiment. It uses AMT's External Question HIT type, meaning that you can collect data using any website. As long as you can turn your experiment into a website, you can run it with PsiTurk!
R. Snow, B. O'Connor, D. Jurafsky, und A. Ng. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Seite 254--263. Stroudsburg, PA, USA, Association for Computational Linguistics, (2008)
A. Kittur, E. Chi, und B. Suh. Proceeding of the 16thth annual SIGCHI conference on Human Factors in Computing Systems (CHI-08), Florence, Italy, (2008)