This paper presents a framework for modeling and optimizing max algorithms commonly used in crowdsourcing. The model allows tuning the number of human responses needed and the size of item sets to be compared for achieving a certain level of quality given time and budget constraints.
Y. Li, A. Wen, Q. Lin, R. Li, and Z. Lu. Web-Age Information Management, volume 6897 of Lecture Notes in Computer Science, Springer, Berlin/Heidelberg, (2011)
X. Chai, B. Vuong, A. Doan, and J. Naughton. Proceedings of the 35th SIGMOD international conference on Management of data, page 87--100. New York, NY, USA, ACM, (2009)