Educate yourself about inter-rater reliability assessment, and the limitations of the kappa statistic. By Dr. Kilem Gwet, a senior statistical consultant.

http://www.stataxis.com/interrater.htm