If you think research and knowledge are as vital to humanity as air, water, bread and freedom, then you probably know what Peer Evaluation is about.
Peer Evaluation is about giving Open Access to your primary data, working papers, articles, media and having them all reviewed and discussed by your peers. Peer evaluation is a strong supporter of qualified peer reviewing and is, in that respect, a valuable supplement, inspiration and hub for peer reviewed journals and publications. Finally, Peer Evaluation is an independent and community interest project.
It is often said that peer review is one of the pillars of scientific research. It is also well known that peer review doesn't actually do its job very well, and, every few years, people like me start writing articles about alternatives to peer review. This isn't one of those rants. Instead, I'm going to focus on something that is probably less well known: peer review actually has two jobs. It's used to provide minimal scrutiny for new scientific results, and to act as a gatekeeper for funding agencies.
What I would like to do here is outline some of the differences between peer review in these two jobs and the strengths and weaknesses of peer review in each case. This is not a rant against peer review, nor should it be—I have been pretty successful in both publications and grant applications over the last couple of years. But I think it's worth exploring the idea that peer review functions much better in the case of deciding the value of scientific research than it does when acting as a gatekeeper for scientific funding.
Just a small number of bad referees can significantly undermine the ability of the peer-review system to select the best scientific papers. That is according to a pair of complex systems researchers in Austria who have modelled an academic publishing system and showed that human foibles can have a dramatic effect on the quality of published science.
Anyone who thinks peer review is a process of nudges and winks from your mates has never faced the harsh reality of having your work pulled apart, says Jenny Rohn (who has).