User menu

Accès à distance ? S'identifier sur le proxy UCLouvain

A multi-disciplinary perspective on emergent and future innovations in peer review.

Primary tabs

download
  • Open access
  • PDF
  • 3.88 M
Tennant, Jonathan P Dugan, Jonathan M Graziotin, Daniel Jacques, Damien [UCL] Waldner, François [UCL] Colomb, Julien

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

Document type Article de périodique (Journal article) – Synthèse de littérature
Access type Accès libre
Publication date 2017
Language Anglais
Journal information "F1000Research" - Vol. 6, p. 1151 [1-64] (2017)
Peer reviewed yes
Publisher Faculty of 1000 Ltd. ((United Kingdom) London)
e-issn 2046-1402
Publication status Publié
Affiliation UCL - SST/ELI - Earth and Life Institute
Keywords Incentives ; Open Peer Review ; Open Science ; Quality Control ; Scholarly Publishing ; Social Media ; Web 2.0
Links
  1. Kovanis Michail, Trinquart Ludovic, Ravaud Philippe, Porcher Raphaël, Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication, 10.1007/s11192-017-2375-1
  2. V Barbour (2017)
  3. Zuckerman Harriet, Merton Robert K., Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system, 10.1007/bf01553188
  4. N Zamiska, Wall Str J. (2006)
  5. Yli-Huumo Jesse, Ko Deokyoon, Choi Sujin, Park Sooyong, Smolander Kari, Where Is Current Research on Blockchain Technology?—A Systematic Review, 10.1371/journal.pone.0163477
  6. Yeo Sara K., Liang Xuan, Brossard Dominique, Rose Kathleen M., Korzekwa Kaine, Scheufele Dietram A., Xenos Michael A., The case of #arseniclife: Blogs and Twitter in informal peer review, 10.1177/0963662516649806
  7. Yarkoni Tal, Designing next-generation platforms for evaluating scientific output: what scientists can learn from the social web, 10.3389/fncom.2012.00072
  8. Xiao Lu, Askin Nicole, Academic opinions of Wikipedia and Open Access publishing, 10.1108/oir-04-2013-0062
  9. Xiao Lu, Askin Nicole, Wikipedia for academic publishing: advantages and challenges, 10.1108/14684521211241396
  10. Wodak Shoshana J., Mietchen Daniel, Collings Andrew M., Russell Robert B., Bourne Philip E., Topic Pages: PLoS Computational Biology Meets Wikipedia, 10.1371/journal.pcbi.1002446
  11. (2017)
  12. Wicherts Jelte M., Peer Review Quality and Transparency of the Peer-Review Process in Open Access and Subscription Journals, 10.1371/journal.pone.0147913
  13. WHITTAKER R, Journal review and gender equality: a critical comment on Budden et al., 10.1016/j.tree.2008.06.003
  14. D Whaley (2017)
  15. Weicher Maureen, Peer review and secrecy in the “Information Age”, 10.1002/meet.2008.14504503155
  16. Webb Thomas J., O’Hara Bob, Freckleton Robert P., Does double-blind review benefit female authors?, 10.1016/j.tree.2008.03.003
  17. Warne Verity, Rewarding reviewers - sense or sensibility? A Wiley study explained : Rewarding reviewers - sense or sensibility?, 10.1002/leap.1002
  18. Ware Mark, Peer Review: Recent Experience and Future Directions, 10.1080/13614576.2011.566812
  19. Ware Mark, Peer review in scholarly journals: Perspective of the scholarly community – Results from an international study, 10.3233/isu-2008-0568
  20. Wang Wei‐Tsong, Wei Zu‐Hao, Knowledge sharing in wiki communities: an empirical study, 10.1108/14684521111176516
  21. Wang Wei, Kong Xiangjie, Zhang Jun, Chen Zhen, Xia Feng, Wang Xianwen, Editorial behaviors in peer review, 10.1186/s40064-016-2601-y
  22. Walsh Elizabeth, Rooney Maeve, Appleby Louis, Wilkinson Greg, Open peer review: A randomised controlled trial, 10.1192/bjp.176.1.47
  23. Walker Richard, Rocha da Silva Pascal, Emerging trends in peer review—a survey, 10.3389/fnins.2015.00169
  24. Wakeling Simon, Willett Peter, Creaser Claire, Fry Jenny, Pinfield Stephen, Spezi Valérie, Open-Access Mega-Journals: A Bibliometric Profile, 10.1371/journal.pone.0165359
  25. (2017)
  26. M von Muhlen (2011)
  27. Vitolo Claudia, Elkhatib Yehia, Reusser Dominik, Macleod Christopher J.A., Buytaert Wouter, Web technologies for environmental Big Data, 10.1016/j.envsoft.2014.10.007
  28. Vines Timothy H., The Core Inefficiency of Peer Review and a Potential Solution, 10.1002/lob.10022
  29. T Vines, The Molecular Ecologist. (2015a)
  30. van Rooyen Susan, Godlee Fiona, Evans Stephen, Smith Richard, Black Nick, Effect of Blinding and Unmasking on the Quality of Peer Review : A Randomized Trial, 10.1001/jama.280.3.234
  31. van Rooyen S., Godlee F., Evans S., Black N., Smith R., Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial, 10.1136/bmj.318.7175.23
  32. van Rooyen S., Delamothe T., Evans S. J. W., Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial, 10.1136/bmj.c5729
  33. Van Noorden Richard, Web of Science owner buys up booming peer-review platform, 10.1038/nature.2017.22094
  34. van Assen Marcel A. L. M., van Aert Robbie C. M., Nuijten Michèle B., Wicherts Jelte M., Why Publishing Everything Is More Effective than Selective Publishing of Statistically Significant Results, 10.1371/journal.pone.0084896
  35. J Ubois, 1-35 (2003)
  36. Tregenza Tom, Gender bias in the refereeing process?, 10.1016/s0169-5347(02)02545-4
  37. K Torpey, Inside Bitcoins. (2015)
  38. Tomkins Andrew, Zhang Min, Heavlin William D., Reviewer bias in single- versus double-blind peer review, 10.1073/pnas.1707323114
  39. Thung F., Bissyande T. F., Lo D., Lingxiao Jiang, Network Structure of Social Coding in GitHub, 10.1109/csmr.2013.41
  40. Thompson Neil, Hanley Douglas, Science Is Shaped by Wikipedia: Evidence from a Randomized Control Trial, 10.2139/ssrn.3039505
  41. Teytelman Leonid, Stoliartchouk Alexei, Kindler Lori, Hurwitz Bonnie L., Protocols.io: Virtual Communities for Protocol Development and Discussion, 10.1371/journal.pbio.1002538
  42. Tennant Jonathan P., Waldner François, Jacques Damien C., Masuzzo Paola, Collister Lauren B., Hartgerink Chris. H. J., The academic, economic and societal impacts of Open Access: an evidence-based review, 10.12688/f1000research.8460.3
  43. Tennant Jonathan, , The Dark Side of Peer Review, 10.18243/eon/2017.10.8.1
  44. Tausczik Yla R., Kittur Aniket, Kraut Robert E., Collaborative problem solving : a study of MathOverflow, 10.1145/2531602.2531690
  45. C Szegedy (2014)
  46. M Swan (2015)
  47. C Sutton (2017)
  48. Stemmle Laura, Collier Keith, RUBRIQ: tools, services, and software to improve peer review, 10.1087/20130406
  49. Steen R. Grant, Casadevall Arturo, Fang Ferric C., Why Has the Number of Scientific Retractions Increased?, 10.1371/journal.pone.0068397
  50. Report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion, 10.1632/prof.2007.2007.1.9
  51. Squazzoni Flaminio, Grimaldo Francisco, Marušić Ana, Publishing: Journals could share peer-review data, 10.1038/546352a
  52. Squazzoni Flaminio, Brezis Elise, Marušić Ana, Scientometrics of peer review, 10.1007/s11192-017-2518-4
  53. Spier Ray, The history of the peer-review process, 10.1016/s0167-7799(02)01985-6
  54. P Sobkowicz (2008)
  55. Snodgrass Richard T., Editorial : Single- versus double-blind reviewing, 10.1145/1206049.1206050
  56. Snell Linda, Spencer John, Reviewers' perceptions of the peer review process for a medical education journal, 10.1111/j.1365-2929.2004.02026.x
  57. Smith Richard, Classical peer review: an empty gun, 10.1186/bcr2742
  58. Smith Richard, Peer Review: A Flawed Process at the Heart of Science and Journals, 10.1177/014107680609900414
  59. Smith John W T, The deconstructed journal - a new model for academic publishing, 10.1087/09531519950145896
  60. Smith Arfon M., Katz Daniel S., Niemeyer Kyle E., , Software citation principles, 10.7717/peerj-cs.86
  61. D Singh Chawla, Retraction Watch. (2016)
  62. Siler Kyle, Strang David, Peer Review and Scholarly Originality : Let 1,000 Flowers Bloom, but Don’t Step on Any, 10.1177/0162243916656919
  63. Siler Kyle, Lee Kirby, Bero Lisa, Measuring the effectiveness of scientific gatekeeping, 10.1073/pnas.1418218112
  64. Siler Kyle, Lee Kirby, Bero Lisa, Measuring the effectiveness of scientific gatekeeping, 10.1073/pnas.1418218112
  65. Shuttleworth Sally, Charnley Berris, Science periodicals in the nineteenth and twenty-first centuries, 10.1098/rsnr.2016.0026
  66. Shotton David, The Five Stars of Online Journal Articles - a Framework for Article Evaluation, 10.1045/january2012-shotton
  67. Seeber Marco, Bacchelli Alberto, Does single blind peer review hinder newcomers?, 10.1007/s11192-017-2264-7
  68. Schroter Sara, Differences in Review Quality and Recommendations for Publication Between Peer Reviewers Suggested by Authors or by Editors, 10.1001/jama.295.3.314
  69. Schroter Sara, Black Nick, Evans Stephen, Carpenter James, Godlee Fiona, Smith Richard, Effects of training on quality of peer review: randomised controlled trial, 10.1136/bmj.38023.700775.ae
  70. B Schmidt, 62-74 (2017)
  71. Schiermeier Quirin, ‘You never said my peer review was confidential’ — scientist challenges publisher, 10.1038/nature.2017.21342
  72. L Sanger, 307-338 (2005)
  73. Salager-Meyer Françoise, Writing and publishing in peripheral scholarly journals: How to enhance the global influence of multilingual scholars?, 10.1016/j.jeap.2013.11.003
  74. Salager-Meyer Françoise, Scientific publishing in developing countries: Challenges for the future, 10.1016/j.jeap.2008.03.009
  75. Rughiniş Răzvan, Matei Stefania, Digital Badges: Signposts and Claims of Achievement, Communications in Computer and Information Science (2013) ISBN:9783642394751 p.84-88, 10.1007/978-3-642-39476-8_18
  76. Ross-Hellauer Tony, What is open peer review? A systematic review, 10.12688/f1000research.11369.1
  77. N Ross, rOpenSci Blog. (2016)
  78. Ross Joseph S., Gross Cary P., Desai Mayur M., Hong Yuling, Grant Augustus O., Daniels Stephen R., Hachinski Vladimir C., Gibbons Raymond J., Gardner Timothy J., Krumholz Harlan M., Effect of Blinded Peer Review on Abstract Acceptance, 10.1001/jama.295.14.1675
  79. Rodríguez-Bravo Blanca, Nicholas David, Herman Eti, Boukacem-Zeghmouri Chérifa, Watkinson Anthony, Xu Jie, Abrizah Abdullah, Świgoń Marzena, Peer review: The experience and views of early career researchers : Peer review and early career researchers, 10.1002/leap.1111
  80. Rodriguez Marko A., Bollen Johan, Van de Sompel Herbert, The convergence of digital libraries and the peer-review process, 10.1177/0165551506062327
  81. Rodriguez Marko A., Bollen Johan, An algorithm to determine peer-reviewers, 10.1145/1458082.1458127
  82. Rodgers Peter, Decisions, decisions, 10.7554/elife.32011
  83. Roberts Seán G., Verhoef Tessa, Double-blind reviewing at EvoLang 11 reveals gender bias, 10.1093/jole/lzw009
  84. Riggs Jack E., Priority, Rivalry, and Peer Review, 10.1177/088307389501000325
  85. (2016)
  86. (2008)
  87. D Rennie (2003)
  88. Raaper Rille, Academic perceptions of higher education assessment processes in neoliberal academia, 10.1080/17508487.2015.1019901
  89. Putman Tim E., Burgstaller-Muehlbacher Sebastian, Waagmeester Andra, Wu Chunlei, Su Andrew I., Good Benjamin M., Centralizing content and distributing labor: a community model for curating the very long tail of microbial genomes, 10.1093/database/baw028
  90. G Pullum, Nat Lang Linguist Th., 2, 261-267 (1984)
  91. (2016)
  92. R Procter (2010b)
  93. Procter R., Williams R., Stewart J., Poschen M., Snee H., Voss A., Asgari-Targhi M., Adoption and use of Web 2.0 in scholarly communications, 10.1098/rsta.2010.0155
  94. Priem Jason, Hemminger Bradley M., Decoupling the scholarly journal, 10.3389/fncom.2012.00019
  95. Priem Jason, Hemminger Bradely H., Scientometrics 2.0: New metrics of scholarly impact on the social Web, 10.5210/fm.v15i7.2874
  96. Priem Jason, Beyond the paper : Scholarship, 10.1038/495437a
  97. Prechelt Lutz, Graziotin Daniel, Fernández Daniel Méndez, A community’s perspective on the status and future of peer review in software engineering, 10.1016/j.infsof.2017.10.019
  98. Pöschl Ulrich, Multi-Stage Open Peer Review: Scientific Evaluation Integrating the Strengths of Traditional Peer Review with the Virtues of Transparency and Self-Regulation, 10.3389/fncom.2012.00033
  99. Pontille David, Torny Didier, From Manuscript Evaluation to Article Valuation: The Changing Technologies of Journal Peer Review, 10.1007/s10746-014-9335-z
  100. D Pontille, Ada: A Journal of Gender, New Media, and Technology. (2014)
  101. Pocock Stuart J., Hughes Michael D., Lee Robert J., Statistical Problems in the Reporting of Clinical Trials, 10.1056/nejm198708133170706
  102. A Plume, Research Trends., 38 (2014)
  103. S Pinfield, Times Higher Education. (2016)
  104. Pierie Jean-Pierre EN, Walvoort Henk C, Overbeke A John PM, Readers' evaluation of effect of peer review and editing on quality of articles in the Nederlands Tijdschrift voor Geneeskunde, 10.1016/s0140-6736(96)05016-7
  105. Peters Douglas P., Ceci Stephen J., Peer-review practices of psychological journals: The fate of published articles, submitted again, 10.1017/s0140525x00011183
  106. Perkel Jeffrey M., Annotating the scholarly web, 10.1038/528153a
  107. Perakakis Pandelis, Taylor Michael, Mazza Marco, Trachana Varvara, Natural selection of academic papers, 10.1007/s11192-010-0253-1
  108. Penev Lyubomir, Mietchen Daniel, Chavan Vishwas, Hagedorn Gregor, Smith Vincent, Shotton David, Ó Tuama Éamonn, Senderov Viktor, Georgiev Teodor, Stoev Pavel, Groom Quentin, Remsen David, Edmunds Scott, Strategies and guidelines for scholarly publishing of biodiversity data, 10.3897/rio.3.e12431
  109. Patel Jigisha, Why training and specialization is needed for peer review: a case study of peer review for randomized controlled trials, 10.1186/s12916-014-0128-z
  110. Parnell Laurence D., Lindenbaum Pierre, Shameer Khader, Dall'Olio Giovanni Marco, Swan Daniel C., Jensen Lars Juhl, Cockell Simon J., Pedersen Brent S., Mangan Mary E., Miller Christopher A., Albert Istvan, BioStar: An Online Question & Answer Resource for the Bioinformatics Community, 10.1371/journal.pcbi.1002216
  111. S Palus (2015)
  112. Pallavi Sudhir Abhimanyu, Knöpfel Rahel, PhysicsOverflow: A postgraduate-level physics Q&A site and open peer review system, 10.1142/s2251158x15000193
  113. Paglione Laura Dorival, Lawrence Rebecca Naomi, Data exchange standards to support and acknowledge peer-review activity, 10.1087/20150411
  114. S Owens, Sci Am. (2014)
  115. J Overbeke, 32-44 (1999)
  116. Oldenburg H., Epistle Dedicatory, 10.1098/rstl.1665.0001
  117. Okike Kanu, Hug Kevin T., Kocher Mininder S., Leopold Seth S., Single-blind vs Double-blind Peer Review in the Setting of Author Prestige, 10.1001/jama.2016.11014
  118. Nosek Brian A., Lakens Daniël, Registered Reports : A Method to Increase the Credibility of Published Results, 10.1027/1864-9335/a000192
  119. Nobarany Syavash, Booth Kellogg S., Understanding and supporting anonymity policies in peer review, 10.1002/asi.23711
  120. J Nicholson, The Winnower. (2016)
  121. Neylon Cameron, Wu Shirley, Article-Level Metrics and the Evolution of Scientific Impact, 10.1371/journal.pbio.1000242
  122. Promoting reproducibility with registered reports, 10.1038/s41562-016-0034
  123. Response required, 10.1038/468867a
  124. S Nakamoto (2008)
  125. Murphy Tony, Sage Daniel, Perceptions of the UK’s Research Excellence Framework 2014: a media analysis, 10.1080/1360080x.2014.957890
  126. Munafò Marcus R., Nosek Brian A., Bishop Dorothy V. M., Button Katherine S., Chambers Christopher D., Percie du Sert Nathalie, Simonsohn Uri, Wagenmakers Eric-Jan, Ware Jennifer J., Ioannidis John P. A., A manifesto for reproducible science, 10.1038/s41562-016-0021
  127. Mulligan Adrian, Hall Louise, Raphael Ellen, Peer review in a changing world: An international study measuring the attitudes of researchers, 10.1002/asi.22798
  128. Mulligan Adrian, Akerman Richard, Granier Bruno, Tamber Pritpal S., Pöschl Ulrich, Quality, certification and peer review, 10.3233/isu-2008-0582
  129. Mudambi, Schuff, Research Note: What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.com, 10.2307/20721420
  130. N Moxham, Hist J. (2017)
  131. Morrison Jill, The case for open peer review, 10.1111/j.1365-2929.2006.02573.x
  132. Moore Samuel, Neylon Cameron, Paul Eve Martin, Paul O’Donnell Daniel, Pattinson Damian, “Excellence R Us”: university research and the fetishisation of excellence, 10.1057/palcomms.2016.105
  133. Moher David, Galipeau James, Alam Sabina, Barbour Virginia, Bartolomeos Kidist, Baskin Patricia, Bell-Syer Sally, Cobey Kelly D., Chan Leighton, Clark Jocalyn, Deeks Jonathan, Flanagin Annette, Garner Paul, Glenny Anne-Marie, Groves Trish, Gurusamy Kurinchi, Habibzadeh Farrokh, Jewell-Thomas Stefanie, Kelsall Diane, Lapeña José Florencio, MacLehose Harriet, Marusic Ana, McKenzie Joanne E., Shah Jay, Shamseer Larissa, Straus Sharon, Tugwell Peter, Wager Elizabeth, Winker Margaret, Zhaori Getu, Core competencies for scientific editors of biomedical journals: consensus statement, 10.1186/s12916-017-0927-0
  134. Moed Henk F., The effect of “open access” on citation impact: An analysis of ArXiv's condensed matter section, 10.1002/asi.20663
  135. D Mietchen, Wellcome Open Res. (2017)
  136. Ní Mhurchú Aoileann, McLeod Laura, Collins Stephanie, Siles-Brügge Gabriel, The Present and the Future of the Research Excellence Framework Impact Agenda in the UK Academy: A Reflection from Politics and International Studies, 10.1177/1478929916658918
  137. R Merton (1973)
  138. Merton R. K., The Matthew Effect in Science: The reward and communication systems of science are considered, 10.1126/science.159.3810.56
  139. R Melero, Food Sci Technol Int., 7, 521-527 (2001)
  140. McNutt Robert A., The Effects of Blinding on the Quality of Peer Review : A Randomized Trial, 10.1001/jama.1990.03440100079012
  141. G McKiernan (2003)
  142. McKiernan Erin C, Bourne Philip E, Brown C Titus, Buck Stuart, Kenall Amye, Lin Jennifer, McDougall Damon, Nosek Brian A, Ram Karthik, Soderberg Courtney K, Spies Jeffrey R, Thaney Kaitlin, Updegrove Andrew, Woo Kara H, Yarkoni Tal, How open science helps researchers succeed, 10.7554/elife.16800
  143. N McCormack, Law Libr J., 101, 59 (2009)
  144. Mayden MSN, FNP, AOCNP® Kelley D., Peer Review: Publication’s Gold Standard, 10.6004/jadpro.2012.3.2.8
  145. M Marra, 100-117 (2017)
  146. Margalida Antoni, Colomer Mª Àngels, Improving the peer-review process and editorial quality: key errors escaping the review and editorial process in top scientific journals, 10.7717/peerj.1670
  147. A Manten, Development of science publishing in Europe., 1-22 (1980)
  148. Mahoney Michael J., Publication prejudices: An experimental study of confirmatory bias in the peer review system, 10.1007/bf01173636
  149. Maharg Paul, Duncan Nigel, Black Box, Pandora's Box or Virtual Toolbox? An Experiment in a Journal's Transparent Peer Review on the Web, 10.1080/13600860701492104
  150. Magee Joe C., Galinsky Adam D., 8 Social Hierarchy: The Self‐Reinforcing Nature of Power and Status, 10.1080/19416520802211628
  151. Lyman R. Lee, A Three-Decade History of the Duration of Peer Review, 10.3138/jsp.44.3.001
  152. Luzi Daniela, Trends and evolution in the development of grey literature: a review, 10.1108/14666180010345537
  153. Lui Kim Man, Chan Keith C.C., Pair programming productivity: Novice–novice vs. expert–expert, 10.1016/j.ijhcs.2006.04.010
  154. Lloyd Margaret E., Gender factors in reviewer recommendations for manuscript publication, 10.1901/jaba.1990.23-539
  155. List Benjamin, Crowd-based peer review can be good and fast, 10.1038/546009a
  156. Lipworth Wendy, Kerridge Ian H., Carter Stacy M., Little Miles, Should Biomedical Publishing Be “Opened Up”? Toward a Values-Based Peer-Review Process, 10.1007/s11673-011-9312-4
  157. Lipworth Wendy, Kerridge Ian, Shifting Power Relations and the Ethics of Journal Peer Review, 10.1080/02691728.2010.534567
  158. Link Ann M., US and Non-US Submissions : An Analysis of Reviewer Bias, 10.1001/jama.280.3.246
  159. Li Lan, Steckelberg Allen L., Srinivasan Sribhagyam, Utilizing peer interactions to promote learning through a web-based peer assessment system, 10.21432/t21c7r
  160. Lerback Jory, Hanson Brooks, Journals invite too few women to referee, 10.1038/541455a
  161. Leek Jeffrey T., Taub Margaret A., Pineda Fernando J., Cooperation between Referees and Authors Increases Peer Review Accuracy, 10.1371/journal.pone.0026895
  162. D Lee, IMM-press Magazine. (2015)
  163. Lee Carole J., Sugimoto Cassidy R., Zhang Guo, Cronin Blaise, Bias in peer review, 10.1002/asi.22784
  164. Lee Carole J., Moher David, Promote scientific integrity via journal peer review data, 10.1126/science.aan4141
  165. Larsen Peder Olesen, von Ins Markus, The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index, 10.1007/s11192-010-0202-z
  166. Larivière Vincent, Haustein Stefanie, Mongeon Philippe, The Oligopoly of Academic Publishers in the Digital Era, 10.1371/journal.pone.0127502
  167. Larivière Vincent, Sugimoto Cassidy R., Macaluso Benoit, Milojević Staša, Cronin Blaise, Thelwall Mike, arXiv E-prints and the journal of record: An analysis of roles and relationships : arXiv E-Prints and the Journal of Record, 10.1002/asi.23044
  168. Kuhn T, Peer Review #1 of "Software citation principles (v0.2)", 10.7287/peerj-cs.86v0.2/reviews/1
  169. Kuhn T, Peer Review #1 of "Software citation principles (v0.1)", 10.7287/peerj-cs.86v0.1/reviews/1
  170. Kuehn Bridget M, Rooting out bias, 10.7554/elife.32014
  171. J Kubátová, Lex ET Scientia International Journal (LESIJ)., 135-145 (2012)
  172. Kronick David A., Peer Review in 18th-Century Scientific Journalism, 10.1001/jama.1990.03440100021002
  173. Kriegeskorte Nikolaus, Walther Alexander, Deca Diana, An emerging consensus for open evaluation: 18 visions for the future of scientific publishing, 10.3389/fncom.2012.00094
  174. Kovanis Michail, Porcher Raphaël, Ravaud Philippe, Trinquart Ludovic, The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise, 10.1371/journal.pone.0166387
  175. Kostoff R. N., Federal research impact assessment: Axioms, approaches, applications, 10.1007/bf02020420
  176. A Kosner, Forbes. (2012)
  177. Klyne G, Peer Review #2 of "Software citation principles (v0.1)", 10.7287/peerj-cs.86v0.1/reviews/2
  178. M Khan, IJCISIM., 4, 283-299 (2012)
  179. Kelty C.M., Burrus C.S., Baraniuk R.G., Peer Review Anew: Three Principles and a Case Study in Postpublication Quality Assurance, 10.1109/jproc.2008.921613
  180. E Katsh, SCL Rev., 67, 329 (2015)
  181. Justice Amy C., Cho Mildred K., Winker Margaret A., Berlin Jesse A., Rennie Drummond, and the PEER Investigators, Does Masking Author Identity Improve Peer Review Quality? : A Randomized Controlled Trial, 10.1001/jama.280.3.240
  182. Jubb Michael, Peer review: The current landscape and future trends : Peer review landscape, 10.1002/leap.1008
  183. Jefferson Tom, Measuring the Quality of Editorial Peer Review, 10.1001/jama.287.21.2786
  184. Jefferson Tom, Rudin Melanie, Brodney Folse Suzanne, Davidoff Frank, Editorial peer review for improving the quality of reports of biomedical studies, 10.1002/14651858.mr000016.pub3
  185. JANOWICZ Krzysztof, HITZLER Pascal, Open and transparent: the review process of theSemantic Webjournal, 10.1087/20120107
  186. Jadad Alejandro R., Moore R.Andrew, Carroll Dawn, Jenkinson Crispin, Reynolds D.John M., Gavaghan David J., McQuay Henry J., Assessing the quality of reports of randomized clinical trials: Is blinding necessary?, 10.1016/0197-2456(95)00134-4
  187. Isenberg S J, Sanchez E, Zafran K C, The effect of masking manuscripts for the peer-review process of an ophthalmic journal, 10.1136/bjo.2008.151886
  188. Ioannidis John P. A., Why Most Published Research Findings Are False, 10.1371/journal.pmed.0020124
  189. J Hukkinen, Wired. (2017)
  190. Hu Meiqun, Lim Ee-Peng, Sun Aixin, Lauw Hady Wirawan, Vuong Ba-Quy, Measuring article quality in wikipedia : models and evaluation, 10.1145/1321440.1321476
  191. Horrobin David F., The Philosophical Basis of Peer Review and the Suppression of Innovation, 10.1001/jama.1990.03440100162024
  192. Hettyey Attila, Griggio Matteo, Mann Marlene, Raveh Shirley, Schaedelin Franziska C., Thonhauser Kerstin E., Thoß Michaela, van Dongen Wouter F.D., White Joël, Zala Sarah M., Penn Dustin J., Peerage of Science: will it work?, 10.1016/j.tree.2012.01.005
  193. Helmer Markus, Schottdorf Manuel, Neef Andreas, Battaglia Demian, Gender bias in scholarly peer review, 10.7554/elife.21718
  194. Heller Lambert, The Ronald, Bartling Sönke, Dynamic Publication Formats and Collaborative Authoring, Opening Science (2014) ISBN:9783319000251 p.191-211, 10.1007/978-3-319-00026-8_13
  195. Heaberlin Bradi, DeDeo Simon, The Evolution of Wikipedia’s Norm Network, 10.3390/fi8020014
  196. Haug Charlotte J., Peer-Review Fraud — Hacking the Scientific Publication Process, 10.1056/nejmp1512330
  197. Hasty Robert, Garvalosa Ryan, Barbato Vincenzo, Valdes Pedro, Powers David, Hernandez Emmanuel, John Jones, Suciu Gabriel, Qureshi Farheen, Popa-Radu Matei, San Jose Sergio, Drexler Nathaniel, Patankar Rohan, Paz Jose, King Christopher, Gerber Hilary, Valladares Michael, Somji Alyaz, Wikipedia vs Peer-Reviewed Medical Literature for Information About the 10 Most Costly Medical Conditions, 10.7556/jaoa.2014.035
  198. A Harmon (2010)
  199. Halavais Alexander, Kwon K. Hazel, Havener Shannon, Striker Jason, Badges of Friendship: Social Influence and Badge Acquisition on Stack Overflow, 10.1109/hicss.2014.206
  200. Haider Jutta, Åström Fredrik, Dimensions of trust in scholarly communication: Problematizing peer review in the aftermath of John Bohannon's “Sting” in science, 10.1002/asi.23669
  201. Gupta Sunetra, How has publishing changed in the last twenty years?, 10.1098/rsnr.2016.0035
  202. Gropp Robert E., Glisson Scott, Gallo Stephen, Thompson Lisa, Peer Review: A System under Stress, 10.1093/biosci/bix034
  203. Grivell Les, Through a glass darkly: The present and the future of editorial peer review, 10.1038/sj.embor.7400718
  204. Greenberg S. A, How citation distortions create unfounded authority: analysis of a citation network, 10.1136/bmj.b2680
  205. S Greaves, Nature. (2006)
  206. K Graf, Archivalia. (2014)
  207. C Goues (2017)
  208. Gøtzsche Peter C., Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis, 10.1016/0197-2456(89)90017-2
  209. Goodman Steven N., Manuscript Quality before and after Peer Review and Editing at Annals of Internal Medicine, 10.7326/0003-4819-121-1-199407010-00003
  210. Godlee Fiona, Gale Catharine R., Martyn Christopher N., Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports : A Randomized Controlled Trial, 10.1001/jama.280.3.237
  211. Godlee Fiona, Making Reviewers Visible : Openness, Accountability, and Credit, 10.1001/jama.287.21.2762
  212. Ginsparg Paul, Winners and Losers in the Global Research Village, 10.1300/j123v30n03_13
  213. Gibson Mark, Spong Catherine Y., Simonsen Sara Ellis, Martin Sheryl, Scott James R., Author Perception of Peer Review : , 10.1097/aog.0b013e31818425d4
  214. Gibney Elizabeth, Toolbox: Low-cost journals piggyback on arXiv, 10.1038/530117a
  215. Ghosh Satrajit S., Klein Arno, Avants Brian, Millman K. Jarrod, Learning from open source software projects to improve scientific review, 10.3389/fncom.2012.00018
  216. M Gashler, KDnuggets. (2008)
  217. Galipeau James, Moher David, Campbell Craig, Hendry Paul, Cameron D. William, Palepu Anita, Hébert Paul C., A systematic review highlights a knowledge gap regarding the effectiveness of health-related training programs in journalology, 10.1016/j.jclinepi.2014.09.024
  218. A Fyfe, Zenodo. (2017)
  219. Fukuzawa Naomi, Characteristics of papers published in journals: an analysis of open access journals, country of publication, and languages used, 10.1007/s11192-017-2414-y
  220. P Frishauf, J Participat Med., 1, e13a (2009)
  221. Fox Mary Frank, Scientific Misconduct and Editorial and Peer Review Processes, 10.2307/2943969
  222. Fox Charles W., Albert Arianne Y. K., Vines Timothy H., Recruitment of reviewers is becoming harder at some journals: a test of the influence of reviewer fatigue at six journals in ecology and evolution, 10.1186/s41073-017-0027-x
  223. Ford Emily, Defining and Characterizing Open Peer Review: A Review of the Literature, 10.3138/jsp.44-4-001
  224. K Fitzpatrick (2011b)
  225. Fitzpatrick Kathleen, Peer Review, Judgment, and Reading, 10.1632/prof.2011.2011.1.196
  226. Fitzpatrick Kathleen, Peer‐to‐peer Review and the Future of Scholarly Authority, 10.1080/02691728.2010.498929
  227. T Farley, James Randi Educational Foundation Swift Blog. (2011)
  228. Ernst E., Kienbacher T., Chauvinism, 10.1038/352560b0
  229. R Emilsson (2015)
  230. Elkhatib Yehia, Tyson Gareth, Sathiaseelan Arjuna, Does the Internet Deserve Everybody?, 10.1145/2793013.2793018
  231. M Eisen (2011)
  232. Edgar Brian D, Willinsky John, A Survey of Scholarly Journals Using Open Journal Systems, 10.22230/src.2010v1n2a24
  233. Eckberg Douglas Lee, When nonreliability of reviews indicates solid science, 10.1017/s0140525x00065791
  234. Rennie Drummond, Let’s make peer review scientific, 10.1038/535031a
  235. V Dhillon, Bitcoin Magazine. (2016)
  236. Davis Philip M., Fromerth Michael J., Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles?, 10.1007/s11192-007-1661-8
  237. P Davis, The Scholarly Kitchen. (2017)
  238. Darling E. S., Use of double-blind peer review to increase author diversity : Diversity and Double-Blind Review, 10.1111/cobi.12333
  239. Dappert Angela, Farquhar Adam, Kotarski Rachael, Hewlett Kirstie, Connecting the Persistent Identifier Ecosystem: Building the Technical and Human Infrastructure for Open Research, 10.5334/dsj-2017-028
  240. Daniel H.-D., Guardians of Science, ISBN:3527290419, 10.1002/3527602208
  241. D’Andrea Rafael, O’Dwyer James P., Can editors save peer review from peer reviewers?, 10.1371/journal.pone.0186111
  242. P Dall’Aglio (2006)
  243. Csiszar Alex, Peer review: Troubled from the start, 10.1038/532306a
  244. D Crotty (2009)
  245. Cope William W., Kalantzis Mary, Signs of epistemic disruption: Transformations in the knowledge system of the academic journal, 10.5210/fm.v14i4.2309
  246. S Cole, 109-142 (2000)
  247. Cohen Barak A, How should novelty be valued in science?, 10.7554/elife.28699
  248. Chevalier Judith A., Mayzlin Dina, The Effect of Word of Mouth on Sales: Online Book Reviews, 10.1509/jmkr.43.3.345
  249. Chauvin Anthony, Ravaud Philippe, Baron Gabriel, Barnes Caroline, Boutron Isabelle, The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors, 10.1186/s12916-015-0395-3
  250. Chambers Christopher D., Forstmann Birte, Pruszynski J. Andrew, Registered reports at theEuropean Journal of Neuroscience: consolidating and extending peer-reviewed study pre-registration, 10.1111/ejn.13519
  251. D. Chambers Christopher, , Feredoes Eva, D. Muthukumaraswamy Suresh, J. Etchells Peter, Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond, 10.3934/neuroscience.2014.1.4
  252. figshare. (2017)
  253. Casnici Niccolò, Grimaldo Francisco, Gilbert Nigel, Squazzoni Flaminio, Attitudes of referees in a multidisciplinary journal: An empirical analysis, 10.1002/asi.23665
  254. B Carlisle, The Grey Literature. (2014)
  255. CAMPANARIO JUAN MIGUEL, Peer Review for Journals as it Stands Today—Part 2, 10.1177/1075547098019004002
  256. CAMPANARIO JUAN MIGUEL, Peer Review for Journals as it Stands Today—Part 1, 10.1177/1075547098019003002
  257. Burris Val, The Academic Caste System: Prestige Hierarchies in PhD Exchange Networks, 10.1177/000312240406900205
  258. Burnham John C., The Evolution of Editorial Peer Review, 10.1001/jama.1990.03440100023003
  259. Burghardt Keith, Alsina Emanuel F., Girvan Michelle, Rand William, Lerman Kristina, The myopia of crowds: Cognitive load and collective evaluation of answers on Stack Exchange, 10.1371/journal.pone.0173610
  260. BUDDEN A, TREGENZA T, AARSSEN L, KORICHEVA J, LEIMU R, LORTIE C, Double-blind review favours increased representation of female authors, 10.1016/j.tree.2007.07.008
  261. Bruce Rachel, Chauvin Anthony, Trinquart Ludovic, Ravaud Philippe, Boutron Isabelle, Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis, 10.1186/s12916-016-0631-5
  262. Breuning Marijke, Backstrom Jeremy, Brannon Jeremy, Gross Benjamin Isaak, Widmeier Michael, Reviewer Fatigue? Why Scholars Decline to Review their Peers’ Work, 10.1017/s1049096515000827
  263. Brembs Björn, Button Katherine, Munafò Marcus, Deep impact: unintended consequences of journal rank, 10.3389/fnhum.2013.00291
  264. Brembs Björn, The cost of the rejection-resubmission cycle, 10.15200/winn.142497.72083
  265. Bornmann Lutz, Wolf Markus, Daniel Hans-Dieter, Closed versus open reviewing of journal manuscripts: how far do comments differ in language use?, 10.1007/s11192-011-0569-5
  266. Bornmann Lutz, Mutz Rüdiger, Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references : Growth Rates of Modern Science: A Bibliometric Analysis Based on the Number of Publications and Cited References, 10.1002/asi.23329
  267. BORNMANN L., DANIEL H.-D., Reliability of reviewers' ratings when using public peer review: a case study, 10.1087/20100207
  268. Bornmann Lutz, Daniel Hans-Dieter, How Long is the Peer Review Process for Journal Manuscripts?
    A Case Study on Angewandte Chemie International Edition    , 10.2533/chimia.2010.72
  269. M Bon (2017)
  270. Boldt Axel, Extending ArXiv.org to Achieve Open Peer Review and Publishing, 10.3138/jsp.42.2.238
  271. Blatt Michael R., Vigilante Science, 10.1104/pp.15.01443
  272. R Blank, Am Econ Rev., 81, 1041-1067 (1991)
  273. Black Erik W., Wikipedia and academic peer review : Wikipedia as a recognised medium for scholarly publication?, 10.1108/14684520810865994
  274. Biagioli Mario, From Book Censorship to Academic Peer Review, 10.1080/1045722022000003435
  275. Berthaud Christine, Capelli Laurent, Gustedt Jens, Kirchner Claude, Loiseau Kevin, Magron Agnès, Medves Maud, Monteil Alain, Rivrieux Gaëlle, Romary Laurent, EPISCIENCES – An overlay publication platform, 10.3233/isu-140749
  276. Bernstein Rachel, Updated: Sexist peer review elicits furious Twitter response, PLOS apology, 10.1126/science.aab2568
  277. Benda Wim G.G., Engels Tim C.E., The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science, 10.1016/j.ijforecast.2010.03.003
  278. Belojevic Nina, Sayers Jentery, Research Teams INKE and MVP, Peer Review Personas, 10.3998/3336451.0017.304
  279. Begel Andrew, Bosch Jan, Storey Margaret-Anne, Social Networking Meets Software Development: Perspectives from GitHub, MSDN, Stack Exchange, and TopCoder, 10.1109/ms.2013.13
  280. Bedeian Arthur G., The Manuscript Review Process : The Proper Roles of Authors, Referees, and Editors, 10.1177/1056492603258974
  281. Baxt William G, Waeckerle Joseph F, Berlin Jesse A, Callaham Michael L, Who Reviews the Reviewers? Feasibility of Using a Fictitious Manuscript to Evaluate Peer Reviewer Performance, 10.1016/s0196-0644(98)70006-x
  282. S Bartling, Zenodo. (2016)
  283. C Ball (2016)
  284. What it was like to be peer reviewed in the 1860s, 10.1063/pt.5.9098
  285. Baldwin Melinda, In referees we trust?, 10.1063/pt.3.3463
  286. Baldwin Melinda, Credibility, peer review, andNature, 1945–1990, 10.1098/rsnr.2015.0029
  287. Baggs Judith Gedney, Broome Marion E., Dougherty Molly C., Freda Margaret C., Kearney Margaret H., Blinding in peer review: the preferences of reviewers for nursing journals, 10.1111/j.1365-2648.2008.04816.x
  288. (2017)
  289. Armstrong J. Scott, Peer review for journals: Evidence on quality control, fairness, and innovation, 10.1007/s11948-997-0017-3
  290. A Antonopoulos (2014)
  291. Anderson Steve, McPherson Tara, Engaging Digital Scholarship: Thoughts on Evaluating Multimedia Scholarship, 10.1632/prof.2011.2011.1.136
  292. Alvesson Mats, Sandberg Jörgen, Habitat and Habitus: Boxed-in versus Box-Breaking Research, 10.1177/0170840614530916
  293. Almquist Martin, von Allmen Regula S., Carradice Dan, Oosterling Steven J., McFarlane Kirsty, Wijnhoven Bas, A prospective study on an innovative online forum for peer reviewing of surgical science, 10.1371/journal.pone.0179031
  294. Albert Arianne Y. K., Gow Jennifer L., Cobra Alison, Vines Timothy H., Is it becoming harder to secure reviewers for peer review? A test with data from five ecology journals, 10.1186/s41073-016-0022-7
  295. I Al-Rahawi
  296. D Adam, The Guardian. (2010)
Bibliographic reference Tennant, Jonathan P ; Dugan, Jonathan M ; Graziotin, Daniel ; Jacques, Damien ; Waldner, François ; et. al. A multi-disciplinary perspective on emergent and future innovations in peer review.. In: F1000Research, Vol. 6, p. 1151 [1-64] (2017)
Permanent URL http://hdl.handle.net/2078.1/204821