Faculty of Physics, Hanoi University of Science

English (United Kingdom)

San Francisco Declaration on Research Assessment

E-mail Print PDF
User Rating: / 2
PoorBest 
There are no translations available.

 

San Francisco Declaration on Research Assessment
Putting science into the assessment of research


There is a pressing need to improve the ways in which the output of scientific research is evaluated by funding agencies, academic institutions, and other parties.
To address this issue, a group of editors and publishers of scholarly journals met during the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco, CA, on December 16, 2012. The group developed a set of recommendations, referred to as the San Francisco Declaration on Research Assessment. We invite interested parties across all scientific disciplines to indicate their support by adding their names to this Declaration. The outputs from scientific research are many and varied, including: research articles reporting new knowledge, data, reagents, and software; intellectual property; and highly trained young scientists. Funding agencies, institutions that employ scientists, and scientists themselves, all have a desire, and need, to assess the quality and impact of scientific outputs. It is thus imperative that scientific output is measured accurately and evaluated wisely.

The Journal Impact Factor is frequently used as the primary parameter with which to compare the scientific output of individuals and institutions. The Journal Impact Factor, as calculated by Thomson Reuters,* was originally created as a tool to help librarians identify journals to purchase, not as a measure of the scientific quality of research in an article. With that in mind, it is critical to understand that the Journal Impact Factor has a number of well-documented deficiencies as a tool for research assessment. These limitations include:

A) citation distributions within journals are highly skewed [1–3];

B) the properties of the Journal Impact Factor are field-specific: it is a composite of multiple, highly diverse article types, including primary research papers and reviews [1, 4];

C) Journal Impact Factors can be manipulated (or “gamed”) by editorial policy [5]; and

D) data used to calculate the Journal Impact Factors are neither transparent nor openly available to the public [4, 6, 7].

Below we make a number of recommendations for improving the way in which the quality of research output is evaluated. Outputs other than research articles will grow in importance in assessing research effectiveness in the future, but the peer-reviewed research paper will remain a central research output that informs research assessment.
Our recommendations therefore focus primarily on practices relating to research articles published in peer-reviewed journals but can and should be extended by recognizing additional products, such as datasets, as important research outputs. These recommendations are aimed at funding agencies, academic institutions, journals,
organizations that supply metrics, and individual researchers.


A number of themes run through these recommendations:
‐ the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations.
‐ the need to assess research on its own merits rather than on the basis of the journal in which the research is published, and
‐ the need to capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).


We recognize that many funding agencies, institutions, publishers, and researchers are already encouraging improved practices in research assessment. Such steps are beginning to increase the momentum toward more sophisticated and meaningful approaches to research evaluation that can now be built upon and adopted by all of the key constituencies involved. The signatories of the San Francisco Declaration on Research Assessment support the adoption of the following practices in research assessment.

General Recommendation

1. Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

For funding agencies
2. Be explicit about the criteria used in evaluating the scientific productivity of grant applicants and clearly highlight, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published.
3. For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.

For institutions
4. Be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published.
5. For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.


For publishers
6. Greatly reduce emphasis on the journal impact factor as a promotional tool, ideally by ceasing to promote the impact factor or by presenting the metric in the context of a variety of journal-based metrics (e.g., 5-year impact factor, EigenFactor [8], SCImago [9], h-index, editorial and publication times, etc.) that provide a richer view of journal performance.
7. Make available a range of article-level metrics to encourage a shift toward assessment based on the scientific content of an article rather than publication metrics of the journal in which it was published.
8. Encourage responsible authorship practices and the provision of information about the specific contributions of each author.
9. Whether a journal is open-access or subscription-based, remove all reuse limitations on reference lists in research articles and make them available under the Creative Commons Public Domain Dedication [10].
10. Remove or reduce the constraints on the number of references in research articles, and, where appropriate, mandate the citation of primary literature in favor of reviews in order to give credit to the group(s) who first reported a finding.

For organizations that supply metrics
11. Be open and transparent by providing data and methods used to calculate all metrics.
12. Provide the data under a licence that allows unrestricted reuse, and provide computational access to data, where possible.
13. Be clear that inappropriate manipulation of metrics will not be tolerated; be explicit about what constitutes inappropriate manipulation and what measures will be taken to combat this.
14. Account for the variation in article types (e.g., reviews versus research articles), and in different subject areas when metrics are used, aggregated, or compared.

For researchers
15. When involved in committees making decisions about funding, hiring, tenure, or promotion, make assessments based on scientific content rather than publication metrics.
16. Wherever appropriate, cite primary literature in which observations are first reported rather than reviews in order to give credit where credit is due.
17. Use a range of article metrics and indicators on personal/supporting statements, as evidence of the impact of individual published articles and other research outputs [11].
18. Challenge research assessment practices that rely inappropriately on Journal Impact Factors and promote and teach best practice that focuses on the value and influence of specific research outputs.

References

1. Adler, R., Ewing, J., and Taylor, P. (2008) Citation statistics. A report from the International Mathematical Union. www.mathunion.org/publications/report/citationstatistics0
2. Seglen, P.O. (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314, 498–502.
3. Editorial (2005). Not so deep impact. Nature 435, 1003–1004.
4. Vanclay, J.K. (2012) Impact Factor: Outdated artefact or stepping-stone to journal certification. Scientometric 92, 211–238.
5. The PLoS Medicine Editors (2006). The impact factor game. PLoS Med 3(6): e291 doi:10.1371/journal.pmed.0030291.
6. Rossner, M., Van Epps, H., Hill, E. (2007). Show me the data. J. Cell Biol. 179, 1091–1092.
7. Rossner M., Van Epps H., and Hill E. (2008). Irreproducible results: A response to Thomson Scientific. J. Cell Biol. 180, 254–255.
8. http://www.eigenfactor.org/
9. http://www.scimagojr.com/
10. http://opencitations.wordpress.com/2013/01/03/open-letter-to-publishers
11. http://altmetrics.org/tools/
*The Journal Impact Factor is now published by Clarivate Analytics.

Original Signers
Euan Adie, Altmetric LLP
Sharon Ahmad, Executive Editor, The Journal of Cell Science
Bruce Alberts, Editor-in-Chief, Science
Parker Antin, Editor-in Chief, Developmental Dynamics
Ellen Bergfeld, American Society of Agronomy, Crop Science Society of America, and
Soil Science Society of America
Stefano Bertuzzi, Executive Director, American Society for Cell Biology
Pete Binfield, Co-Founder and Publisher, PeerJ
David Botstein, Founding Editor-in-Chief of Molecular Biology of the Cell; Director
Lewis-Sigler Institute for Integrative Genomics, Princeton University
Michael Caplan, Yale University
Paul Courant, Dean of Libraries and Harold Shapiro Professor of Public Policy,
University of Michigan
Brendan Crabb, President, Association of Australian Medical Research Institutes;
Director, The Burnet Institute, Melbourne
Ana-Maria Cuervo, co-Editor-in-Chief of Aging Cell; Professor, Albert Einstein College
of Medicine
Stephen Curry
Tracey dePellegrin, Executive Editor, Genetics
Mara Dierssen, President, Spanish Society of Neuroscience
David Drubin, Editor-in-Chief, Molecular Biology of the Cell; Professor, University of
California, Berkeley
Martha Fedor, Editor-in-Chief, Journal of Biological Chemistry
Sir Alan Fersht, FRS, Medical Research Council Laboratory of Molecular Biology
László Fésüs, Chairman of Publications Committee, Federation of European
Biochemical Societies
Toni Gabaldón, Centre for Genomic Regulation
Christian Gericke, Associate Editor, BMC Health Services Research; CEO, The Wesley
Research Institute
Paul A. Gleeson, Head, Department of Biochemistry and Molecular Biology, The
University of Melbourne
Robert Graham, Executive Director, Victor Chang Cardiac Research Institute
Petra Gross, Reviews Editor, Journal of Cell Science
Peter Gunning, President, American Society for Biochemistry and Molecular Biology;
Editor-In-Chief, BioArchitecture
Lisa Hannan, Managing Editor,Traffic
Brian Hoal, Society of Economic Geologists
Jason Hoyt, Co-Founder and CEO, PeerJ
Phil Hurst, Publisher, Royal Society (UK)
Paul Hutchinson
Reinhard Jahn, Department of Neurobiology, MPI for Biophysical Chemistry; EMBO
Publications Advisory Committee (chair); EMBL Scientific Advisory Board (vice chair);
Dean, Göttingen Graduate School for Neurosciences, Biophysics, and Molecular
Biosciences
David James, Director Diabetes and Obesity Program, Garvan Institute of Medical
Research
Mark Johnston, Editor-in-Chief of GENETICS; Professor and Chair, Department of
Biochemistry and Molecular Genetics, the University of Colorado School of Medicine
Kozo Kaibuchi, Editor-in-Chief of Cell Structures and Functions (the official journal of
the Japanese Society for Cell Biology)
Karl Kuchler, Medical University Vienna, Max F. Perutz Laboratories
Pekka Lappalainen, Research Director, Institute of Biotechnology, University of Finland
W. Mark Leader, Publications Director, American Society for Cell Biology
Daniel Louvard, Director of the Research Centre Institut Curie
Vivek Malhotra, Centre for Genomic Regulation, Barcelona, Spain
Michael Marks, Co-editor, Traffic; Professor, University of Pennsylvania
Mark Marsh, Co-editor, Traffic
Thomas Marwick, Director, Menzies Research Institute Tasmania
Paul Matsudaira, National University of Singapore
Satyajit Mayor, Director, National Centre for Biological Science, Bangalore, India
Tom Misteli, Editor-in-Chief, Journal of Cell Biology
Lucia Monaco, Italian Telethon Foundation
Eric Murphy, Editor-in-Chief, Lipids, a Journal of the American Oil Chemists’ Society
Mark Patterson, Executive Director, eLife
Olivier Pourquie, Université de Strasbourg
Jason Priem, ImpactStory
Bernd Pulverer, Chief Editor of The EMBO Journal; and Head of Scientific Publications,
EMBO
Jordan Raff, President of the British Society of Cell Biology, Editor-in-Chief of Biology
Open
Brian Ray, Senior Editor, Science Magazine; Founding Editor, Science Signaling
Francisco X. Real, Spanish National Cancer Research Center and Universitat Pompeu
Fabra
Alyson Reed, Executive Director, Linguistic Society of America
Phillip J. Robinson, Head, Cell Signalling Unit, Children's Medical Research Institute
Michael Rossner, Executive Director, The Rockefeller University Press
Jean-Louis Salager, Editor-in-Chief, Journal of Surfactants and Detergents
Randy Schekman, Editor-in-Chief, eLife
Sandra Schmid, former editor, Molecular Biology of the Cell and Traffic
Michael Sheetz, Director and Principal Investigator, The Mechanobiology Institute,
Singapore
Robert Shepherd, Director, Bionics Institute, University of Melbourne
Stuart Shieber, Harvard University
Michele Solimena, Max Planck Institute, Dresden, Germany
Tom Stevens, co-Editor,Traffic; Professor, University of Oregon
Jennifer L. Stow, Deputy Director, Research, Institute for Molecular Bioscience, The
University of Queensland
Robert Tjian, President, Howard Hughes Medical Institute
Gerrit van Meer, Dean of the Faculty of Sciences, Utrecht University
Michael Way, Editor-in-Chief, Journal of Cell Science
Liz Williams, Executive Editor, The Journal of Cell Biology
Mitsuhiro Yanagida, Editor-in-Chief, Genes to Cells
Alpha Yap, Head, Division of Molecular Cell Biology, Institute for Molecular Bioscience
Marino Zerial, Max Planck Director, Max Planck Institute of Molecular Cell Biology and
Genetics, Dresden
Ya-ping Zhang, Vice-President of the Chinese Academy of Sciences

Attachments:
Download this file (San Francisco Declaration on Research Assessment - SFdora.pdf)San Francisco - DORA[San Francisco Declaration on Research Assessment ]74 Kb