Chapter 3 Open Science as a Step Toward Social Responsivity in Research
Seattle Pacific University
Seattle Pacific University was built on the unceded ancestral lands of the Duwamish people, a people that are still here, continuing to honor and bring light to their ancient heritage. You can learn more about the Duwamish tribe here. People who live and work on Duwamish land can pay rent here
In 2015, the Open Science Collaboration demonstrated that psychological science suffers from a problem of replicability. This has turned the field of psychology’s attention toward the benefits that open science can provide. Engaging in practices such as preregistering research studies, sharing data, open peer review, open access, and utilizing open education resources can assist with issues such as transparency and access and, in turn, could lead to greater social and cultural responsivity in research.
3.1 Learning Objectives
Learning objectives for this chapter include the following:
- Distinguish reproducibility from replicability.
- Identify mechanisms for increasing transparency in the research process.
- Describe potential benefits of preregistering research studies.
- List elements of data sharing that would improve reproducibility.
- Obtain an ORCID persistent digital identifier to help track your scholarly record.
3.2 Recommended Readings
The following served as critical references in the development of this chapter. We encourage you to review them.
- Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), 943–943. https://doi.org/10.1126/science.aac4716
- This was the study that called attention to the failures of replicability in psychological science. Given its historic and activating role in promoting open science in psychology, it is well worth reviewing the original document.
- Bosnjak, M., Fiebach, C. J., Mellor, D., Mueller, S., O’Connor, D. B., Oswald, F. L., & Sokol, R. I. (2021). A template for preregistration of quantitative research in psychology: Report of the joint psychological societies preregistration task force. American Psychologist, 77(4), 602-615. https://doi.org/10.1037/amp0000879
- This document creates a detailed argument for why preregistrations are necessary and it provides instructions for completing a preregistration for quantitative research.
3.3 Defining “Open Science”
Broadly defined, open science is the movement to make scientific results and processes accessible and reusable by all. As such it involves dimensions of both principles or culture and practices or technology. Although Steven Mann has been credited with coining the phrase “open science” in 1998 when he registered the domain names openscience.com and openscience.org, an earlier use the phrase was found in 1985, by Daryl E. Chubin in the article, “Open Science and Closed Science: Tradeoffs in a Democracy” (“Open Science,” 2022). Curiously, Chubin opened their article by referencing framework that is commonly used in introductory research methods textbooks (e.g., Krathwohl, 2009). That is: Merton’s norms of science.
Merton (1942) argued that communism (i.e., common owership of information), universalism (i.e., universal standards for claims of knowledge), disinterestedness (i.e., integrity in gathering and interpreting data), and organized skepticism (i.e., critical review of scientific reports) were necessary for producing objective knowledge. Separated by more than four decades, Merton and Chubin (1985) discussed the role of science in a democracy. While further elaboration of their arguments and conclusions extend beyond the goals of this chapter, the norms of communism and organized skepticism are foundational to the current understanding of open science.
Given the expansiveness of the notion of open science and its connectedness to other open initiatives (e.g., education, government, advocacy) numerous conceptual models may apply. Fecher and Friesike’s (2013) model organizes open science into five schools of thought. The public school of thought advocates that science needs to be accessible to a wider audience. Themes within this school include accessibility to the production of research (e.g., the citizen scientist) and consumability/accessibility to research results. The democratic school of thought is concerned with access to knowledge – especially when it is publicly funded. Subthemes within the democratic school emphasize that data be open for forseen and unforseen re-use and that there be open access to research publications. This is particularly true for government-funded research (e.g., the citizen should not be twice-taxed to access research findings). The pragmatic school of thought is concerned with efficiency, specifically utilizing online tools to allow for greater collaboration. The infrastructure school is specifically concerned with the technical infrastructure including software tools, applications, and computing networks. Finally, the measurement school of thought considers alternative standards to evaluating scientific impact of findings. For example this school challenges the “impact factor” (i.e., a measure of the average number of citations to an article in a journal), and suggests also counting (and publishing) elements like peer review and dissemination via emerging publishing formats.
In light of these foundational inputs, we will approach open science with a simultaneous review of its principles and practices in the three broad areas of (a) transparency of the research process; (b) access to research findings; and (c) tools for open science.
3.4 Transparency of the Research Process
Replicability (re-performing an experiment and collecting new data) and reproducibility (re-performing the same analysis with the same code using a different analyst; Patil et al. (2016)) are foundational to science. Yet investigations across scientific disciplines have demonstrated significant failures to both. Through its large scale, collaborative effort, the Reproducibility Project (Open Science Collaboration, 2015) revealed that psychological research often fails to replicate prior research. The collaborative selected 100 studies (97% of which had significant results) from three leading journals. Research teams committed to conducting high-powered, high-fidelity replications. Results were compared to the original studies with five metrics that assessed various dimensions of the analyses. Only 36% of the replications yielded statistically significant results; similarly a subjective analysis concurred that only 38% of the effects replicated the original results. Regarding effect sizes, 47% of the original effects were in the 95% confidence interval of the replication effect size. Finally, when original and replication results were combined (e.g., a meta-anlytic combination), 70% of the effects were statistically significant.
When studies are replicated, there are many reasons that the outcomes may differ. There may have been small-to-large differences in design and methods that are impactful enough to change the result. There could be a Type I (false positive) or Type II (false negative) error. Or there could be confirmation biases at any (or multiple) stage(s) of the design (Stevens, 2017). It is these biases that are particularly problematic. Bias at the individual-study-level occurs when the researcher reports, out of the many possible analyses, the one(s) that provide the most consistent or significant results (Hengartner, 2018; vant_veer_pre-registration_2016?).
Bias occurs at the journal level when reviewers and editors favor significant findings over non-significant ones (Stevens, 2017). Driessen et al. (2015) has suggested that there is a 25% reduction in the estimated effect of psychotherapy because 24% of all NIH-funded trials aimed at evaluating the efficacy of psychological treatment for major depressive disorder were never published. Dubbed the “file drawer problem” by Rosenthal (1979), this problem is compounded by researchers who are skeptical about the possibility of finding null results and the tendency of journals to not publish them.
Across scientific disciplines, transparency has been suggested as a potential remedy (Open Science Collaboration, 2015; vant_veer_pre-registration_2016?). Stated another way, for research to be truly reproducible, the entire process must be open to scrutiny (Stevens, 2017). We review three practices that would lead to more transparent, open, science. These include preregistration, data sharing, and open peer review.
3.4.1 Preregistration
Preregistration of a study involves specifying, in advance, the research questions, variables, research design and planned analyses (Stevens, 2017). There are a number of websites for such preregistrations; a common one for psychological scientists is the Open Science Framework. Researchers are encouraged to preregister studies that involve the testing of a priori hypotheses and models as well as studies that are intended as exploratory (Bosnjak et al., 2021). Although studies can be preregistered at any time, posting the preregistration in advance of seeing (or even more strictly, in advance of collecting) the data provides a mechanism for reviewers and consumers to evaluate the degree of consistency with which the research design and planned analyses were followed (Bosnjak et al., 2021; Haven et al., 2020).
Scholars have noted a number of benefits to preregistration. The most formal preregistration is a registered report (OSF, 2022). Registered reports first appeared in 2012 in the journals Cortex and Perspectives on Psychological Science. In 2013, Social Psychology adopted the practice. As of 2022, 300 journals across a wide variety of disciplines invite (or require) registered reports (Chambers & Tzavella, 2022).
Whereas traditional publication in peer-reviewed journals involves submitting the paper for review after the study is completed, the registered report is submitted and reviewed – twice. During the first stage, authors submit a detailed research proposal specifying research questions, hypotheses, methods,and planned analyses. Proposals that successfully pass through the review and revision(s) phase receive an “in principal acceptance”, which commits the journal to publishing the final paper regardless of whether the hypotheses are supported. Once the research is completed, the authors submit the completed manuscript. Any deviations or additions to the protocol must be clearly identified. Substantial departures from the proposed analyses may result in a rejection at this second review. Because the purpose is to prioritize sound scientific practice over significant outcomes, if the authors followed the a priori specified protocol, the article should be published (Chambers & Tzavella, 2022). By making the publication decision to accept-or-reject before the results are known, preregistered reports are an attempt to reduce biased research practices on the part of the researcher (e.g., HARKing [hypothesizing after the results are known], p-hacking, and selective reporting) and publication bias on the part of journals and reviewers (Chambers & Tzavella, 2022; vant_veer_pre-registration_2016?).
Not all journals invite registered reports. In these cases preregistration of studies is voluntary and researchers may wonder if preregistration is worth the time and effort. van’t Veer & Giner-Sorolla (vant_veer_pre-registration_2016?) have suggested that the detailed mapping involved in a preregistration may improve the overall quality of the study and that this, alone, is beneficial. Further, when reviewers and readers are aware that a study was preregistered, and that the plan was followed, the credibility of the findings may be enhanced.
Not surprisingly, there is resistance to preregistration. Moore (2016) described three primary concerns. First, preregistration constrains flexibility, exploration, and serendipitous discovery. Second, the preregistration invites additional scrutiny to the research process. Third, preregistration is more work.
Moore (2016) countered these concerns by suggesting that researchers are still free to engage in exploratory work. The preregistration itself is not limited to a priori hypotheses; researchers can specify their intentions to be purely exploratory or consider post-hoc analysis that follow the preregistered ones. Further, researchers can deviate from a preregistration; researchers are just expected to narrate how and why they did so. Regarding the concern of additional scrutiny, Moore argues that a study that followed a registered report or voluntary preregistration is likely to have greater credibility in the eyes of the reader or reviewer. Finally, Moore suggests that preregistration templates, specific to psychology, can streamline the process.
As an example for quantitative empirical research in psychology, the PRP-QUANT Template was designed by the Preregistration Task Force (2021). The PRP-QUANT has three primary sections: an introduction, method, and analysis plan. Each section includes multiple items that are accompanied with brief instructions (Bosnjak et al., 2021). For qualitative researchers, a Delphi method (Haven et al., 2020) provides a 13-item, pregistration template that is freely available at the Open Science Framework. Both templates align with the Journal Article Reporting Standards[JARS; American Psychological Association (2020)].
3.4.2 Data sharing
Data sharing (including the releasing of raw data, measures, codebooks, and analytic scripts for data cleaning and analysis; Alter & Gonzalez (2018)) is a second pathway to a more transparent science. Proponents of data sharing argue that it (a) increases scientific integrity through greater transparency and the increased probability of reproducibility (Martone et al., 2018), (b) optimizes the value of data and will accelerate scientific progress when data are exposed to secondary analysis or combined in meta-analyses (Ross et al., 2018), and (c) creates a structure for greater collaboration (Bezjak et al., 2018). Along with public and private entities, the U.S. Government contributes to data sharing when its agencies such as the Bureau of Labor Statistics, Department of Education, and Census Bureau offer data freely over the internet or restricted license.
The concept of data sharing is not new. In 1983, an article in the American Psychologist called for a mandate for data sharing (Ceci & Walker, 1983)). This was codified in the APA ethical principles as early as 1992. The current ethical principle (American Psychological Association, 2017) states that, “psychologists should freely share published data with peers requesting access for the purpose of verification or reanalysis.” The language in the ethics code implies a one-to-one (researcher-to-requester) relationship; the current calls from funders and proponents of open science are calling for broader access (Martone et al., 2018).
Whether in an institutional repository, with the journal, or in an open source, collaborative, internet platform such as the Open Science Framework, data sharing involves archiving the data (along with descriptions and codebooks) and the record of the analysis (Stevens, 2017). At the time of this writing, funding entities are increasingly requiring that data be made available for use by others (Ross et al., 2018) and journals are, similarly, requiring or encouraging such practices.
In contrast to the language in the APA ethical principles which suggest that sharing data should be “for the purpose of verification or reanalysis,” most proponents of open data encourage the researcher to license the data such that there are no restrictions on reuse or redistribution. Exceptions could be made to protect the identity of the human participants or special limitations or restrictions related to ethical concerns (Bezjak et al., 2018).
Not surprisingly, there is resistance to data sharing. Beyond ethical considerations, common concerns include (a) being critiqued for analytical errors, (b) being “scooped” (i.e., someone else analyzes and publishes first), and (c) expending significant effort that will be unrewarded (Martone et al., 2018). Regarding the first concern of “being critiqued,” a core principle of open science is that all research results are available for challenge through reexamination, reanalysis, reproducibility, and replication (Alter & Gonzalez, 2018). So, yes; being critiqued is a very possible consequence of sharing data.
In contrast, “being scooped” is a less likely consequence. In microbiology, the original data creators tended to publish two years after the data were made available whereas other authors tended to publish five or more years after the data were made available (Martone et al., 2018). “Being scooped” may be less of a concern if researchers limit data sharing to the variables used in their analysis and if it is shared at the time of publication. Countering the concern about “unrewarded effort,” there are initiatives underway that would recognize the contributions of those who share data and analytic code. For example, the Association for Psychological Science has adopted the use of Open Science Foundation badges in its journal, Psychological Science. Further, evaluation of this project has suggested that a display of badges is correlated with significant increases in data sharing (Martone et al., 2018). Several have argued that shared data, analytic code, and preregistrations should be assigned persistent identifiers (e.g., the DOI, digital object identifier) and be treated as scholarly products. That is, they should be listed on the contributor’s curriculum vita and counted in professional evaluations such as promotion and tenure applications (Alter & Gonzalez, 2018).
There are also significant ethical concerns related to the protections and rights of research participants. In the U.S. it has been standard practice to (a) inform participants of the restricted purpose for which their data will be used and (b) assure participants that all the information they provide during the research study will (to the extent permitted by the law) be kept confidential, only to be viewed by members of the research team. When non-anonymous data are collected, the informed consent may also indicate that data will be de-identified. These standard practices are counter to the notion that data be shared broadly with researchers-in-general and could be used for purposes other than that which was stated in the informed consent (Ross et al., 2018). Data collected where informed consent forms contained these traditional practices should not be shared. However, for new data collection, researchers who wish to share data should inform potential participants that consent extends beyond the present study; further, the nature and intent of future uses of the data are unknown (Ross et al., 2018). The participants should also be informed about the types of identifiable private information that will be retained and the types of researchers who may have access to that information (Alter & Gonzalez, 2018).
Conveying this information to potential participants in a manner that provides them fully informed consent while also encouraging their participation in the study can be tricky. Here is some language we use on our informed consents:
The information in the study records will be kept confidential. We do request your e-mail address so that we can send and link the quarterly surveys. Data will be stored securely and will be made available only to persons conducting the study unless you specifically give permission in writing to do otherwise. No reference will be made in oral or written reports that could link you to the study. Your de-identified data may be used in present and future (a) research collaborations, (b) scholarly/professional publications and presentations, and (c) in classroom teaching, projects, and demonstrations.
Consistent with both journal/guild expectations and the ethical principles of open science, a fully anonymous and non-identifiable version of the response (i.e., dataset) may be posted online (e.g., to the APA-endorsed “Open Science Framework” (www.osf.io) or to the journal, submitted with the research article). This data may be reanalyzed for purposes that we cannot anticipate. No data posted will contain any information that could identify participants in any way, either directly or indirectly. All data will be thoroughly inspected by the Principal Investigators prior to posting to confirm that no participant-provided responses could inadvertently identify or expose a participant.
Posting data (commonly referred to as “data sharing”) is necessary for reproducibility and replicability in science, allows peer reviewers and meta-analysts to check statistical assumptions, protects the field against data fraud, and is increasingly seen as an ethical obligation within psychological science.
Even with updated and IRB-approved informed consent forms, data sharing can be problematic, however. When samples or specific cell frequencies are small, certain combinations of information (e.g., tenure status, department, gender) could render a row of data identifiable. Another risk is when the data are used in unexpected ways that result in harm to the individuals or community. Ross et al. (2018) shared the story of blood samples being collected from an Indigenous group for what was believed to be a diabetes study. Later it was discovered that the data had been shared with other researchers to study topics that brought social and psychological harms to the tribe as a whole. Further, data that is shared in a public repository will likely be available globally where laws and ethical conventions for using research data may vary.
As chapter authors, we are generally proponents of open science. However, we urge researchers to give thoughtful consideration of plans for data sharing at the beginning of the project, to imagine intended and unintended consequences, and to seek IRB consultation and review. Further, data takes many different forms (e.g., qualitative, clinical interviews, survey data, geographical identifiers) and protection of the research participants will require different types of considerations before deciding if and how it can be shared (Ross et al., 2018).
3.4.3 Open peer review
Open peer review is another avenue for increased transparency in the scientific process. Although the term is interpreted differently, there are two primary mechanisms: open identities and open reports (Ross-Hellauer, 2017). With open identities, neither the reviewers nor authors are anonymized. With open reports, the review reports are published alongside the relevant article (Ross-Hellauer, 2017). As journals and other outlets experiment with open peer review, there may be variations of one or more of these mechanisms. Some have argued that published open reports – which could be cited by others and counted as a scholarly product for evaluative activities such as tenure and promotion – might incentivize scholars to accept peer review assignments (which are usually completed with no compensation) and invest the time and energy necessary to provide a constructive critique of the work and formative feedback to the researchers (Bezjak et al., 2018).
At the time of this writing, there is evidence of gradual movement toward open reviews. Some journals will now ask if the peer review can be transferred to another journal (if the manuscript is rejected) and if the reviewer’s name can be transferred with it. While this is neither open identity nor open reports, it is a small step in the direction of sharing the work of peer review. Additionally, many journals are now inviting peer reviewers to register with Publons, a commercial organization that provides a mechanism for collecting and summarizing scholarly impact as a peer reviewer.
In creating this textbook, we engaged in a form of open review. It was important to us that each chapter be peer reviewed, yet we desired the peer review process to be non-anonymous, constructive, and formative (as opposed to anonymized, critical, and summative/gate-keeping). Along with a common rubric, drafts and revisions of each chapter were placed in a Google Docs folder. Two reviewers and the authors could access these materials at any time. Peer reviewers were asked to comment on each element of the rubric and leave suggested edits and comments/questions directly on the chapter draft.
3.5 Access to the Scientific Literature
Another aspect of open science concerns access to the literature. There are multiple forms of open access publishing (Bezjak et al., 2018; Shah, 2017). Self-archiving is the process of placing a published version of an author’s article into institutional repositories or websites. Sometimes self-archived articles have an embargo period (months to years) that must elapse before the article becomes open access. Open access publishing is immediately, freely available, upon its publication. This level of access usually involves an article processing charge (APC), a one-time payment by the author. A third type is the hybrid article. This happens when a pay-walled journal offers individual open access articles. This usually requires the author to pay a fee that is higher than the APC associated with open access. If an article is accepted into a hybrid model, it means that in a single journal issue, readers will find both open access and paywalled articles. Traditional journals are often motivated to use this model because it fulfills funder policies such as requiring immediate public access to research.
Not surprisingly, there are pros and cons to open access literature. Regarding pros, with no subscription fees, fees for individual articles, nor requirements to be associated with an institution who has access to the book or journal (Bezjak et al., 2018; Shah, 2017) open access materials are free to everyone, including in international contexts (e.g., low-middle income countries) where barriers to scientific literature may be greater. This benefits the potential readership as well as the author – in that there may be greater dissemination [and, in turn, citations; Hagger (2022)]. Another positive aspect of open access journals is an expedited submission-to-publication timeline (Shah, 2017).
Alternatively, while the rapid turnaround for open access articles may be viewed as a positive attribute of open access journals, this is not always the case. Sometimes, legitimate open-access publishers feel forced into this accelerated submission-to-publication timeline in order to compete. As a result, they may weaken their peer-review process to meet this deadline (Beall, 2012). In addition, not all open access journals are well intentioned or legitimate. Predatory publishers cleverly spam researchers with calls-for-papers and fail to mention required authors fees that range from 1200 USD to 1800 USD. An author who learns of the APC after signing the contract (which generally includes surrendering the copyright) has lost the right to withdraw and is faced with paying the fee and losing the possibility to publish elsewhere – therefore, essentially losing their work.
Another common concern of open access research is that while these journals and articles are free to readers, this doesn’t mean that readers are always able to locate these publication. A crucial piece in making articles accessible to readers is getting the journal indexed in as many relevant databases as possible (Fortney & Murphy, 2016). APA PSYCH Info an example of a commonly used database in psychology.
A well-indexed article is more likely to be discovered and read regardless of whether or not the reader is familiar with the journal itself. Regrettably, for those hoping to join the open science movement, getting journals indexed can be incredibly difficult. In some cases, it will take multiple attempts over several years before a journal is accepted into an index or database (Fortney & Murphy, 2016; Shah, 2017). While indexing journals is historically challenging, PubMed and Wellcome Trust are among some of the databases working with APA to increase accessibility of open access psychology research (Martone et al., 2018).
Finally, we think it is critical to raise equity concerns about the APC. Much of academia involves pressure to “publish or perish.” Open access affords those who can afford APC a wider choice of outlets for dissemination. Those without similar financial resources may be unable to logistically access these journals as an author and may need to rely on institutional support, grant support or the backlog of traditional journals. Thus, the APC associated with open access journals may further widen the equity gaps in tenure and promotion, making it easier for those with greater financial resources to accrue publications more quickly.
3.6 Tools for an Open Science
Tools that support open science abound and are constantly evolving and emerging. We review six types of tools including: statistical software, reference management systems, persistent identifiers, data repositories, collaborative platforms, and open educational resources (OERs). Behind many of these tools is the notion of open source. Open source tools are freely shared and the code (or platform) that powers them can be modified and redistributed.
3.6.1 Statistical Software
The R statistical software environment is an open source tool that includes features for commenting on code and enabling reproducible data analysis (Alter & Gonzalez, 2018; Bezjak et al., 2018). Using R requires statistical training as well as fluency with the integration of base R, R Studio, and R packages. Because each of these elements are continuously updated, the R user must always adapt to changes in the underlying sourcecode that powers the analyses.
The birth of R might be associated with the 1997 launch of the CRAN (Comprehensive R Archive Network). The CRAN hosts R’s executable files, source code, and packages contributed by the users (“R (Programming Language),” 2022). Mirrors are the network of ftp (file-transfer-protocol) and web servers around the world that store identical (hence, “mirror”) materials. There were only three mirrors in 1997; currently there are more than 100. As can be seen on the global list, they represent a variety of organizational types ranging from higher education to corporations to governments and nonprofits. Users are encouraged to select the CRAN that is geographically closest.
R Studio is a public benefit corporation, that is, a statutory or government owned corporation whose mission is to provide free or subsidized services to the public (“RStudio,” 2022). The organization provides open-source (zero-cost) and commercial software that is intended to serve in a “virtuous cycle” for mutual improvement (“RStudio,” 2022).
Some are wary of open source statistical software, voicing concerns of trustworthiness. It may be reassuring to learn that the most common way to install an R package is through the CRAN. Packages available on the CRAN must adhere to its repository policy and are vetted prior to posting. Further, those engaged in psychological science who may scour the internet for “how-to” tutorials and blogs will quickly learn that there are a number of reliable, commonly used R packages to conduct the analyses that are frequently discussed and critiqued. Further, resources such as the peer-reviewed Journal of Statistical Software provide in-depth coverage of many R packages.
Because writing R script can be daunting, GUI (graphical user interface, point-and-click) alternatives are emerging that are open-sourced and, perhaps, easier to use. The R package, shiny, allows developers to build interactive tools known as “shiny apps.” These apps tend to perform limited functions. An example is Shoemann et al.’s (2017) Monte Carlo power analysis for indirect effects (https://schoemanna.shinyapps.io/mc_power_med/).
More comprehensive software tools are also built with R code. For example, the program jamovi (project, 2021) was designed as an alternative to fee-for-use programs (e.g., SPSS) for the social sciences. Although the user points-and-clicks, a syntax mode allows the production of R syntax for inputting directly to R or for retrieval as a completely reproducible project. Although three individuals are credited as co-founding jamovi and their website lists additional team members (project, 2021) there is not information about its organizational status. The jamovi website seeks financial contributions and volunteers for advocacy, content creation, and module development.
JASP (“just another statistics program,” Team, 2022) is another GUI that is commonly used in psychological science. Sponsored by the University of Amsterdam, JASP’s features include both frequentist and Bayesian analyses. Further, JASP produces APA-formatted tables that can be copy-pasted into word processing documents. Although it is a future goal of the developers, at this time, it is not easy to retrieve R code from JASP for reproducible archiving. Users of open-source software should include both text and reference list citations. Citing the software (a) provides a complete description of the method (contributing to reproducibility), (b) documents the usage and development of that software in the developer’s field, and (c) credits (in the form of a scholarly citation) the developer (Smith et al., 2016).
3.6.2 Reference management software
Zotero (Digital Scholarship, 2022) is one example of an open source reference management program operated by the non-profit group, Corporation for Digital Scholarship. Zotero stores, manages, and cites bibliographic references. Zotero allows (a) cloud and locally held storage, (b) collaboration, (c) and full integration with word processing documents and R markdown files. Zotero’s open source nature makes it immediately responsive to change; merely three months after the introduction of the 7th edition of the APA style manual, Zotero upgraded the default style to match. Although a non-profit, Zotero does operate on a freemium model where basic services are free, but cloud storage (allowing synchronization across devices and collaborators) requires a subscription fee.
Another popular tool that offers similar functionality is Mendeley. In 2007, Mendeley was founded in the U.K. by three doctoral students from Germany. In 2018 it was purchased by the academic publisher, Elsevier (“Mendeley,” 2022).
3.6.3 Persistent identifiers
You may have noticed that most items in our reference lists have DOI (digital object identifiers) numbers. The DOI is an example of a persistent identifier (PID). PIDs are long-lasting digital references to objects, people, or organizations that serve to provide a reliable link from citations to the publication (or its source). There are two parts to PIDs. First, is the identifying alphanumeric string, itself. Second, is the organization or agency that commits to providing an infrastructure to ensure that a URL will map to the correct location of the object.
Owing to name, organizational, and geographic changes, researchers can also become disconnected from their work. The ORCID (Open Researcher and Contributor ID) was created as an independent nonprofit organization to provide a unique, persistent identifier to researchers. The sponsoring organization has also created an infrastructure to ensure the reliability of these connections. When scholarly products include the authors’ ORCID, there can be a permanent and clear record of research activities. Many journals request ORCID for authors and co-authors at the time of submission; ORCIDs are also commonly requested of peer reviewers.
3.6.4 Data repositories
Data repositories collect, maintain, and disseminate data over time (Alter & Gonzalez, 2018). This is accomplished, in part, by providing a public facing citation and assigning persistent identifiers (e.g., DOI, digital object identifiers). Data repositories exist across a number of institutions and platforms. For example colleges and universities may archive theses and dissertations. Other data repositories are more disciplinary-specific.
3.6.5 Collaborative platforms
Collaborative platforms are online services that provide a virtual environment where multiple people can connect and work on the same task (Bezjak et al., 2018). If you have used Google Docs, Dropbox, or the Microsoft packages such as OneDrive or Sharepoint, you have used a collaborative platform. While these are terrific tools for word processing, spreadsheets, and slide presentations, they may less helpful in co-authoring statistical code.
GitHub (GitHub, 2022a) is an example of a cloud-based, collaborative platform, that specializes in hosting code (e.g., R code) where multiple people can contribute, track, and control changes to the code. GitHub is also host to numerous open source projects. For projects that are openly licensed, others can “fork” (i.e., copy the project for yourself and make changes to it without altering the original) the project. Additionally, so long as the secondary user credits the author/developer, they can use, revise, remix, and further distribute its contents. Initially a start-up business, Microsoft purchased GitHub in 2018 “GitHub” (2022b). GitHub’s basic features are free for individuals and organizations; there are more advanced tools for a fee. Our OER is produced and hosted on GitHub and GitHub pages for zero cost.
The Open Science Framework (OSF, 2022) was created by the not-for-profit Center for Open Science and is a collaborative infrastructure to support the entire research cycle. Across multiple disciplines, including psychology, researchers can preregister studies and use the same project for sharing data and analytic code (as well as literature, IRB materials, experimental materials, presentations, preprints). OSF also offers the capacity to connect with other systems such as GitHub, Google Docs, and Dropbox. The GitHub and OSF are only two examples. Other commonly used collaborative platforms include Zenodo and Figshare (Martone et al., 2018).
3.6.6 Open education resources
Open education resources (OERs) include any tools, materials, or techniques that are used to support access to knowledge (Bezjak et al., 2018). While not specific to open science, there is considerable overlap in the values and resources that contribute to them and emerge from them. This very textbook was created using the open source software R and R Studio. The primary packages used to format the content are R Markdown and Bookdown. The book is hosted in GitHub and is rendered to the internet via GitHub Pages. As described in the preface, the OER holds a CC BY-SA 4.0 license allowing the user to share (copy and redistribute the material in any medium or format) and adapt (remix, transform, and build upon the material for any purpose). The license requires that proper attribution is made (e.g., appropriate credit, a link to the license, and indication of changes were made) and that your redistribution must use the same CC BY-SA 4.0 license.
3.6.7 Bearing the Costs of “Open”
Our attention to “who owns” open tools may be a surprising inclusion in a chapter on open science. We were intentional in providing this information because the who of ownership and the type of business model may have bearing on the trustworthiness with which researchers’ tools, materials, and data may be protected and maintained. Further, as the popularity of open tools increase (and decrease) there is the hope that tools will improve, the risk that the tool will be ignored or deprecated, and the possibility that use fees will be added or increased. Our only recommendation is that potential users take some time to become familiar with open tools before committing to use them.
3.7 Summary, Conclusions, and/or Recommendations
Researchers have demonstrated that psychological science is threatened by problems of reproducibility and replicability (Open Science Collaboration, 2015). Through preregistration, data sharing, and open peer review, proponents of open science have identified pathways to increase the transparency of the scientific process. Ensuring open access to research findings remains problematic. Many findings are behind pay walls (restricting their access) and open access journals often charge fees that are prohibitive for new scholars and those groups who experience marginalization. More work will be needed to solve this access. In the meantime, tools to facilitate open science are abundant, emerging, and constantly improving. We encourage emerging researchers to participate in open science. Further, as it continues to develop, we encourage all to be vigilant so that the new practices are not co-opted in ways that maintain historic and inequitable power structures and privileges.
3.8 Suggestions for Practice, Further Learning, and/or Conversation
- Obtain an ORCID iD for yourself
- Find a published study that had been preregistered. Trace it to its preregistered location. Is there also access to data, analytic code, and other materials? How easy was it to locate? How easy would it be to reproduce the study?
- Download one of the preregistration templates (either quantitative or qualitative) and preregister a study of your own.