EPA Administrator Scott Pruitt was flanked by Sen. Mike Rounds (R-SD), left, and Rep. Lamar Smith (R-TX) at an April 24 press conference on his proposed rule.
(Image credit – EPA)
EPA Cites ‘Replication Crisis’ in Justifying Open Science Proposal
MAY 04, 2018
EPA has issued a proposed rule that would require data and models underlying certain scientific studies to be made publicly available before the agency could use the studies in developing regulations. Among its justifications, the agency is arguing the proposal is consistent with the research community’s recent efforts to address concerns about the irreproducibility of swaths of science.
Director of Science Policy News,
AIP
Partisan split on whether Pruitt has honest intentions
“This is a great day. It’s a banner day,” said EPA Administrator Scott Pruitt at the outset of an April 24 press conference announcing a proposed rule titled “Strengthening Transparency in Regulatory Science.” Pruitt explained that it “represents the third leg of the stool” of process-oriented changes he has implemented, with the other two being his directives barring EPA grant recipients from its scientific advisory panels and ending the agency’s “sue and settle” practices.
He also made clear that he intends for it to be the sturdiest leg:
This is not a policy. This is not a memo. This is a proposed rule. And the reason that’s important is because this is not just something that we’re proposing that may last for two months or two years. It is a codification of an approach that says, as we do our business at the agency, the science that we use is going to be transparent, it’s going to be reproducible, it’s going to be able to be analyzed by those in the marketplace, and those that watch what we do can make informed decisions about whether we’ve drawn the proper conclusions or not.
“Today’s announcement shows that we have an administrator that is committed to scientific integrity and making our government more accountable to the American people,” Smith said. “For too long, the EPA has issued rules and regulations based on data that has been withheld from the American people. Today, Administrator Pruitt rightfully is changing business as usual and putting a stop to hidden agendas.”
Meanwhile, congressional Democrats argue that Pruitt and the proponents of the “HONEST Act” are the ones pursuing a hidden agenda. Slamming the proposed rule, House Science Committee Ranking Member Eddie Bernice Johnson (D-TX) remarked ,
The push to allow EPA to only consider publicly available data in its rulemaking is an insidious plan designed from the outset to prevent the EPA from using the best available science to meet its obligations under the law. … Republicans weren’t able to get their ‘secret science’ bills signed into law, but now they have Administrator Pruitt to do the bidding of industry at the EPA. This rule isn’t about ‘scientific transparency.’ It’s about undermining public health and the environment.
A number of scientific societies — such as the American Association for the Advancement of Science , the American Chemical Society , the Ecological Society of America , and the American Geophysical Union — have already made initial comments, expressing concern that the proposal could unduly limit EPA’s ability to use science. For example, AGU’s statement points to how many scientific societies opposed the “HONEST Act” on the grounds it may not adequately protect study participants’ confidential health information and would be very expensive to implement. The Congressional Budget Office estimated that a prior incarnation of the bill would cost about $250 million per year over several years to implement.
Proponents of the rule maintain there are proven methods for redacting such confidential information and that implementation costs are overstated. The proposal also is narrower than the “HONEST Act,” only applying to pollutant exposure data and models underlying “pivotal” regulatory science. It also would permit the EPA administrator to grant exceptions on a case-by-case basis if it is impractical to comply. Critics maintain this exception would allow the administrator to selectively exempt industry studies from scrutiny.
Top reproducibility scholar and scientific journals weigh in
Another focal point of debate on the proposed rule is whether the open science requirements it imposes represent a rational and good faith extension of the scientific community’s own recent efforts to increase transparency. EPA prominently points to this trend in a press release announcing the proposal, saying it is “in line with the scientific community’s moves toward increased data sharing to address the ‘replication crisis’ — a growing recognition that a significant proportion of published research may not be reproducible.”
Furthermore, the proposed rule itself says it is “informed by the policies recently adopted by some major scientific journals, spurred in some part by the ‘replication crisis.’” A footnote at the end of the sentence links to several articles in scientific journals. Among them is a 2005 paper titled “Why most published research findings are false.” Penned by John Ioannidis, a medicine professor, the article has been credited with drawing widespread attention to reproducibility issues in science.
Also linked is a 2017 “Manifesto for reproducible science,” to which Ioannidis is a signatory. The manifesto references a 2016 survey by the journal Nature to which 52 percent of the 1,576 researchers queried across a variety of fields said there is a “significant [reproducibility] crisis” and 38 percent said there is a “slight crisis.”
“Whether ‘crisis’ is the appropriate term to describe the current state or trajectory of science is debatable,” the manifesto states, “but accumulated evidence indicates that there is substantial room for improvement with regard to research practices to maximize the efficiency of the research community’s use of the public’s financial investment in research.”
The research community’s focus on such issues has not abated. For instance, the National Academies is currently conducting a congressionally mandated study of reproducibility and replicability issues across all fields of science.
Ioannidis laid out his views on EPA’s proposal in an op-ed published yesterday titled “All science should inform policy and regulation,” saying that preventing consideration of science that does not meet strict transparency requirements would be a “nightmare.” While noting that the government can play an instrumental role in promoting open science practices through funding and regulatory mandates, he warns against it adopting a dogmatic insistence on transparency requirements.
The editors of several top journals, including Science and Nature, have also commented on EPA’s proposal. In a joint statement , they note that transparency policies recently adoped by many journals recognize cases in which data should not be openly shared. They conclude,
It does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making. Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes.
