Post by icemandios on Sept 22, 2022 15:14:18 GMT
FDA Authorized Problematic COVID-19 Tests Due to Agency's EUA Approach, HHS OIG Finds
Sep 22, 2022 | Madeleine Johnson
Save for later
NEW YORK – Combining a survey of diagnostics developers with a deep review of the US Food and Drug Administration's pandemic response, the US Department of Health and Human Services' Office of the Inspector General has recommended that US Food and Drug Administration adjust its course for future health emergencies.
In a 51-page report published on Wednesday, the OIG detailed the policy adaptations that FDA made as it developed it's COVID pandemic response oftentimes on the fly and recommended six tweaks be implemented before the next pandemic.
The OIG at HHS is the largest office of the inspector general in the US federal government and is tasked to evaluate the efficiency and effectiveness of more than 100 HHS programs. As part of its review of FDA's Emergency Use Authorization process, the OIG team conducted analyses of FDA data and documents, interviewed FDA staff, and also surveyed diagnostics developers.
Specifically, Elizabeth Sandefer, a social science research analyst and OIG's team leader for the study, said in an email that the survey was sent to every developer that contacted FDA about tests for COVID-19. This included every developer that reached out to FDA with questions about the EUA process as well as all that submitted a formal EUA request.
"The intention of this evaluation is to provide stakeholders — including FDA, Congress, and the public — with timely, useful, and reliable information on this significant issue," Sandefer said.
In total, 237 developers provided feedback. The OIG team also interviewed test developers and representatives from a professional lab association for context about the developers' experiences, according to the report.
Diagnostics were necessary because without sufficient testing, efforts to control a virus are significantly limited, the OIG report noted. Nevertheless, what began with a single case of "2019 novel coronavirus" in Washington state in January of 2020 ultimately resulted in more that 83 million SARS-CoV-2 cases and 1 million deaths in the US.
And although the FDA pivoted to use its EUA capabilities to help resolve the initial rollout failures of the US Centers for Disease Control and Prevention's RT-qPCR assay, FDA's need to engage with private developers "revealed vulnerabilities in the federal approach to early emergency testing," the OIG report concluded.
Overall, the report found that some of FDA's approaches were effective, while others were less so.
The generation of templates for EUA submission, and the creation of forums for communication between FDA and developers — such as the weekly town hall calls, guidance documents, a dedicated email inbox, and FAQ webpages — were counted in the win column by OIG.
But OIG also noted in the report that diagnostic developers were sometimes "confused and frustrated" by FDA's evolving policies. Some spent a long time in the review queue only to find out that the agency decided to stop evaluating submissions, for example.
Most damning, the report suggested that FDA's initial take-all-comers approach diluted the agency's focus and increased its workload, and an emphasis on rapidly increasing the total number of tests may have led to authorization of substandard assays.
Indeed, "FDA’s decision to accept all EUA requests resulted in a record number of submissions — often low-quality and from developers lacking experience with FDA's processes," the report noted.
The report highlighted that between January and May of 2020, a total of 731 tests were submitted to FDA. The agency reviewed and authorized 117 of these tests, and granted EUA to 101 more over the subsequent 12 months.
But FDA staff told OIG that most of the submissions were "low-quality" and lacking essential data, particularly attributing this to smaller device makers and labs that had not previously had experience working with FDA.
Supporting this, the OIG report found that only 28 percent of the survey responders had previously gone through any FDA approval or clearance process. And experience helped, as 91 percent of those with past experience received EUA for their first submitted COVID-19 test, according to the report, while only 40 percent of those with no prior experience did.
On the other hand, developers told the OIG team that FDA's evolving policies were sometimes not clearly communicated, and that templates sometimes changed after submissions were sent. Also, FDA's prioritization criteria were murky and poorly communicated, with some developers reporting getting bumped to the back of the line and then receiving a "decline to review" notice after months of patiently waiting.
And still, "The FDA eased EUA requirements to expedite getting COVID-19 tests to market ... [and] its decisions to reduce or forgo EUA requirements allowed problematic tests to be distributed," HHS' OIG team wrote.
The report specified that FDA made "calculated decisions to increase availability of COVID-19 testing, but these decisions often came at a potential cost to test quality," basing authorization on lower levels of evidence when developers struggled to access clinical samples.
This led to a feedback cycle, whereby "FDA's policies allowed diagnostic and serology tests to get on the market quickly; however, that resulted in some problematic tests on the market, requiring further action by FDA," the OIG authors also wrote.
In addition, the OIG assessment revealed FDA needs to have "clear, direct communication with the lab community during an emergency."
It also found that developers are likely to require further FDA guidance on how to validate tests during shortages of clinical samples, and that, in general, inexperienced assay developers require more support from FDA during the EUA process. The report further found that FDA needs contingency plans to handle any increased workloads in a future emergency.
Importantly, "FDA addressed early testing challenges using EUA flexibilities, but this experience underscores the need for a national testing strategy," the OIG team wrote in the report.
For the next pandemic
The OIG report recommended six steps FDA should take in anticipation of future pandemics.
OIG's Sandefer said the recommendations include ones that FDA can implement internally, but also ones that "emphasize the need for FDA to work with federal partners to implement lessons learned from COVID-19 that go beyond the EUA process," she said.
The recommendations included assessing and revising the guidance for EUA submissions, and developing a suite of EUA templates for future emergencies as well as a suite of resources to help guide test developers through the EUA process.
The OIG also suggested FDA revise its device-tracking platform to facilitate EUA submission and monitoring, and that it establish formal communication channels between FDA and the lab community to be engaged during emergencies.
Finally, OIG guided FDA to work with federal partners "to implement lessons learned about a national testing strategy that go beyond the EUA process"
The OIG report emphasized that FDA was consulted and concurred with all of the recommendations.
HHS and OIG will now follow up with FDA within six months to get a "Final Management Decision," which outlines how the agency plans to execute the recommendations, OIG's Sandefer said.
And, HHS and OIG intend to continue to follow up with FDA annually about the status of the recommendations.
In a separate report issued earlier this month, OIG also evaluated FDA's participation in the Tri-Agency Task Force, a working group which assigns distinct roles to FDA, CDC, and the Centers for Medicare and Medicaid Services in any health crisis requiring labs to implement tests
That evaluation was motivated by the fact that the pandemic was the first emergency since the 2019 establishment of the task force. It concluded that "multiple setbacks plagued the early testing rollout for COVID-19," but that this "failure" was offset by the fact that FDA's engagement with the task force showed "the value of interagency coordination and collaboration to facilitate emergency test implementation in labs."
Sep 22, 2022 | Madeleine Johnson
Save for later
NEW YORK – Combining a survey of diagnostics developers with a deep review of the US Food and Drug Administration's pandemic response, the US Department of Health and Human Services' Office of the Inspector General has recommended that US Food and Drug Administration adjust its course for future health emergencies.
In a 51-page report published on Wednesday, the OIG detailed the policy adaptations that FDA made as it developed it's COVID pandemic response oftentimes on the fly and recommended six tweaks be implemented before the next pandemic.
The OIG at HHS is the largest office of the inspector general in the US federal government and is tasked to evaluate the efficiency and effectiveness of more than 100 HHS programs. As part of its review of FDA's Emergency Use Authorization process, the OIG team conducted analyses of FDA data and documents, interviewed FDA staff, and also surveyed diagnostics developers.
Specifically, Elizabeth Sandefer, a social science research analyst and OIG's team leader for the study, said in an email that the survey was sent to every developer that contacted FDA about tests for COVID-19. This included every developer that reached out to FDA with questions about the EUA process as well as all that submitted a formal EUA request.
"The intention of this evaluation is to provide stakeholders — including FDA, Congress, and the public — with timely, useful, and reliable information on this significant issue," Sandefer said.
In total, 237 developers provided feedback. The OIG team also interviewed test developers and representatives from a professional lab association for context about the developers' experiences, according to the report.
Diagnostics were necessary because without sufficient testing, efforts to control a virus are significantly limited, the OIG report noted. Nevertheless, what began with a single case of "2019 novel coronavirus" in Washington state in January of 2020 ultimately resulted in more that 83 million SARS-CoV-2 cases and 1 million deaths in the US.
And although the FDA pivoted to use its EUA capabilities to help resolve the initial rollout failures of the US Centers for Disease Control and Prevention's RT-qPCR assay, FDA's need to engage with private developers "revealed vulnerabilities in the federal approach to early emergency testing," the OIG report concluded.
Overall, the report found that some of FDA's approaches were effective, while others were less so.
The generation of templates for EUA submission, and the creation of forums for communication between FDA and developers — such as the weekly town hall calls, guidance documents, a dedicated email inbox, and FAQ webpages — were counted in the win column by OIG.
But OIG also noted in the report that diagnostic developers were sometimes "confused and frustrated" by FDA's evolving policies. Some spent a long time in the review queue only to find out that the agency decided to stop evaluating submissions, for example.
Most damning, the report suggested that FDA's initial take-all-comers approach diluted the agency's focus and increased its workload, and an emphasis on rapidly increasing the total number of tests may have led to authorization of substandard assays.
Indeed, "FDA’s decision to accept all EUA requests resulted in a record number of submissions — often low-quality and from developers lacking experience with FDA's processes," the report noted.
The report highlighted that between January and May of 2020, a total of 731 tests were submitted to FDA. The agency reviewed and authorized 117 of these tests, and granted EUA to 101 more over the subsequent 12 months.
But FDA staff told OIG that most of the submissions were "low-quality" and lacking essential data, particularly attributing this to smaller device makers and labs that had not previously had experience working with FDA.
Supporting this, the OIG report found that only 28 percent of the survey responders had previously gone through any FDA approval or clearance process. And experience helped, as 91 percent of those with past experience received EUA for their first submitted COVID-19 test, according to the report, while only 40 percent of those with no prior experience did.
On the other hand, developers told the OIG team that FDA's evolving policies were sometimes not clearly communicated, and that templates sometimes changed after submissions were sent. Also, FDA's prioritization criteria were murky and poorly communicated, with some developers reporting getting bumped to the back of the line and then receiving a "decline to review" notice after months of patiently waiting.
And still, "The FDA eased EUA requirements to expedite getting COVID-19 tests to market ... [and] its decisions to reduce or forgo EUA requirements allowed problematic tests to be distributed," HHS' OIG team wrote.
The report specified that FDA made "calculated decisions to increase availability of COVID-19 testing, but these decisions often came at a potential cost to test quality," basing authorization on lower levels of evidence when developers struggled to access clinical samples.
This led to a feedback cycle, whereby "FDA's policies allowed diagnostic and serology tests to get on the market quickly; however, that resulted in some problematic tests on the market, requiring further action by FDA," the OIG authors also wrote.
In addition, the OIG assessment revealed FDA needs to have "clear, direct communication with the lab community during an emergency."
It also found that developers are likely to require further FDA guidance on how to validate tests during shortages of clinical samples, and that, in general, inexperienced assay developers require more support from FDA during the EUA process. The report further found that FDA needs contingency plans to handle any increased workloads in a future emergency.
Importantly, "FDA addressed early testing challenges using EUA flexibilities, but this experience underscores the need for a national testing strategy," the OIG team wrote in the report.
For the next pandemic
The OIG report recommended six steps FDA should take in anticipation of future pandemics.
OIG's Sandefer said the recommendations include ones that FDA can implement internally, but also ones that "emphasize the need for FDA to work with federal partners to implement lessons learned from COVID-19 that go beyond the EUA process," she said.
The recommendations included assessing and revising the guidance for EUA submissions, and developing a suite of EUA templates for future emergencies as well as a suite of resources to help guide test developers through the EUA process.
The OIG also suggested FDA revise its device-tracking platform to facilitate EUA submission and monitoring, and that it establish formal communication channels between FDA and the lab community to be engaged during emergencies.
Finally, OIG guided FDA to work with federal partners "to implement lessons learned about a national testing strategy that go beyond the EUA process"
The OIG report emphasized that FDA was consulted and concurred with all of the recommendations.
HHS and OIG will now follow up with FDA within six months to get a "Final Management Decision," which outlines how the agency plans to execute the recommendations, OIG's Sandefer said.
And, HHS and OIG intend to continue to follow up with FDA annually about the status of the recommendations.
In a separate report issued earlier this month, OIG also evaluated FDA's participation in the Tri-Agency Task Force, a working group which assigns distinct roles to FDA, CDC, and the Centers for Medicare and Medicaid Services in any health crisis requiring labs to implement tests
That evaluation was motivated by the fact that the pandemic was the first emergency since the 2019 establishment of the task force. It concluded that "multiple setbacks plagued the early testing rollout for COVID-19," but that this "failure" was offset by the fact that FDA's engagement with the task force showed "the value of interagency coordination and collaboration to facilitate emergency test implementation in labs."