Part of what the medical device industry bought with the substantial increase in FDA user fees to which it agreed in the FDA Safety and Innovation Act of 2012 was an independent review and evaluation of FDA’s primary premarket device review programs, the 510(k) and PMA programs. The review was scheduled to be completed in two phases, a first phase to result in priority recommendations and a second phase to result in the complete evaluation and FDA’s plan for implementing the priority recommendations. Phase one was recently completed when FDA published a summary evaluation by the consulting firm Booz | Allen | Hamilton (BAH).  

After conducting interviews with FDA personnel and industry stakeholders, and reviewing 510(k) and PMA submissions from FY2011-12, among other things, BAH arrived at four priority recommendations, quoted verbatim below:

• (1) Develop criteria and establish mechanisms to improve consistency in decision making throughout the review process

• (2) Provide mandatory full staff training for the three primary IT systems that support MDUFA III reviews

• (3) Identify metrics and incorporate methods to better assess review process training satisfaction, learning, and staff behavior changes

• (4) Adopt a holistic, multi-pronged approach to address five quality component areas to standardize process lifecycle management activities and improve consistency of reviews

With respect to the first recommendation above, BAH observed “inconsistent decision-making throughout the various stages of the review process, in particular a lack of transparency in thresholds or requirements used to trigger additional information (AI) requests.” Industry stakeholders also reported that FDA reviewers sometimes used either outdated guidance documents or new standards that had not been released at the time of a submission to evaluate a 510(k) or PMA submission. As examples of the type of criteria or mechanisms that could expedite reviews and make them more consistent, BAH suggested having lead reviewers identify early the standards that would be applied to a particular submission and whether the submission would be subject to any new standards released during the review. BAH also suggested that FDA develop a standard AI request checklist to alert applicants to the types of deficiencies they might receive.

For the second recommendation, BAH found that FDA’s reviewers were offered training prior to the effective date of the new user fee legislation, but that awareness and retention of knowledge of changes to specific review processes varied. Thus, BAH recommended that all of FDA’s reviewers should complete the appropriate training courses.

The third recommendation appears to be closely-related to the second. BAH analyzed FDA’s four training programs and “uncovered gaps in FDA’s ability to objectively assess the impact of learning and the extent to which participants’ behavior changed as a result of the training.” In perhaps understated fashion, BAH added that “[t]raining administrators need to understand whether training courses are meeting set objectives and if not, what aspects need to be modified to accomplish that goal.” Thus, it was suggested that “[m]ore timely, comprehensive, and detailed surveys could provide FDA with information to tailor and refine their training programs to be more effective” and that “post-training surveys and/or interviews regarding participants’ experience with integrating the knowledge learned can serve as a valuable resource in validating training or identifying a need for change.”

For the fourth recommendation, BAH relied on standard quality system components to make recommendations in five specific areas:

• Senior management responsibility – The evaluation notes that FDA’s senior management monitors the implementation of new processes and reviews new issues as they arise, but does not formally document the results to promote accountability and facilitate follow-up on the issues raised

• Resource management – This incorporated recommendations two and three above

• Document management – While FDA has mechanisms for quality document control and management, they are applied inconsistently; again, training and follow-up is needed

• Corrective and Preventive Action and Continuous Process Improvement – BAH cited FDA for lacking a formal method to log, track, or prioritize Division-specific issues or communicate feedback concerning them

• System evaluation – BAH observed that program operations staff communicates due dates to lead reviewers in an effort to keep reviews on track, but that “more granular internal metrics” could improve this process

A copy of the report can be found here.