The Challenge

In August 2015, the FDA published a draft document titled “Request for Quality Metrics - Guidance to Industry.” In this guidance document, the FDA proposes to calculate four key performance indicators of drug manufacturing quality. They are - Lot Acceptance Rate, Product Quality Complaint Rate, Invalidated Out of Specification (OOS) Rate and the Annual Product Review (APR) or Product Quality Review (PQR) On-Time Rate. If implemented, this guidance will require U.S.-based manufacturing sites to submit data to the Agency for the purpose of calculating the KPIs listed above. The quality metrics calculated by the Agency will then be used as a tool for driving improvements to overall industry quality performance, reducing the potential for drug shortages and focusing inspections using a risk-based approach. It is also the Agency’s intention to foster innovation and improve quality throughout the industry by evaluating, comparing and publishing these metrics.

xCell Strategic Consulting Principals attended a review of the FDA’s proposed Quality Metrics as part of a Parental Drug Association (PDA) sponsored conference held on November 9-10, 2015 in Bethesda, Maryland.

This conference was attended by over 200 representatives from industry and government who came together to discuss the proposed program for using quality metrics as health performance indicators for the pharmaceutical manufacturing industry. Over 20 speakers presented an overview of FDA’s position and approach to implementation, as well as various perspectives from major pharmaceutical companies, Contract Manufacturing Organizations and other interested stakeholders. A number of major companies provided an overview of their current quality metrics programs, as well as the pros and cons of developing a harmonized system as proposed by the FDA.

The Solution

While, in general, attendees were in agreement that metrics improved performance with respect to quality manufacturing of drug products, the complexity by which each organization manages its highly varied facilities led to a series of challenges. Of particular concern was the difficulty involved in aggregating data from contract manufacturers and aligning legacy manufacturing processes acquired through mergers and acquisitions. The table below summarizes the FDA’s perspective and intentions for collecting quality metrics and the industry’s reaction to each of these positions.

FDA Perspective	Industry Reaction
Quality Metrics are based on key performance specifications derived by industry consensus.	Industry generally agreed, but there are differences in definitions and priorities between companies and contract manufacturers.
There has been a noted improvement for product quality as a result of the Quality By Design program, however legacy products still suffer from less robust processes.	Agreed, in general, but most felt legacy products were not properly incentivized and therefore did not result in the same level of process improvement.
FDA is trying to encourage the industry to shift from a compliance focus to a more robust process performance using tools like Six Sigma.	All thought new products would benefit from this new approach. Many companies have aggressive programs in this regard.
FDA would like to move toward risk-based inspections with priorities developed from the highest performing manufacturing sites i.e., a “Dean’s List” of manufacturers.	While companies favored using a risk-based approach based on quality performance, they generally did not support the “Dean’s List” concept.
FDA emphasized that the quality metrics program is a surveillance program and will not be used for enforcement.	Companies, in general, were very skeptical and required assurance that this would not result in an enhanced enforcement program.
The results of the quality metrics program ideally will lead to fewer recalls and a reduction in drug shortages.	This approach will help, however there are a variety of reasons for drug shortages other than quality that this program will not address.
The FDA is expanding the Department of Informatics to manage the massive amounts of data that are expected as part of the implementation of the Quality Metrics program. They are currently developing a guide on how to submit quality metrics data to the FDA. This technical guidance is expected to be available next year.	A number of companies presented the challenges of aligning data to meet the FDA’s requirements. Contract manufacturers, mergers and acquisitions, and complex supply chains all pose a tremendous burden for collecting the data in a format required by the proposal.
The Quality Metrics learning cycle is expected to last at least one year or more as the draft proposal is piloted by industry groups and put into place.	The timing for the learning cycle is undetermined and poses uncertainties for industry planning and priorities.
The Quality Metrics program is focused on registered drug manufacturing sites in the U.S. and is based on U.S. pharmaceutical manufacturing terminology and definitions.	A number of cultural and geographical differences were cited and it was recommended that the FDA focus only on high-risk products within the definition of drug establishment requiring registration.
The FDA is proposing to collect quality data from individual sites and then calculate quality performance metrics using as yet undefined internal criteria. This process creates a data collection and management challenge for both the Agency and the industry. It is undetermined as to the value this practice will create for the patients.	The EU Medicines and Healthcare products Regulatory Agency’s (MHRA) position is more risk-based, where data is collected during inspections and used to schedule future follow-up. They prefer the metrics to be part a self-regulation program that each site is expected to manage. Industry representatives seem to prefer this approach.

The Result

Overall, it was agreed by those in attendance that key quality metrics proposed by the FDA provided an overall indicator of the health of the quality of drug manufacturing.

There is still work to be done to develop consensus regarding the development, application and use of the quality metrics. The U.S. FDA is proposing to collect manufacturing performance data and aggregate it into four key performance indicators as a measure of the overall quality health of a company. By way of comparison, a representative from the EU’s MHRA indicated that they prefer to leave the metrics assessment to the sites and allow a higher degree of self-regulation. The MHRA could then focus their future inspection efforts based on risk.

Attendees felt that sites with positive quality performance metrics should be properly incentivized through a reduction in inspection and compliance oversight. Industry, in general, felt that the use of quality metrics and a risk-based approach should lead to a reduction in the amount of time spent preparing for and hosting FDA inspections.

The majority of those in attendance were not in favor of a "Dean’s List" for preferred pharmaceutical manufacturers. Many opinions and examples were given regarding how this may confuse the general public and lead to biases in enforcement.

Meeting participants felt that there needed to be a continuing discussion around what metrics actually constituted indicators of better quality. They also felt that drug pricing and affordability needed to be considered, as the FDA’s proposed additional data aggregation, alignment, collecting, transmitting, etc. would lead to higher manufacturing costs. These additional costs would ultimately be passed on to the marketplace. This important element is currently not part of the FDA evaluation criteria.

In all cases, regardless of the metrics used and how they are collected, attendees felt that the Agency’s proposal for collecting the required data must be supported by evidence of improved clinical safety and significance to the patient.

For the complete conference agenda, click here .

For information on individual presentations and copies of supporting material please contact Craig Seyfried at xCell Strategic Consulting.