02.09.2016 |

LOGFILE No. 46/2016 - Quality Metrics - What will the FDA require in the future? How are companies already benefitting today?

Quality Metrics – What will the FDA require in the future? How are companies already benefitting today?

(5 min. reading time)

by Dr. Sabine Paris

In July 2015 the U.S. FDA published the draft of a guidance for industry entitled „Request for Quality Metrics“ . The FDA thus became the first government agency to take up the topic of quality metrics, and it has plans to collect data from pharmaceutical companies in the future.

During this year’s GMP Conference (GMP-BERATER-Tage), on 28 October 2016 participants in one of six GMP dialogues, together with experts Claudia Pachl, ValTex GmbH, and Dr. Rainer Gnibl, Government of Upper Bavaria, Germany, discussed a range of topics including the FDA’s plans, the current EU regulations and particularly the significance of metrics for the companies themselves.

Today we are summarising the most important results for all who could not follow the discussion live.

What data will the FDA require in the future?

Rainer Gnibl introduced the topic of quality metrics in an initial talk. He introduced the data that will have to be submitted in the future to the FDA per manufacturing site, according to the current draft. These are, in detail:

  • number of lots
  • number of specification-related rejected lots
  • number of lots pending disposition
  • number of product-OOS results (including stability testing)
  • number of lot releases and stability tests
  • number of invalidated out-of-specification (OOS) results for lot release and stability tests (laboratory errors)
  • number of product quality complaints
  • number of lots released for distribution
  • number of Annual Product Reviews (APRs) and/or Product Quality Reviews (PQRs)
  • number of APRs and/or PQRs completed/not completed within due date

The following quality metrics can then be calculated from these figures:

  • Lot Acceptance Rate
  • Product Quality Complaint Rate
  • Invalidated Out-of-Specification (OOS) Rate
  • APR or PQR on Time Rate (Note: This parameter is no longer included in the just published, revised guidance draft (see below).)

“The data is already present in European companies. The only new part is that in the future it will have to be submitted to the FDA”, explained Rainer Gnibl.

What are the regulations within the EU like?

The EU-GMP Guide does not yet include requirements on quality metrics that are comparable to the U.S. plans, nor are there any in planning. “Metrics per se are nothing new. As part of the Pharmaceutical Quality System (PQS), it is already expected today that certain quality data be compiled and evaluated”, says Rainer Gnibl. As examples he mentioned:

  • Management Review (Chapter 1.6)
  • EU PQR (Chapters 1.10 and 1.11)
  • Ongoing Process Verification (incl. Process Capability/-Performance) (Annex 15, Chapter 5.28 - 5.32)
  • Register (Release Register) (Guideline 2001/83/EC, Art. 51)
  • Complaint Management System (Chapter 8)
  • CAPA-System (incl. CAPA Reviews) (Chapter 1.4.xiv)
  • Change Management System (deviations, changes, OOS) (Chapter 1.4)

These metrics are also subject to queries by GMP inspectors from monitoring authorities during inspections. Companies located in the EU are only affected by the future FDA regulations if they manufacture for the American market.

The FDA also plans to use quality metrics to determine inspection intervals. For this purpose European authorities have long been using a Quality Risk Management tool from the Compilation of Community Procedures (CoCP), with which the compliance factors are calculated.

What metrics will help me specifically?

It quickly became clear in the discussion that for the companies themselves quality metrics are of inestimable value for improving quality. The experts presented their case for moderation when dealing with quality metrics: The objective is not to define all sorts of parameters as metrics and to generate data cemeteries. Parameters relevant to quality must rather be internally defined and seriously processed within the company (root cause analysis).

For Rainer Gnibl quality deficiencies, deviations and complaints (from the outside) are particularly relevant to quality. “A prerequisite for defining the truly relevant metrics is a well-founded process comprehension. This is the only way to improve,” emphasised Claudia Pachl.

Do metrics have to delve deeper? What other factors are important?

“Individual figures generally do not reveal much. It is important to observe the data and its development over an extended period of time. Keyword: Trending,” says Claudia Pachl, “for that is when I have to do causal research. Do I know the causalities behind a figure? Why has it changed?”

“It is also important not overly isolate the metrics under consideration. Defining levels can be relevant. For example, it could make sense to analyse quality-relevant complaints concerning all products of a contract manufacturer that were produced on the same line. This makes it possible to discover systematic faults as well,” advised Rainer Gnibl.

During the lively discussion it became clear that the classification of deviations and complaints also plays a role. Under no circumstances is it acceptable to classify major deviations as merely being “minor”.

By the same token, a clear definition of metrics is important. What do I understand by deviation?

What about the comparability of manufacturing sites on the basis of metrics?

The FDA also wants to compare manufacturing sites on the basis of quality metrics. “In large companies this is precisely what is already happening. In order to actually be able to make comparisons it must be clear whether or not the same database is on hand at all. How was the data gathered? What will be done with it?” asks Claudia Pachl.

Rainer Gnibl referred to the EU-GMP Guide: “According to Chapter 1 of the Guide, only the site-specific functionality must be proven. The QA system must function independently in each particular production site – and not merely within a group of companies, as is often preferred by global QA units”.


Even if the guideline of the FDA is still a draft and European authorities are not planning comparable regulation within the near future, it became clear that gathering and evaluating quality metrics are extremely important for manufacturers:

  • Quality metrics are not new.
  • Every manufacturer should have them and use them to improve quality.
  • A practised quality policy/culture (from top to bottom) is essential for a meaningful gathering and processing of the metrics.


Editorial Note:

On 23 November 2016, after the GMP conference, the FDA published a revised draft of its guidance: Submission of Quality Metrics Data, Guidance for Industry, Draft Guidance, November 2016, U.S. FDA, CDER, CBER.

You can read more about the new draft in our GMP News.


Dr. Sabine Paris
Maas & Peither AG - GMP-Verlag