Creating an EU CER Literature Review Protocol and Reviewing Medical Device Clinical Data
Once you have created a robust clinical evaluation plan for your medical device(s), it’s time to get busy figuring out what data you need, drafting a protocol, and then taking stock of that data. This is considered Stages 1 and 2 of the CER process and the specifics can be found in MEDDEV 2.7/1 rev. 4, as shown below.
Clinical Evaluation Stage 1:
Identifying Which Data You Need to Fulfill the Requirements of Your Plan
There are two broad categories of clinical data.
1 – Data not generated by your company (retrieved from online literature searches and other offline data)
2 – Data generated by your company, which can include:
- Premarket clinical investigations, including bench testing reports
- Postmarket data gathered from risk management and PMS activities, including PMCF studies, device registries, and more
Crafting a CER Literature Search and Review Strategy
For many companies, the data retrieved from literature searches will represent most, if not all, of the data they collect. That’s why it is so critically important that you develop a literature search strategy that is robust and can be replicated during subsequent updates to your CER. The output of your search and review should obviously include literature on your device and any identified equivalent device, plus a review of the current state of the art. Your literature search protocol should include the following elements (for more specifics on this, see Appendix A5 in MEDDEV 2.7/1 Rev. 4):
- Sources of data you will use (e.g., MEDLINE/PubMed, Embase, Google Scholar, ResearchGate, internet searches, etc.)
- The methodology you plan to use for searches
- Exact search terms and parameters (e.g., dates) used to search scientific databases and the internet
- Your specific selection or exclusion criteria along with justification for each
- How you will address duplication of data from multiple sources
- How you will ensure data integrity (e.g., QC methods or second reviewers)
- How you will appraise each data source and its relevance to your device
- How you will go about analyzing and processing the data
You should think of your plan as you would a standard operating procedure (SOP) or a detailed instruction. As an example, if you were to leave the company a year from now, your successor should be able to read your protocol and understand exactly what was done during the previous update.
Make sure you treat the literature searches related to device equivalence as a separate activity from the searches on current state of the art. Separately define and track search terms used, database sources, date parameters, etc. Also, be cognizant that your idea of “state of the art” may differ from reality – here’s why. Before finalizing your protocol, test it. Identify known papers relevant to your device or current state of the art and then test those search parameters to make sure those known papers appear in your search results.
Review It All: The Favorable…and the Not So Favorable
Your literature searches should be extremely thorough and encompass a breadth of search criteria. They also need to be documented in detail so the results can be independently verified and replicated. Your selection of literature should be objective (the good and the bad) and justifiable. Nearly every published study has citations that may lead you to additional relevant data. Again, this is where planning and staying within your device search parameters become really important, because you can very quickly wander down a virtual rabbit hole and not be able to replicate how you got there.
While most publications are available in English (e.g., 93% on MEDLINE), make some attempts to do basic searches in French, German, or Japanese using online translation tools. Obviously, if you work for a larger company that has offices in Europe or Asia, getting help from native-speaking colleagues would be ideal. While the odds are good that you will find what you need in English, you might uncover valuable data that can be translated and further appraised if needed. Also, if you are updating an existing CER, we recommend that you start your search a few weeks before the end date of previous search to ensure you don’t miss anything that may have been added at the very end of your previous search period but had not yet been indexed online.
Keep in mind that some data may be publicly available but not easily accessible online. For instance, the label and IFU of a competitive device (if using one) may yield useful information. Also, presentations made at industry conferences may provide information that is not available (or has not yet been published) online.
Clinical Evaluation Stage 2:
Choosing the Appropriate Data Sets Also Requires a Plan
During Stage 0, you came up with a clinical evaluation plan focused on determining which data you need and from what sources. In Stage 2, your focus will turn to choosing the right data among the data sets you have found. This requires yet more planning. Your data appraisal plan should address:
- Criteria for determining the quality, relevance, and validity of the data
- How you will weigh the data related to favorable and unfavorable results
According to Section 9.2 of MEDDEV 2.7/1 Rev. 4, the criteria you select should “reflect the nature, history and intended clinical use of the device.” This is something you must document and justify based on current state of the art.
Is Your Data Valid and Relevant?
So you found plenty of data in Stage 1 that seem relevant to your device and its intended purpose. But how do you know if the quality is good? At first glance, you don’t. Sure, a paper published in The Lancet is instantly credible, but the quality of a publication cannot serve as a rationale for selection. Ultimately you need to dive headfirst into the pool and do investigative research of your own to appraise the quality of the data.
Doing this requires some skill, because it is on you to evaluate the methodology used to collect the data and therefore determine its scientific merit. As part of this appraisal process you will also need to weigh the contribution of each data set to the overall clinical evaluation. Section 9.3 of the MEDDEV contains seven pages of advice on how to evaluate the methodological quality and scientific validity of data sets.
A Higher Bar for Medical Device Equivalency Under the MDR
Many companies use equivalency claims to avoid having to conduct redundant pre- or postmarket clinical studies that prove safety and performance. This part of the clinical evaluation process is not new. What is new is the level of scrutiny those comparative evaluations will endure. The clinical evaluation requirements in the existing Medical Devices Directive (93/42/EEC) and MEDDEV 2.7/1 Rev. 3 largely favored device equivalency but did not define it. Thus, some companies took a liberal view of “equivalent.” When MEDDEV 2.7/1 Rev. 4 was released in mid-2016, it gave manufacturers a lot less latitude for determining which devices could be considered equivalent. In fact, Appendix A1 in Rev. 4 is quite clear about the clinical, technical, and biological characteristics your device must have in common with an “equivalent” device.
The MDR furthers tightens the screws for Class III and implantable devices, requiring a more in-depth assessment and making it more challenging to leverage competitor data for new devices. That’s because Article 61, Section 5 of the MDR requires manufacturers of such devices to have access to the full technical documentation of the competitive device(s). It also instructs Notified Bodies to ask for proof that you have a contract in place granting you permanent access to that technical documentation. Good luck with that.
Appraising the Medical Device Clinical Data
With an appraisal plan created and a grip on how to execute it, you can begin the hard work of appraising the data you have found. This appraisal must be done based on the complete text of publications you find, not just by reading the abstracts or summaries. For each document you appraise, you are required to document your appraisal of it to the point that it could reasonably be reviewed by others. The appraisal results should also support conclusions you are making about clinical safety and clinical performance of the finished device (e.g., citing non-device-related literature would be ranked low for appraisal).
Appendix A6 in MEDDEV 2.7/1 Rev. 4 can be helpful in performing your appraisal of data. It provides some examples of red flags that should make you pause, including clinical data that:
- Lacks basic information such as the methods used, number of patients, identity of products, etc.
- Has data sets that are too small to be statistically significant
- Contains data that applies improper statistical methods
- Employs studies that lack adequate controls
- Has an improper collection of mortality and serious adverse event data
- Depicts a misrepresentation by the authors
The issue of evaluating statistical methods and significance intimidates many RA/QA professionals. If you feel uncomfortable about your “stats skills” for conducting statistical analysis or reviewing statistical information, hire some outside expertise to evaluate these specific aspects. Better to invest in doing that now rather than have a Notified Body reviewer challenge your appraisal later.
Next Step: Analyzing Your Clinical Data and Drawing Conclusions
If you found this blog post to be informative, continue reading our next post on Stage 3 in the process. Also, if you need to get up to speed on all CER requirements contained in MEDDEV 2.7/1 Rev. 4 and the EU Medical Device Regulation, consider our 3-day EU CER training class.