An opinion is the result of CNEDIMTS assessment of safety and efficacy of a device in a specified indication, expressed by the qualification of its “expected benefit” (EB) as insufficient or sufficient; in the latter case its “improvement of expected benefit” (IEB) level is documented. EB is a multifactor criterion based on expertise of clinical benefit, efficacy/safety ratio, and public health benefit. IEB is given in comparison to the established standard of care: level I means there is a major improvement in expected benefit with the IMD; level II is a significant improvement; level III is a moderate improvement; level IV is a minor improvement; and level V is no improvement. An IMD could be used in different indications, resulting in one opinion for each indication. Therefore there could be more opinions than devices; the statistical unit for this study was one opinion, given for one IMD used for one precise clinical indication.
Opinions can be issued for a new application (first or subsequent), a modification, or a renewal. A “new application” was considered as “first”, only for the first application, and as “subsequent” when re-application was due to a previous insufficient EB opinion. After the IMD is included on the list, each modification (e.g. restriction of use or indication) must be assessed by CNEDIMTS. Every 5 years (maximum) after inclusion on the list, a renewal application must be submitted, leading to assess the observed benefit and the improvement of observed benefit. For convenience, EB will refer to both expected (new application) and observed (renewal application) benefit.
Public documents presenting opinions summarise information on clinical data: the number of studies provided by the applicant, the number of studies selected by CNEDIMTS for the assessment process, and the abstract outlining the main characteristics of selected studies (design, number of patients, results). Supporting clinical data are specific or non-specific to the device assessed. Non-specific data relating to the device come from studies performed on an equivalent medical device (same technological characteristics and same intended use) i.e. previous model or device manufactured by a competitor.
Search and opinion selection
The present study included all opinions concerning IMDs issued by the CNEDIMTS in 2008 and publicly available on the website of the French National Authority for Health (http://www.has-sante.fr).
For each opinion, two reviewers (LH and ED) independently examined and collected the data using predefined case report forms. Discrepancies between the two reviewers were resolved by consensus.
Study selection within opinions
Only studies reporting efficacy were examined. For modification or renewal applications, only the new clinical data, i.e. that became available since the previous opinion, were considered. The reviewers extracted the number of studies: i) provided by the manufacturer (when explicitly provided); and ii) selected by CNEDIMTS (including those from manufacturer and those from the in-house literature search) as mentioned in its opinion.
Among the studies selected by CNEDIMTS, the one presenting the highest level of evidence was chosen for methodological description, whether it was specific or non-specific to the device. The levels of evidence of the studies were defined as follows (from the highest to the lowest ): 1) meta-analysis of randomised controlled trials (RCTs); 2) RCTs; 3) non-randomised comparative studies; 4) non-comparative studies i.e. meta-analysis of non-comparative studies, prospective observational registries, prospective case-series, and retrospective case-series. Systematic literature reviews were considered if no specific study was available. When two studies had the same level of evidence, the one with the highest number of patients was analysed.
The following general information was extracted: the date of the opinion; the year of the CE marking (Conformité Européenne); the type of application i.e. new application (first or subsequent), modification, or renewal; the date of previous opinion if relevant; and the opinion result i.e. EB qualification (sufficient or insufficient) and IEB level.
The following main methodological characteristics of the highest level of evidence studies, as available in public documents, were collected:
number of centres (single centre, national multicentre, and international multicentre);
study design (as described above);
total number of patients included in statistical analysis;
output (statistical significance of the main result).
When no studies had been retained as scientifically sound by the CNEDIMTS, we collected the other sources of data (if any) used to provide the opinion.
The categorical variables were described as frequencies and percentages, and the continuous variables as average or median and range (minimum and maximum). For analysis, we considered a variable “EB + IEB” defined with 6 categories: insufficient EB, IEB level V, IEB level IV, IEB level III, IEB level II, and IEB level I. Types of application and collected studies characteristics were cross-tabulated with the results of the opinions (EB + IEB), and types of application were cross-tabulated with study design. Fisher exact tests were used. All data analyses were performed using SAS version 9.2 (SAS Institute Inc, Cary, NC).