Annual report on screening for disease 2007 - The self-testing of body samples

Health Council of the Netherlands. Annual report on screening for disease 2007 - The self-testing of body samples. The Hague: Health Council of the Netherlands, 2007; publication no. 2007/26.

The value of medical self-test kits

The market for so-called medical self-tests is expanding. An increasing number of products and services have become available which enable people to test or have tests done on their blood, urine, faeces or saliva to detect the presence of specific markers. The manufacturers claim that this will enable users to detect the presence of a disease or an increased risk of disease, and that this early detection will be beneficial to their health.
The question is whether this is true. This 2007 Annual Report on Population Screening focuses on the value of self-testing of body samples. Using the available research in this subject area, we have investigated the extent to which self-tests actually live up to their claim of high test accuracy and providing a health benefit through early detection. To this end, we have examined 20 self-tests which, as a group, provide a good overview of what is currently on offer.
This review is preceded by a discussion of the relevant legislation. There is an effective authorisation system in place in the Netherlands for population screening. This protects citizens against tests which may be detrimental to their health. The question now is whether the current legislation and regulations are also sufficient to protect the population in cases where early detection is undertaken on the initiative of the people themselves or individuals, or whether there are gaps in the legislation.

Regulations are not yet adequate to regulate self-tests

In terms of legislation, the Decree on In-Vitro Diagnostic Devices (Besluit in-vitrodiagnostica (IVD)) covers self-tests on body samples. This decree is based on the European Directive 98/79/EC and covers recent regulations which will soon be evaluated. The European Commission is expected to make a proposal for this evaluation in 2009. The evaluation should focus on a number of gaps in the legislation.
Firstly, the ‘essential requirements’ of the IVD Directive are open to interpretation, making it difficult to clearly and consistently assess the tests in terms of diagnostic validity or clinical benefit.
Secondly, there is a lack of clarity about what actions manufacturers and notified bodies should undertake for the Conformity assessment required in order to affix the CE marking for a test. A CE marking should guarantee that a self-test fulfils the essential requirements listed in the IVD Directive. The files are not public, however, and can only be requested for viewing by the Netherlands Health Care Inspectorate. Furthermore, the assessment is generally left to the manufacturer, even for genetic tests or those which detect cancer. This calls into question the credibility of the CE marking system.
Thirdly, for the conformity assessment procedure of self-test kits manufacturers are only required in exceptional cases to provide results of studies with lay persons, even though this could have important implications in terms of reliability of test results. The main exception relates to HIV tests. Thus, for most other tests any available study results are usually based on use by experienced professionals.
Finally, the importance of the Population Screening Act (Wet op het bevolkingsonderzoek (WBO)) as an assessment framework may diminish through the increase in self-tests, even though self-tests and screening do share many similarities. As more people start using cancer self-test kits, the similarity to population screening (e.g. for colorectal cander for which a permit is required) will become greater. Legally, however, the self-tests are products which fall in the free market category and therefore cannot be made subject to the WBO permit requirement as this would be considered an obstacle to trade. However, testing services (home-collecting and street-corner testing) which seek to detect cancer or a serious untreatable condition are subject to WBO evaluation.

Scientific evidence is an important requirement in the evaluation procedure

To evaluate the self-tests, four general criteria have been applied in this Annual Report: diagnostic validity, clinical utility, favourable risk-benefit ratio, and favourable cost-benefit ratio. If no peer-reviewed publications are available for these points, the requirements of objective evaluation of the test’s quality and of publicised and verifiable results are not considered to have been fulfilled.
How do these criteria relate to the legislative and regulatory requirements? They correspond to the essential requirements in the IVD Directive. These should ensure that the purpose of the test is clear, that the test actually works as it is supposed to (and that it therefore performs well in evaluations) and that the risks to the user’s health are acceptable when weighed up against the benefit.
In addition, the assessment in this Annual Report focuses on a number of criteria specific to self-test kits: whether the test can be used responsibly by lay persons, the availability of adequate information/recommendations and whether the test meets the legal requirements.

A handful of tests do have added value, but for many the value is unproven

Twenty tests, chosen to provide an adequate reflection of the growing field of self-tests, were evaluated. They range from self-tests to measure the glucose level in blood or urine to tests for cancer and genetic tests. The findings are summarised in table 1 (see pdf-file).
The conclusion is that self-test kits and services could have added value but, based on the present available evidence, only a small number of the evaluated tests currently represent a welcome addition. Of those tests discussed in the Annual Report, the HPV home-test for cervical cancer and tests which monitor blood glucose concentration and clotting time received a positive evaluation. The faecal occult blood test (FOBT) was also positively assessed. This has been used in a number of trial regions in the Netherlands as a home collecting test for colorectal cancer screening.
The remaining seventeen self-tests cannot be recommended due to a lack of scientific basis. It should be noted that this not only applies to new tests, but also to some which have been routinely used for decades.

The conformity assessment procedure before coming onto the market is unsatisfactory

Seventeen tests did not pass the evaluation in this Annual Report, and therefore also fail to fulfil the essential requirements of the IVD Directive. In some cases it was not possible to properly investigate whether they met these criteria, since there is considerable scope for questioning current interpretations.
This strongly suggests that self-tests are not properly evaluated according to the essential criteria before being put on the market. Either the required evaluation is not carried out, or the requirements are interpreted in a minimal manner. It is also questionable whether the ‘notified bodies’ are adequately equipped to carry out this type of assessment, as their primary skills do not lie in the field of medicine and epidemiology.

Provision of information to the consumer is inadequate

A further conclusion is that the information provided through the Internet by the manufacturer or supplier regarding a self-test kit is generally inadequate. This makes it difficult for the consumer to reach a well-considered decision about the value of a particular self-test. A description of the precise aim, the anticipated gain and health risk and the diagnostic value are often missing. Furthermore, it is often not possible to find out whether the self-test has the CE marking before purchasing it over the Internet.
A further problem is that the IVD decree often leaves it up to the manufacturer to describe the aim of a self-test. This leaves scope for claims of outstanding test performance, using terms such as ‘reliability index’ or ‘agreement’ with an unspecified reference test, which have little to do with the diagnostic validity and the actual aim of the test. Consequently, sound information on the diagnostic value of the test is lacking and the risk of false-positive and false-negative results is masked. In addition, there are no guarantees that consumers will be able to properly interpret the results of a self-test.

Recommendations

The Annual Report contains a set of recommendations to deal with the identified problems. Some of these recommendations relate to regulatory issues. For example, it is necessary to define the current IVD essential requirements more clearly, and to improve the quality of the conformity assessment. Furthermore, an investigation is necessary into how CE marking is assigned.
Even if a product is allowed to be introduced onto the market, the necessary changes can still be made. For example, product information requirements should be put in place to enable consumers to make a well-considered choice. An investigation into the most appropriate channel of sales is recommended, and advertising claims should only be permitted if backed up by scientific evidence.