Science without sense…double nonsense

Píldoras sobre medicina basada en pruebas

Posts tagged Number needed to treat

The tribulations of an interval

The number needed to treat (NNT) is an impact measure that tells us in a simple way about the effectiveness of an intervention or its side effects. If the treatment tries to avoid unpleasant events, the NNT will show us an appreciation of the patients that we have to submit to treatment to avoid one of these events. In this case we talk about NNTB, the number to deal with to benefit.

In other cases, the intervention may produce adverse effects. Then we will talk about the NNTH or number to try to harm one (produce an unpleasant event).

nnt_enThe calculation of the NNT is simple when we have a contingency table like the one we see in the first table. It is usually calculated as the inverse of the absolute risk reduction (1 / ARR) and is given as a point estimate value. The problem is that this ignores the probabilistic nature of the NNT, so the most correct would be to specify its 95% confidence interval (95CI), as we do with the rest of the measures.

We already know that the 95CI of any measure responds to the following formula:

95CI (X) = X ± (1.96 x SE (X)), where SE is the standard error.

Thus the lower and upper limits of the interval would be the following:

X – 1.96 SE (X), X + 1.96 SE (X)

And here we have a problem with the NNT’s 95CI. This interval cannot be calculated directly because NNT does not have a normal distribution. Therefore, some tricks have been invented to calculate it, such us to calculate the 95CI of the ARR and use its limits to calculate the NNT’s, as follows:

95CI (ARR) = ARR – 1,96(SE(ARR)) , ARR + 1,96(SE(ARR))

CI(NNT) = 1 / upper limit of the 95CI (ARR), 1 / lower limit of the 95CI (ARR) (we use the upper limit of the ARR to calculate the lower limit of the NNT, and vice versa, because being the treatment beneficial, risk reduction would in fact be a negative value [RT – RNT], although we usually speak of it in absolute value).

We just need to know how to calculate the RAR’s SE, which turns out to be done with a slightly unfriendly formula that I put to you just in case anyone is curious to see it:SE(ARR) = \sqrt{\frac{R_{T}\times(1-R_{T})}{Treated}+\frac{R_{NT}\times(1-R_{NT})}{Non\ treated}}nnt2_enIn the second table you can see a numerical example to calculate the NNT and its interval. You see that the NNT = 25, with an 95CI of 15 to 71. Look at the asymmetry of the interval since, as we have said, does not follow a normal distribution. In addition, far from the fixed value of 25, the interval values say that in the best case we will have to treat 15 patients to avoid an adverse effect, but in the worst case this value can rise to 71.

To all the above difficulty for its calculation, another added difficulty arises when the ARR’s 95CI includes zero. In general, the lower the effect of the treatment (the lower the ARR) the higher the NNT (it will be necessary to treat more to avoid an unpleasant event), so in the extreme value of the effect is zero, the NNT’s value will be infinite (an infinite number of patients would have to be treated to avoid an unpleasant event).

So it is easy to imagine that if the 95CI of the ARR includes zero, the 95CI of the NNT will include infinity. It will be a discontinuous interval with a negative value limit and a positive one, which can pose problems for its interpretation.

For example, suppose we have a trial in which we calculated an ARR of 0.01 with a 95CI of -0.01 to 0.03. With the absolute value we have no problem, the NNT is 100 but, what about with the interval? For it would go from -100 to 33, going through infinity (actually, from minus infinity to -100 and from 33 to infinity).

How do we interpret a negative NNT? In this case, as we have already said, we are dealing with an NNTB, so its negative value can be interpreted as a positive value of its alter ego, the NNTH. In our example, -100 would mean that we will have an adverse effect for every 100 treated. In short, our interval would tell us that we could produce one event for every 100 treated, in the worst case, or avoid one for every 33 treated, in the best. This ensures that the interval is continuous and includes the point estimate, but it will have little application as a practical measure. Basically, it may make little sense to calculate the NNT when the ARR is not significant (its 95CI includes zero).

At this point, the head begins to smoke us out, so let’s go ending today. Needless to say, everything I have explained about the calculation of the interval can be done clicking with any of the calculators available on the Internet, so we will not have to do any math.

In addition, although the NNT calculation is simple when we have a contingency table, we often have adjusted risk values obtained from regression models. Then, the maths for the calculation of the NNT and its interval gets a little complicated. But that is another story…

They do not trick you with cheese

If you have at home a bottle of wine that has gotten a bit chopped up, take my advice and don’t throw it away. Wait until you receive one of those scrounger visits (I didn’t mention any brother-in-law!) and offer it to drink it. But you have to combine it with a rather strong cheese. The stronger the cheese is, the better the wine will taste (you can have other thing with any excuse). Well, this trick almost as old as the human species has its parallels in the presentation of the results of scientific work.

Let’s suppose we conduct a clinical trial to test a new antibiotic (call it A) for the treatment of a serious infection that we are interesting in. We randomize the selected patients and give them the new treatment or the usual one (our control group), as chance dictates. Finally, we measure in how many of our patients there’s a treatment failure (how many has the event we want to avoid).

Thirty-six out of the 100 patients receiving drug A presented the event to avoid. Therefore, we can conclude that the risk or incidence of presenting the event in the exposed group (Ie) is 0.36 (36 out of 100). Moreover, 60 out of the 100 controls (we call them the non-exposed group) presented the event, so we quickly compute the risk or incidence in non-exposed (Io) is 0.6.

We see at first glance that risks are different in each group, but as in science we have to measure everything, we can divide risks between exposed and RAR_Anon-exposed to get the so-called relative risk or risk ratio (RR = Ie/Io). A RR = 1 means that the risk is the same in both groups. If RR > 1, the event is more likely in the exposed group (and the exposure we’re studying will be a risk factor for the production of the event); and if RR is between 0 and 1, the risk will be lower in the exposed. In our case, RR = 0.36 / 0.6 = 0.6. It’s easier to interpret the RR when its value is greater than one. For example, a RR of 2 means that the probability of the event is two times higher in the exposed group. Following the same reasoning, a RR of 0.3 would tell us that the event is two-thirds less common in exposed than in controls.

But what interests us is how much decreases the risk of presenting the event with our intervention, in order to estimate how much effort is needed to prevent each event. So we can calculate the relative risk reduction (RRR) and the absolute risk reduction (ARR). The RRR is the difference in risk between the two groups with respect to the control group (RRR = [Ie-Io] / Io). In our case its value is 0.6, which mean that the tested intervention reduces the risk by 60% compared to standard therapy.

The ARR is simpler: it’s the subtraction between the exposes’ and control’s risks (ARR = Ie – Io). In our case is 0.24 (we omit the negative sign; that means that for every 100 patients treated with the new drug, it will occur 24 less events than if we had used the control therapy. But there’s more: we can know how many patients we have to treat with the new drug to prevent each event just using a rule of three (24 is to 100 as 1 is to x) or, more easily remembered, calculating the inverse of the ARR. Thus, we come up with the number needed to treat (NNT) = 1 / ARR = 4.1. In our case we would have to treat four patients to avoid an adverse event. The clinical context will tell us the clinical relevance of this figure.

As you can see, the RRR, although technically correct, tends to magnify the effect and don’t clearly quantify the effort required to obtain the result. In addition, it may be similar in different situations with totally different clinical implications. Let’s look at another example. Suppose another trial with a drug B in which we get three events in the 100 patients treated and five in the 100 controls.

If you do the calculations, the RR is 0.6 and the RRR is 0.4, as in our previous example, but if you compute the ARR you’ll come up with a very RAR_Bdifferent result (ARR = 0.02) and a NNT of 50. It’s clear that the effort to prevent an event is much higher (four vs. 50) despite matching the RR and RRR.

So, at this point, let me advice you. As the data needed to calculate RRR are the same than to calculate the easier ARR (and NNT), if a scientific paper offers you only the RRR and hide the ARR, distrust it and do as with the brother-in-law who offers you wine and strong cheese, asking him to offer an Iberian ham pincho. Well, I really wanted to say that you’d better ask your shelves why they don’t give you the ARR and compute it using the information from the article.

One final thought to close the topic. There’s a tendency and confusion when using or analyzing another measure of association employed in some observational studies: the odds ratio. Although they can sometimes be comparable, as when the prevalence of the effect is very small, in general, odd ratio has other meaning and interpretation. But that’s another story…