Correct use of NNT
Once upon a time, the scientific paradigms and the way of thinking of researchers (and also of clinicians) began to change, from “we’re doing well” to really wanting to know what was the validity of the information they collected in their experiments or in their daily practice.
It is in this context that, 33 years ago, what would become one of the lords of the impact measures of clinical studies came to light: the number needed to treat, known worldwide by its acronym, NNT.
Number needed to treat (NNT)
Its initial usefulness was to assess the beneficial effect of a treatment to reduce the risk of an unpleasant event occurring in an intervention group of interest, always with respect to what was observed in a control group. Put more simply, it emerged as a measure of impact in the context of randomized and controlled clinical trials.
The NNT was initially well received, since it has the great merit of combining, in a single parameter, the concepts of statistical significance and clinical relevance (given that the p-value or its confidence interval is provided). In addition, it is easy for clinicians to interpret without the need for in-depth statistical knowledge.
Following a parallel with the ages of life, everything went well during his childhood and he continued to grow during his youth, extending to many other areas other than the conventional parallel clinical trial.
The problem with youth, in addition to being at the beginning of life, is that it does not last long. And in maturity, although it continues to have enthusiastic followers, the NNT has also begun to accumulate detractors and critics who have begun to draw defects from it.
It is about these tribulations that the NNT suffers during its maturity that we are going to talk in this post. Although there are complaints produced by some of its unpleasant characteristics from the mathematical point of view due to its logarithmic inheritance, most are based, in reality, on a poor understanding of its meaning or on a not entirely correct use of the parameter.
The tribulations of NNT
As we have already said, the NNT was initially developed to assess the efficacy of a treatment to reduce the risk of producing an event in an interest group.
For example, if the mortality of fildulastrosis is 5% per year and with a new treatment it drops to 3%, it means that the treatment reduces the risk by 2%, so the NNT will be 1 / 0.02 = 50. This means that for every 50 patients we treat for 1 year, we will prevent one from dying thanks to the treatment. Of course, she can die if we prolong the follow-up, as we will see later.
The dual nature of NNT
This approach is very simplistic, since the NNT is actually dual in nature.
The NNT can be understood as the number necessary to treat in order to increase the number of expected positive events by one or to decrease the number of a negative event by 1, all during a specific follow-up period.
But it can also be understood in the opposite sense, in that of harming: the number needed to treat to increase the number of a negative event by 1 or to decrease the expected number of a positive event by 1, all during a follow-up period determined.
This can make it confusing to assess the NNT just by its numerical value. To avoid this inconvenience, some authors have thought that we can call each of the NNT dualities differently, so that we will always know exactly what we are talking about.
Basically, we would be specifying the direction of the effect we are studying.
Thus, we would talk about the number necessary to treat to benefit (NNTB) when we want to express the number to treat to achieve a beneficial effect (or avoid an unpleasant one), and the number necessary to harm (NNTH) when we want to refer to the number necessary for a negative effect to occur (or to avoid a positive one that could occur without treatment).
Another problem that greatly bothers NNT detractors is that its calculation can, at times, generate negative values of its point estimate and, at other times, confidence intervals that cross zero into the realm of negative numbers. Here, the difficulty is in finding a logical meaning to a negative NNT.
Let’s imagine that we obtain a NNT = 10 with a 95% confidence interval (95 CI) of 8 to 12. Here we have no problem, we have to treat 10 (point estimate), although this value can range from 8 to 12 (estimate per interval).
The problem arises when negative numbers appear. For example, if the NNT = 5 with a 95 CI of 8 to -12, how do we assess it?
Well, for this it is good for us to resort to the dual nature of the NNT that we have mentioned earlier. If we think about it, NNT values between -1 and 1 are impossible. Thus, the interval from 8 to -12 could be divided into two: NNTB of 8 (up to infinity) and NNTH of 12 (up to infinity). I think the usefulness will be limited from a clinical point of view, but at least we will have given it a meaning.
Everything would be easier with a time machine
As we discussed in a previous post, when we want to study the efficacy of an intervention, the ideal would be to give the new treatment, end the follow-up period and measure the effect. Then we would use our time machine to go back to the initial moment and, instead of the treatment under study, we would give its alternative, end the follow-up and measure the result.
Once this is done, we would compare the two results. The problem, the most awake of you will have already noticed, is that the time machine has not yet been invented. This means that to obtain this other result, which we call potential or counterfactual, we have to resort to the control group of the trials, which serves as a substitute.
If we think a bit about the implications of the counterfactual theory, although the efficacy of the treatment under study is always the same, the value of the NNT will depend on which intervention we compare it with. Therefore, to correctly interpret the NNT, the comparator that we have used must always be explicitly specified.
An NNT of 10 for a given treatment can only be assessed if it is specified which was the control intervention and how long the follow-up time were, especially if they are different.
So keep it in mind: the value of the NNT must be expressed together with the treatment alternative and the follow-up period. Failure to do so may make your assessment difficult and misleading.
Never despise a decimal
Virtually all books and manuals agree that the numerical value obtained for the NNT should be rounded to the nearest higher integer. It’s logical, it doesn’t seem like it makes much sense to say that we have 4.8 patients to treat, so we round it up to 5.
The problem with rounding is that it adds imprecision to the estimate and can be misleading.
For example, any of the absolute risk reduction values between 0.52 and 0.9 will be equivalent, after rounding, to NNT = 2. However, there is a big difference between a reduction of 52% and a reduction of 90 %. We should not value them with the same NNT.
So, if nobody throws up her hands in horror when she hears that the average is to bear 1.2 children, why the phobia of giving decimals with the NNT? After all, it is still an estimator that we have to know how to interpret.
If we see a NNT of 6.7 we can conclude that we would have to treat an average of 6 to 7 patients to achieve a beneficial effect during a certain period of time. A warning, if we do so, it will be necessary to make it clear that the estimate will be between 6 and 7, but that these are not the limits of the 95 CI. Let us not get confused.
Don’t forget about confounding variables
You already know that all studies are subject to the effect of confounding variables, especially when they are not randomized. We are used to seeing association measures adjusted for variables that the authors believe may act as confounders. However, this is often seen that this is not so for NNT and only its raw value is provided.
Other times, what happens is that not appropriate adjustment methods are used. A wide variety of methods have been developed to calculate the NNT in multiple scenarios, crude and adjusted. If you don’t know which one to use, find someone who knows before applying the wrong one.
Time also passes for NNT
We have said it before, but it does not hurt to insist: the value of the NNT depends on the length of the follow-up period, so it must be specified. The proportion of events that take place increases as time goes by.
For example, if the treatment produces a reduction in the risk of the event that remains constant over time, the value of the NNT will be lower as the temporal duration of the follow-up period increases. It is easily understood that the duration of follow-up must be known to correctly interpret the value of the NNT.
Consider two clinical trials of two different interventions, one with a follow-up period of 2 years and the other of 5. Even if the two studies gave us an NNT of 8, it would not be the same to treat 2 years as 5 to avoid an unpleasant event in 1 out of 8 patients treated.
Beware of survival studies
In survival studies and when evaluating the results per person-time, the frequency of the study event must be taken into account when deciding the method of calculating risk reduction, which, in turn, we will use to calculate the NNT. This should be done even if the risk is constant and the follow-up period of the participants is homogeneous.
We are going to see how the NNT would be calculated with an example that we are going to invent on the fly. Imagine that we conducted the study and observed 10 cases of death per 100 person-years in the intervention group and 5 cases per 100 person-years in the control group.
Assuming that survival times are exponentially distributed, we first calculate the proportions or accumulated risks in the intervention group (Ri) and in the control group (Rc):
Rc = 1 – e-10/100 = 0,095
Ri = 1 – e-5/100 = 0,048
Now we can calculate the NNT as the inverse of the risk difference, as we already know:
NNT = 1 / (0.095-0.048) = 21.2
The common mistake in this situation is to directly use the number of events according to the following formula:
NNT = person-time / (difference of events)
If we apply it to the previous case, it would look like this:
NNT = 100 / (10-5) = 20
As you can see, the first method, which is the most appropriate, is somewhat more conservative and gives higher NNT values.
Only in cases where the frequency of the event is very low can we estimate the NNT directly using the number of observed events. Imagine that, in the previous example, we observed 5 deaths in the intervention group and one in the control group. We could calculate the NNT as follows:
NNT = 100 / (5-1) = 25
Anyway, do not get carried away by the easy way. It will only be correct to calculate the NNT without prior conversion of the accumulated proportions when the number of events has a very low frequency and, in addition, the risk differences between the two groups remain proportional over time. When in doubt, convert.
To recapitulate a bit what we have said, we can recommend taking a series of precautions to make a proper use of the NNT and to be able to interpret it correctly.
First, never use the NNT without specifying the alternative treatment, the direction of the effect, and the length of the follow-up period.
Second, always calculate your confidence interval. In case it has negative values, consider the dual nature of benefit and harm to try to make a more understandable interpretation.
Third, don’t shy away from decimals. Remember that this is just another estimate. You should have no problem evaluating its point estimate (even if it is not a whole number) and its confidence interval.
Finally, check that you are using the correct methodology in more complex situations, such as those where confounding factors may be involved or in survival studies.
If we use it wisely, the NNT will be able to continue his adventures through his maturity and be with us for at least another 30 years.
And here we are going to leave it for today.
We have already seen the usefulness of the NNT to assess the efficacy of an intervention. For example, it tells us how many deaths we can avoid during the study follow-up and what would have occurred if we had not intervened.
But what about those who don’t die? Are there participants who will die the same if we do or do not treat them? Well, the NNT doesn’t tell us anything about that. To study this aspect, which would improve the NNT assessment, we need to resort to another parameter: the number remaining at risk. But that is another story…