Fixed-effects model.

Some tongue-twisters trip up your tongue… and others tie your brain in knots. Picture a poet trying to rhyme “fixed-effect model in singular” with “fixed-effects model in plural” without losing their sanity or the rhythm. Tough, right? But not impossible. In fact, there’s a corner of meta-analysis where this tongue-twister not only makes sense, but also reveals nuances so subtle they’re dangerously easy to overlook.

Because it turns out that a fixed-effect model (yes, the lone and minimalist one) is not the same as a fixed-effects model (plural, as if it brought some friends along). And nope, it’s also not the same as the random-effects model, even if sometimes it feels like they’re all partying together at the same methodological bash with swap-out names. The confusion isn’t just about semantics: picking one over the other can totally change how we interpret an entire set of studies. Or worse, it might make us think we’ve understood something when all we’ve really done is tangle the thread even more.

So, in this post, we’re going to try to untangle this statistical tongue-twister. Because when models sound so similar but behave so differently in practice, it’s worth stopping to ask who’s who in this knot of effects.

James Bond style… but backwards

Some confusions are wildly widespread. One of the classics is the famous James Bond line you’ve probably heard in the Spanish-dubbed movies, when he orders his iconic Martini: mixed, not stirred.

Don’t ask me why, but in the original English version (the native tongue of the supposed 007), what he actually liked was the opposite: shaken, not stirred. That means he preferred his drink made in a cocktail shaker (shaken), rather than gently blended with a spoon, which is the exact opposite of what cocktail purists recommend.

In the world of meta-analysis, we have one of these widely spread mix-ups too, when it comes to how we “combine” primary studies, though here we’re neither stirring nor shaking anything.

As we’ve seen in previous posts, to combine the primary studies in a meta-analysis, we need to weight their results. This weighting depends on the characteristics of the studies (and the populations they come from) and on how much they differ, what we call heterogeneity.

If we assume that all studies are estimating a single, shared true effect across all populations, we use what’s called the fixed-effect model (singular).

This model is great, but it usually doesn’t match reality. In practice, the populations involved in the primary studies don’t tend to share a single effect; each one tends to have its own. That means there’s actually a distribution of effects, and in those cases, we use the random-effects model. Feel free to revisit the earlier post where we covered this.

Now, pay attention to this detail. In that second case, the effects are multiple and distributed randomly, hence the name random-effects model, in plural.

But in the first case, there’s just one real effect, always the same one. That’s why we call it fixed-effect, singular. And yet, just like with James Bond’s Martini, it seems we prefer calling it by its plural name.

Who cares, right? Well, it wouldn’t be a big deal, except there really is a fixed-effects model (plural) in meta-analysis too, especially when we talk about subgroup analyses. And, as we’ll see later on, fixed-effect and fixed-effects models are very different beasts.

Subgroup analysis in meta-analysis

Let’s say we’re running a meta-analysis on the effectiveness of a new treatment for that dreadful disease, fildulastrosis. When we crunch the numbers and calculate the overall summary measure, we’re disappointed: no effect in favour of the treatment is detected.

But then, digging a bit deeper into the results, we notice two or three primary studies where the treatment does seem to work. These could be extreme studies (outliers) or influential ones, as we discussed in a previous post.

The tricky part is, while we have techniques to detect outliers or influencers, they don’t tell us why these studies deviate from the rest. One possibility is that they include some factor driving their heterogeneity, and maybe, just maybe, that same factor also makes the treatment more effective. That would be great: finding at least one group of patients who could benefit from the new treatment.

To investigate that, we need to perform a subgroup analysis, which is basically a meta-analysis within the meta-analysis. Here, heterogeneity stops being an annoying obstacle and becomes an intriguing kind of variability that can help us form new research hypotheses.

The general procedure is similar to the original meta-analysis. First, we group the studies and calculate a summary measure for each subgroup. Then, we compare those subgroup summary measures, just like we’d compare individual study estimates in a regular meta-analysis.

To calculate those subgroup summaries, we can use the fixed-effect model (if we believe all the studies in a subgroup estimate the same true effect), or the random-effects model, which, as we’ve said, is usually more realistic and therefore more advisable.

That way, we end up with multiple effect estimates, each with its own variability measure, its own tau-squared. However, in practice, what we often do is estimate a single tau-squared value that covers all subgroups. Or, to put it more elegantly, we assume all subgroups share a common estimator of between-study heterogeneity.

The fixed-effects model

Alright, now we’ve got the estimate for each subgroup, so the only thing left is to test whether they’re all the same. If we end up rejecting the null hypothesis of equality, that means at least one of the groups is different from the rest.

One simple way to approach this is to imagine that each subgroup’s summary measure is actually what we’d get from a larger study, like we had pooled all the participants from that group into a single trial. So, if we have five subgroups, for instance, we pretend we’ve got five studies and want to see whether their estimates differ.

If you think about it, we’re back at square one. The first thing we need to assess is the heterogeneity among these “studies” to figure out how we can combine their results. For that, we can use Cochran’s Q to check whether its value points to differences greater than what we’d expect by chance alone, which would suggest heterogeneity. But that’s not the real issue here. The real question is whether we’re dealing with a single fixed effect or a distribution of random effects.

As we’ve already said, within each subgroup, it’s usually more accurate to assume the effects follow a random-effects model. But the subgroups themselves are based on certain characteristics of the studies, so the summary measure of each subgroup can be considered a fixed effect for populations with those characteristics.

You see where this is going, right? We’re comparing several subgroups, each with its own fixed effect. So, it’s no surprise that this type of analysis is called the fixed-effects model, in plural.

When one letter makes all the difference

If we take a moment to reflect on everything we’ve said, we can start to see the fixed-effects model as a kind of hybrid, blending features from both the fixed-effect model and the random-effects model.

Just like in the random-effects model, we assume there’s more than one true effect size that we’re trying to estimate: specifically, one effect for each subgroup of studies. But the key difference here lies in the fact that these subgroups aren’t formed randomly, they’re based on specific characteristics. So, what we end up with is a series of fixed effects, each determined by the traits that define its subgroup.

As you can see, this particular tongue-twister of effects doesn’t exactly twist your tongue but, as we said at the start of this post, it definitely ties your brain in knots. The confusion comes from the word “fixed,” which can mean different things depending on what part of the meta-analysis we’re talking about.

In a standard meta-analysis, fixed effect is basically synonymous with common effect. All the studies are assumed to share the same effect across their populations. But that changes when we get into subgroup analysis, where the term fixed effect simply means it’s not a random effect.

Now you’re probably thinking, “Okay, now the knot’s fully formed.” But no… we can still tangle it a little more.

Because the fixed-effects model includes elements of both the random-effects model (within subgroups) and the fixed-effect model (assuming the effect level in each subgroup is fixed), some people like to call it by a prettier name: the mixed-effects model.

Now we’re talking. The knot is complete; and tied tight.

We’re leaving…

And now that we’ve seen how a single letter can turn a “fixed” effect into a fixed mess, it’s probably time to wrap it up for today.

To recap: the fixed-effect model (singular) assumes one single truth for all; the random-effects model allows for many possible truths. And the fixed-effects model (plural)? Well… that one’s a bit like a double agent playing both sides. A hybrid between order and chaos, between Dry Martini and cocktail shaker.

In the end, what really matters isn’t just mixing, but knowing what we’re mixing, and why. And just like James Bond’s famous line, where what sounds stylish might actually be a mistranslation, in meta-analysis it’s also worth zooming in to check what model we’re using, so we don’t end up shaking what we only meant to stir… or the other way around.

And if you thought this was the end of the tangle, get ready for the next plot twist. Because subgroup analysis? That’s just the tip of the iceberg. We’re actually one step away from something even more powerful: a model that can include subgroups, continuous variables, even random countries.

We’re talking about meta-regression, another fierce beast in the wild world of meta-analysis. But that is another story…