Are Emulation Trials a Fantasy? Save

This year at #ACR24, emulation trials are being presented.

Are these ‘trials’ helpful, despite the biases that occur with observational data, or do they truly mimic the results of randomized controlled trials (RCTs)? In the latter, both known and unknown confounders should be distributed equally, whereas most observational data has unknown or missing data on factors that are associated with both the intervention and the outcome of interest.

What are the strengths of emulation trials?

Head to head treatments can be compared from large observational / administrative databases but this would not be feasible in actual RCTs. Likewise, safety can be compared as if a large database of real-world evidence is used, then rare events can be compared between various treatments.

Frideres, et al (abstract 1713) compared ILD in RA using the Veteran Affairs data, which is a related study that was presented last year by England et al, and compared TNFi to other advanced therapies in RA to study the rates of new ILD in the various groups. England et al demonstrated that those with TNFi use had more new-onset ILD but when adjusting for several confounders (disease activity, smoking, etc), the rates of new ILD did not differ between TNFi users and those with other advanced therapies. The generalizability of the data may not apply to most RA patients, who are primarily women who don’t smoke, as the VA patients have a large proportion who are men. However, groups within the study were matched so the internal validity should be high. RCTs in this area would be very large and expensive. Frideres added to this knowledge by comparing 3 emulated trials of rituximab vs each abatacept, tocilizumab, and tofacitinib. Patients with RA and ILD who started any of these drugs did not differ in outcomes over 3 years.

We still need to be aware of channeling bias (differences in prescribing various drugs such as patient and physician preferences, tiered access to medications) and selection bias (who is included and excluded in the emulated trial) and missing data as the missingness may not be random.

Other emulation studies asked the questions about steroids with cyclophosphamide in eGPA vs glucocorticoids alone in both patients with or without poor prognostic factors (abstract 1599 and abstract 1600). I would think that patients who did not have poor prognostic factors would be far less likely to be offered cyclophosphamide so I wonder if the groups, although matched, would be less comparable.

So, a well-designed and conducted RCT will always have stronger evidence than emulation trials and if you think any emulation data are not concordant with RCTs, you may suspect that the emulation has several biases leading to a result that you consider to be ‘fantastic’ and out of keeping with the bulk of other high-quality evidence and clinical wisdom.