Fool's gold?
Critics are chipping away at the 'gold standard' for gender clinics
The gist
The “gold standard” Dutch evidence that launched a worldwide surge in medicalised gender change for young people is in truth low-quality research riddled with potential bias, according to a new critique.
The 2011 and 2014 Amsterdam studies that pioneered puberty blockers wrongly concluded that the distress of gender dysphoria eased after treatment, and failed to show any meaningful benefit to the mental health of their transgender-identifying patients.
This is the argument of an article titled “The Myth of ‘Reliable Research’ in Pediatric Gender Medicine”, published earlier this month in the Journal of Sex & Marital Therapy, and authored by researcher E. Abbruzzese from the Society for Evidence-based Gender Medicine, psychiatrist Stephen B. Levine and paediatrician Julia W. Mason.
The authors say the Dutch practitioners may have achieved “technical excellence” within the one clinic as they adapted the approach to medical transition for their young patients, but their research was not robust enough to underpin the rapid international expansion of paediatric gender medicine that they inadvertently inspired.
Nor have more recent “gender-affirming” researchers applying the Dutch medical technique remedied the shortcomings of those 2011 and 2014 foundational studies, according to the Abbruzzese article. (The 2011 study covered puberty blockers only; the 2014 paper followed up the patient group after opposite-sex hormones and surgery. This three-stage intervention is known as “the Dutch protocol”.)
The Abbruzzese article says gender clinics especially in the United States have ignored the weak evidence base for their invasive treatments, relying on the “eminence-based medicine” of policy endorsements from health professional societies captured by identity politics.
Hence the “false narrative [that] gender-affirming medical and surgical interventions for youth are as benign as aspirin, as well-studied as penicillin and statins, and as essential to survival as insulin for childhood diabetes — and that the vigorous scientific debate currently underway is merely ‘science denialism’ motivated by ignorance, religious zeal, and transphobia,” the Abbruzzese article says.
“The key problem in pediatric gender medicine is not the lack of research rigor in the past — it is the field’s present-day denial of the profound problems in the existing research, and an unwillingness to engage in [the] high-quality research requisite in evidence-based medicine.”
The article argues that the field of gender medicine is unique in that “a number of clinicians tasked with caring for gender-distressed [youth] have taken on the role of political campaigners — and in doing so, have traded wisdom and nuance for blunt activism.
“When clinician-activists misuse the eminence of their institutions and medical societies to deny or obfuscate important facts about pediatric gender transition — that puberty blockers are prescribed to peri-pubertal children as young as 8–9; that mastectomies are commonly provided to teens; that the wave of detransition is rising and already far exceeds what’s been historically recorded; and that no other pediatric intervention of similarly drastic nature has ever been delivered at scale based [on] such low quality of evidence — they may succeed in scoring a political or legal ‘victory’ in the short-term, but they also contribute to the longer-term erosion of public trust in the medical profession.”
The Abbruzzese critique likens the swift international adoption of the Dutch medical treatments to an “escape from the lab” of a lone clinic’s “small, innovative experiment” — with the potential to do harm.
“ ‘Affirmative treatment’ with hormones and surgery rapidly entered general clinical practice worldwide, without the necessary rigorous clinical research to confirm the hypothesized robust and lasting psychological benefits of the practice,” the article says.
“Nor was it ever demonstrated that the benefits were substantial enough to outweigh the burden of lifelong dependence on medical interventions, infertility and sterility, and various physical health risks.
“The [Dutch] studies also failed to quantify the risk to ‘false positives’ — that is, those gender dysphoric youth whose distress would have remitted with time, without resorting to irreversible medical and surgical intervention.”
The Abbruzzese article identifies an “urgent need to determine whether the benefits of medical interventions outweigh the now much better understood risks.
“The scale of the potential harm can be fully appreciated if one considers that an astounding 1 in 10–20 middle school, high school, and college students in the West currently claim a transgender identity.
“The field of gender medicine has a short time to self-correct before a growing number of authorities step in and impose guardrails to safeguard youth. Public health authorities in Finland, Sweden, and most recently England have already done just.”
The detail
Data deficit
Gender clinics in the U.S. have been presiding over paediatric transition on a scale far larger than the Amsterdam centre seemingly without adding any research data of better quality than the Dutch foundational studies.
Last October, Florida’s Medical Board was told by a leading U.S. gender-affirming clinician — Aron C. Janssen, a child and adolescent psychiatrist at Lurie Children’s Hospital of Chicago — that “the best longitudinal data we have on transgender youth comes primarily out of the Dutch clinic … the Dutch studies in the Dutch model of care. That’s the prevailing model that most of the American clinics have based their care upon.”
Likewise, the 2011 and 2014 Dutch studies authored by child psychiatrist Annelou de Vries and colleagues have been invoked in —
the Endocrine Society’s 2017 statements on hormonal interventions with dysphoric youth
the 7th “Standards of Care” from the World Professional Association for Transgender Health, the 2011 version under which paediatric transition took off
the 2018 “Australian Standards of Care and Treatment Guidelines” from the Royal Children’s Hospital (RCH) Melbourne, which was promoted as the first gender-affirming guideline specifically for children and adolescents. (In a 2019 research protocol, the RCH gender clinic said there was “an urgent need for more evidence to ensure optimal medical and psychosocial interventions”)
Do your own research
Amid growing international concern about gender clinics, Dr de Vries and her clinical psychologist colleague Thomas Steensma have distanced themselves from their overseas imitators.
In May 2021, Dr Steensma criticised gender clinics around the world for “blindly adopting” the Dutch approach to youth gender dysphoria without doing their own independent studies.
“Can our research from so long ago still be applied to the group of young people who are now reporting [to clinics]?,” Dr Steensma asked during an interview with the Dutch newspaper Algemeen Dagblad, which was reporting the controversy over the world’s largest youth gender clinic, the London-based Tavistock centre.
“And why are so many girls suddenly dissatisfied with their sex? That really needs to be investigated.”
In a 2020 commentary for the journal Pediatrics, Dr de Vries warned that the Dutch protocol’s reported psychological benefits were based on treatment being given only to patients whose gender dysphoria had clearly been present “from early childhood on”.
She raised concerns about clinics today giving blockers to teenagers diagnosed with dysphoria after the onset of puberty — patients who might be more likely to regret treatment and “detransition”.
On this theme, the 2023 Abbruzzese article argues that even if the Dutch had demonstrated clear benefits for their small, carefully selected group of patients, there is no guarantee those benefits would accrue to today’s very different patient profile internationally.
Gender dysphoria used to affect a very small proportion of pre-school children, chiefly boys. Most grew out of it, with many emerging as young gay or bisexual adults, according to multiple studies. Now, however, gender clinics are dominated by female teens.
“Around 2014–2015 the presentation of pediatric gender dysphoria in the Western world sharply shifted, from childhood-onset that skewed toward males, to adolescent-onset with a preponderance of females with mental health problems,” the Abbruzzese article says.
“The Dutch researchers began their experiments with pediatric gender transition well before this demographic shift began to dominate clinical presentations of youth gender dysphoria.”
Under direct fire
In the gender clinic debate, the approach of the Amsterdam centre with its screening of patients has often been cited to warn that less careful medicalisation elsewhere — notably in the U.S., but also in other developed nations — is likely to lead to harm and regret.
Now, however, the quality of the Dutch research and its reported positive outcomes are coming under more direct challenge; the Abbruzzese article is the latest example of this trend.
In October last year, the internationally renowned Finnish psychiatrist Riittakerttu Kaltiala cautioned Florida’s Medical Board that benefits reported in the Dutch studies were questionable and the follow-up time was short. (Professor Kaltiala said she regarded more recent research by other groups as “even worse” in quality)
Last September, Oxford University sociologist Michael Biggs published a detailed paper analysing the puberty blocker innovation of the Dutch researchers, noting their failure to use a control group that could have allowed firmer conclusions about the effect of treatment. The Dutch claim that blockers were reversible and useful as a diagnostic tool was “increasingly implausible”, Dr Biggs said
In 2021, a systematic review of the evidence base for puberty blockers undertaken by the UK National Institute for Health and Care Excellence judged that the landmark 2011 Dutch study of de Vries et al was “at high risk of bias” and of “poor quality overall”
Evidence missing
Last July, the Amsterdam clinic’s Dr de Vries took to Twitter to celebrate the award of a Vidi 2021 research grant which she said would “provide the missing evidence base for care.”
She appeared to concede that the foundational studies of her Amsterdam clinic might not apply to today’s fast-growing caseloads of teenage females, for whom data is lacking.
An Amsterdam centre media release reporting the grant said: “Around the world, specialised transgender clinics like Amsterdam UMC's Knowledge and Care Centre for Gender Dysphoria are inundated with adolescent applications [for treatment].
“The prevailing model of care is based on young people who come out from an early age as unhappy with their gender assigned at birth. [In this new, five-year research project] de Vries shifts her focus to other groups that are thought to be out of the picture or barely in the picture at present,” the release said.
“For instance, there are young people who experience deep dissatisfaction with their own sex only after puberty, or only then come out.”
In the release, Dr de Vries said: “One of the questions I want to answer is whether medical treatment makes sense for this group too, or whether it entails too many risks, such as regret afterwards.”
The release quotes Dr de Vries acknowledging the gathering storm of controversy over the Dutch medical approach that begins with puberty blocker drugs.
She said: “There is doubt how young people can make such a drastic decision [to undergo medicalised gender change] at such a young age? There is also the question of whether there is enough knowledge about the long-term effects of drugs that inhibit puberty.”
Doubting the Dutch
The 2023 Abbruzzese article concedes that the foundational Dutch studies were a success in bringing about the physical transformation in secondary sex characteristics desired by young patients.
“What the studies failed to show, however, is that these physical changes resulted in meaningful psychological improvements significant enough to justify the adverse effects of the treatment—including the certainty of sterility,” the article says.
It concludes that the central claim of relief from gender dysphoria is “entirely invalid”, with the apparent improvement being nothing more than an artefact of the scale used to measure this distressing sense of conflict between the body and an opposite-sex identity.
The 2014 landmark study from the Amsterdam centre reported that the score on the Utrecht Gender Dysphoria Scale fell from a near-maximum 54/60 to a low of 16 (the minimum was 12) after the final, surgical stage of the Dutch approach.
“Rather than a true ‘resolution’ of gender dysphoria, however, this spectacular drop was an artifact of switching the scale from ‘female’ to ‘male’ versions (and vice versa) before and after treatment, prompting a problematic reversal in the scoring,” the Abbruzzese article says.
As an example of the problem, the article says it would be “equally nonsensical” to ask a biological male to respond to a statement from the female version of the scale such as “I hate menstruating because it makes me feel like a girl”. The inevitably low score on this question could not be attributed to relief from dysphoria.
The scale had been designed and validated only to measure the level of disconnect with the body leading to a diagnosis of dysphoria, not to assess the change in dysphoria after treatment.
The Abbruzzese article says at least three research groups — including papers with prominent Dutch gender clinicians Dr Steensma and Dr Peggy Cohen-Kettenis among the authors — had noted this “critical threat to the validity of the finding of ‘resolution of gender dysphoria’ due to the switching of the scale.”
In an exchange of commentaries begun last year, Dr de Vries concedes that using the scale after treatment was “less than ideal” but insists that “What the measure shows, the disappearance or resolution of gender dysphoria, is what the gender-affirming treatment is aimed to resolve.”
Best in show
The Abbruzzese article argues that the unusual patient selection of Dutch studies meant they inadvertently reported “only their best-case outcomes” at each stage of treatment from puberty blockers, through opposite-sex hormones to surgery.
For example, to be eligible for the 2011 blockers study, patients had to be judged ready for opposite-sex hormones, for which the minimum age was 16 years.
This skewed the 2011 study towards those with more maturity, and eliminated weaker prospects including more complex cases or those harmed by treatment,the Abbruzzese article says.
“Given that the 2011 Dutch study’s main goal was to evaluate the novel use of puberty blockers for gender dysphoria in a prospective cohort study, the study should have enrolled, and reported the outcomes of all of the intent-to-treat cases based on the date of eligibility to start puberty suppression—not cross-sex hormones,” the article says.
It says the fact that the 2011 Dutch study was run retrospectively, not prospectively as claimed, might explain the failure of the only attempt to replicate it during the 2011-2015 Tavistock clinic’s “early intervention study”.
“We suspect the key reason for this failure was the fact that the UK researchers truly prospectively selected ‘sequentially eligible’ cases for treatment (Carmichael et al 2021) and as a result, ended with a diverse range of outcomes, including worsening of problems among female subjects during puberty blockade,” the Abbruzzese article says.
Dr de Vries says the article betrays a “misunderstanding” of the patient selection process at the Amsterdam clinic.
Video: Analysis, from the 2:45 mark, of the Tavistock’s attempt to replicate the Dutch puberty blockers study; its failure was unknown until full results were published in 2021
Talking therapy v drugs
The Abbruzzese article says the Dutch studies are often misrepresented as delivering “psychological improvements.”
“An examination of the outcomes reveals that standard measures of psychological functioning such as anxiety, depression, anger, and global function showed very little clinically significant change after treatment,” the article says.
Dr de Vries argues that some more robust measures did in fact show clinically relevant changes, and notes that after treatment, the patients “functioned at a similarly high level as their non-dysphoric peers.”
But the Abbruzzese article’s rejoinder is that it is not known how different the Dutch patients were from the general population before treatment.
Dutch research pre-dating the 2011 and 2014 studies had selected patients who turned out to be in the normal range for psychological functioning both before and after treatment.
And even if the Dutch foundational studies did detect a significant boost in psychological functioning, the Abbruzzese article says there was a confounding factor “hiding in plain sight.
“All the subjects received psychotherapy at the same time they were undergoing gender reassignment. This co-mingling of interventions makes it impossible to determine which of the interventions ‘worked’,” the article says.
A 2006 paper on the Dutch protocol by Dr de Vries and colleagues described the role of psychotherapy in terms that today might be denounced by gender-affirming zealots as unethical “conversion therapy.”
“In cases involving confusion about gender feelings, psychotherapy and peer support can be helpful in resolving the confusion and coming to self-acceptance [emphasis added],” the 2006 paper says.
Note: GCN sought comment from Drs de Vries and Steensma.
It’s clear that any account of this treatment approach makes no sense if it is couched purely in medical terms. It seems to be a form of identity politics harnessed to invasive medical technology. Repeated systematic reviews of the medical literature showing the weakness of the evidence base are not a problem for gender-affirmationists because they can switch to subjective “lived experience” as their justification.
Fantastic summary and compilation of many crucial sources (yet again), thank you for your great work on this topic!