Meta-analysis finds limited benefits from computerized cognitive training

Meta-analysis Finds Limited Benefits from Computerized Cognitive Training

Computerized cognitive training (CCT) uses computers to try to strengthen cognitive skills and processes, reduce ADHD symptoms, and improve executive functioning. Executive functions are cognitive processes and mental skills that help individuals plan, monitor, and successfully execute their goals.

Computerized cognitive training (CCT) uses computers to try to strengthen cognitive skills and processes, reduce ADHD symptoms, and improve executive functioning. Executive functions are cognitive processes and mental skills that help individuals plan, monitor, and successfully execute their goals.

CCT programs target one or more cognitive processes such as motor inhibition, interference inhibition, sustained attention, and working memory. They ramp up task difficulty as performance improves. The goal is to harness the brain’s inherent adaptability (neuroplasticity) to boost performance. 

A European study team that previously probed the efficacy of CCT through meta-analysis had been unable to provide a robust estimate of effect size due to an insufficient number of high-quality trials with probably blinded outcomes. Noting that “there have been a considerable number of new RCTs [randomized controlled trials] published, many with larger samples, well-controlled designs and blinded outcomes,” the team performed an updated systematic review and meta-analysis.

They included RCTs with participants of any age who either had a clinical diagnosis of ADHD or were above cut-off on validated ADHD rating scales. RCTs had to have been peer-reviewed and published in an academic journal, and to have reported a validated outcome measure of ADHD symptoms, neuropsychological processes, and/or academic outcomes.

Fourteen RCTs with a combined total of 631 participants had probably blinded outcomes. Meta-analysis of these studies yielded no significant effect on either overall ADHD symptoms or hyperactivity/impulsivity symptoms. There was a marginally significant reduction in inattention symptoms, but the effect size was small. Between-study variation (heterogeneity) was negligible and there was no sign of publication bias.

Regarding academic outcomes, meta-analyses revealed no gain in arithmetic ability or reading fluency. There was a small but not statistically significant improvement in reading comprehension. Heterogeneity was minimal, with no indication of publication bias.

With two related exceptions, meta-analyses of RCTs measuring executive functions likewise reported no significant improvements in attention, interference inhibition (initial stage in controlling impulsive behavior), motor inhibition (follow-up stage in controlling impulsive behavior), non-verbal reasoning, processing speed, and set shifting (the ability to unconsciously shift attention between one task and another).

The exceptions were for working memory tasks. Meta-analysis of 15 RCTs with a combined 753 participants reported a highly significant small-to-medium effect size improvement in verbal working memory. A separate meta-analysis of nine RCTs with a total of 441 participants similarly reported a highly significant improvement in visuospatial working memory, this time with medium effect size.

The team concluded, “There was no empirical support for the use of CCT as a stand-alone intervention for ADHD symptoms based on the largest and most comprehensive meta-analysis of RCTs conducted to date. Small effects, of likely limited clinical importance, on inattention symptoms were found – but these were limited to the setting in which the intervention was delivered. Robust evidence of small- to-moderate improvements in visual-spatial and verbal STM/WM tasks did not transfer to other domains of executive functions or academic outcomes.”

Daytime Sleepiness, Cognitive Function, and Adult ADHD

What’s the relationship between daytime sleepiness and cognitive functioning in adults with ADHD?

Sleep disorders are one of the most commonly self-reported comorbidities of adults with ADHD, affecting 50 to 70 percent of them.

Sleep disorders are one of the most commonly self-reported comorbidities of adults with ADHD, affecting 50 to 70 percent of them. A team of British researchers set out to see whether this association could be further confirmed with objective sleep measures, using cognitive function tests and electroencephalography (EEG).

Measured as theta/beta ratio, EEG slowing is a widely used indicator in ADHD research. While it occurs normally in non-ADHD adults at the conclusion of a day, during the day it signals excessive sleepiness, whether from obstructive sleep apnea or neurodegenerative and neurodevelopmental disorders. Coffee reverses EEG slowing, as do ADHD stimulant medications.

Study participants were either on stable treatment with ADHD medication (stimulant or non-stimulant medication) or on no medication. Participants had to refrain from taking any stimulant medications for at least 48 hours prior to taking the tests. Persons with IQ below 80 or with recurrent depression or undergoing a depressive episode were excluded.

The team administered a cognitive function test, The Sustained Attention to Response Task (SART). Observers rated on-task sleepiness using videos from the cognitive testing sessions. They wired participants for EEG monitoring.

Observer-rated sleepiness was found to be moderately higher in the ADHD group than in controls. Although sleep quality was slightly lower in the sleepy group than in the ADHD group, and symptom severity slightly greater in the ADHD group than the sleepy group, neither difference was statistically significant, indicating extensive overlap.

Omission errors in the SART were strongly correlated with sleepiness level, and the strength of this correlation was independent of ADHD symptom severity. EEG slowing in all regions of the brain was more than 50 percent higher in the ADHD group than in the control group and was highest in the frontal cortex.

Treating the sleepy group as a third group, EEG slowing was highest for the ADHD group, followed closely by the sleepy group, and more distantly by the neurotypical group. The gaps between the ADHD and sleepy groups on the one hand, and the neurotypical group on the other, were both large and statistically significant, whereas the gap between the ADHD and sleepy groups was not. EEG slowing was both a significant predictor of ADHD and of ADHD symptom severity.

The authors concluded, “These findings indicate that the cognitive performance deficits routinely attributed to ADHD … are largely due to on-task sleepiness and not exclusively due to ADHD symptom severity. … we would like to propose a simple working hypothesis that daytime sleepiness plays a major role in cognitive functioning of adults with ADHD. … As adults with ADHD are more severely sleep deprived compared to neurotypical control subjects and are more vulnerable to sleep deprivation, in various neurocognitive tasks they should manifest larger sleepiness-related reductions in cognitive performance. … One clear testable prediction of the working hypothesis would be that carefully controlling for sleepiness, time of day, and/or individual circadian rhythms would result in a substantial reduction in the neurocognitive deficits in replications of classic ADHD studies.”

March 31, 2022

Cognitive Treatment for ADHD Symtoms May Be Uneffective

Meta-analysis finds no significant effect of cognitive treatment on ADHD symptoms and executive functioning when randomized controlled trials are blinded

A Chinese study team performed a systematic search of peer-reviewed journal literature to identify randomized controlled trials (RCTs) examining the efficacy of cognitive training as a treatment for youths with ADHD.

Seventeen RCTs with a combined total of 1,075 participants met standards for inclusion in a series of meta-analyses. Seven RCTs used waitlist controls, seven used placebo training, two used treatment-as-usual, and one used active knowledge training. Participants were unmediated in four RCTs, with varying proportions of medicated participants in the remaining thirteen.

A meta-analysis of 15 RCTs, with a combined 789 participants, assessed changes in inattention symptoms following treatment, as rated by parents or clinicians. It found a small-to-medium effect size improvement in symptoms of inattention. There was no indication of publication bias, but between-study heterogeneity was very high.

But that gain vanished altogether when combining only the six RCTs that were blinded, meaning the symptom evaluators had no idea which participants had received cognitive treatment and which participants had not. There was zero difference between the treatment and control groups. Significantly, between-study heterogeneity also diminished markedly, becoming low to moderate.

A second meta-analysis, of 15 RCTs with a combined 723 participants, assessed changes in hyperactivity/impulsivity symptoms following treatment, as rated by parents or clinicians. It found no significant difference between participants who received cognitive training and controls. There was no sign of publication bias, and between-study heterogeneity was moderate-to-high.

The three remaining meta-analyses looked for improvements in executive functions, using the Behavior Rating Inventory of Executive Function (BRIEF).

A meta-analysis of 13 RCTs, with a combined 748 participants, found a small-to-medium effect size improvement in the global executive composite index of BRIEF, as evaluated by parents. There was no sign of publication bias, and between-study heterogeneity was moderate-to-high.

But that improvement again disappeared altogether when considering only the five RCTs that were blinded. Between-study heterogeneity also became insignificant.

A meta-analysis of 6 RCTs with 401 participants found no significant improvement in the behavioral regulation index of BRIEF. Heterogeneity was negligible.

Finally, a meta-analysis of 7 RCTs with 463 participants also found no significant improvement in the metacognition index of BRIEF. In this case, between-study heterogeneity was high.

While acknowledging that "when analyses were set in blinded measures, effect sizes were not statistically significant," the author nevertheless concluded, "In summary, multiple cognitive training alleviates the presentation of inattention and improves general executive function behaviors in children with ADHD." This suggests an underlying bias on the part of the study team in favor of treatment even when not supported by best (i.e., blinded) methodological practices.

March 17, 2022

How Effective is Cognitive Training for Preschool Children?

How effective is cognitive training for preschool children?

Further study is needed, but meta-analysis shows small, but not insignificant, effects of pre-school interventions on core executive functions.

A German team of researchers performed a comprehensive search of the medical literature and identified 35randomized controlled trials (RCTs) published in English that explored this question. Participating children were between three and six years old. Children with intellectual disabilities, sensory disabilities, or specific neurological disorders such as epilepsy were excluded.

The total number of participating preschoolers was over three thousand, drawn almost exclusively from the general population, meaning these studies were not specifically evaluating effects on children with ADHD. But given that ADHD results in poorer executive functioning, evidence of the effectiveness of cognitive training would suggest it could help partially reverse such deficits.

RCTs assign participants randomly to a treatment group and a group not receiving treatment but often receiving a placebo. But RCTs themselves vary in risk of bias, depending on:

  • whether the control condition was passive (i.e. waiting list or no treatment) or active/sham (an activity of similar duration and intensity to the treatment condition)
  • whether the outcome was measured by subjective rating (e.g. by questionnaires, susceptible to reporting biases) or more objective neuropsychological testing;
  • whether the assessment of outcome was by blinded assessors unaware of participants' treatment conditions;
  • whether there was a risk of bias from participants dropping out of the trial.

After evaluating the RCTs by these criteria, the team performed a series of meta-analyses.

Combining the 23 RCTs with over 2,000 children that measured working memory, they found that cognitive training led to robust moderate improvements. Looking only at the eleven most rigorously controlled studies strengthened the effect, with moderate-to-large gains.

Twenty-six RCTs with over 2,200 children assessed inhibitory control. When pooled, they indicated a small-to-moderate improvement from cognitive training. Including only the seven most rigorously controlled studies again strengthened the effect, boosting it into the moderate effect zone.

Twelve RCTs with over 1,500 participants tested the effects of cognitive training on flexibility. When combined, they pointed to moderate gains. Looking at only the four well-controlled studies boosted the effect to strong gains. Yet here there was evidence of publication bias, so no firm conclusion can be drawn.

Only four studies with a combined total of 119 preschoolers tested the effects on ADHD ratings. The meta-analysis found a small but non-significant improvement, very likely due to insufficient sampling. As the authors noted, "some findings of the meta-analysis are limited by the insufficient number of eligible studies. Specifically, more studies are needed which use blinded assessments of subjective ratings of ADHD ... symptoms ..."

The authors concluded that their meta-analyses revealed significant, mostly medium-sized effects of the preschool interventions on core EFs [executive functions] in studies showing the low risk of bias."

January 2, 2022

Sex Differences in ADHD Symptoms and Related Cognitive Deficits in Youths

Sex Differences in ADHD Symptoms and Related Cognitive Deficits in Youths

To what extent does sex matter in the expression of ADHD symptoms and associated cognitive deficits among youths with ADHD?

A recently published meta-analysis of 54 studies by a Canadian team of researchers at the University of Quebec at Montreal suggests it makes little to no difference. A meta-analysis of 26 studies with over 5,900 youths found no significant difference in inattention symptoms, and a meta-analysis of 24 studies with over 5,500 youths likewise found no difference in hyperactivity-impulsivity symptoms. Separating out hyperactivity and impulsivity made no difference.

Given these results, it's no surprise that a meta-analysis of 15 studies with over 3,500 youths again found no significant divergence between the sexes for total ADHD symptoms. Parents and teachers differed, however, in their ratings of symptoms. Whereas parents observed no differences, teachers reported boys had slightly more inattention and hyperactive-impulsive behaviors than girls. Turning to cognitive functions, a series of meta-analyses found no significant sex differences for interference control, working memory, and planning scores. But boys performed slightly worse on inhibition and motor response inhibition. While the raw data also showed boys slightly under-performing girls on cognitive flexibility, strong evidence of publication bias made this unreliable.

The team also compared youths with ADHD and youths without ADHD. Both for females and for males, those differences in ADHD symptoms were - as would be expected - extremely large, whether for total symptoms, inattention, or hyperactivity-impulsivity. All cognitive function scores were moderately better for normally developing boys compared with boys with ADHD, and for normally developing girls compared with girls with ADHD. Yet once again, when comparing these effect sizes between girls and boys, there were no significant differences for any of the symptom and cognitive function effects.

"In other words," the authors wrote, "boys and girls with ADHD presented significantly more primary symptoms and executive and attention deficits than did their peers without ADHD, and effect sizes were not significantly different between the sexes." They concluded, "girls with ADHD do not differ from boys with ADHD in many domains of cognitive functioning, and they have significantly more severe difficulties across the executive and attentional functions measured relative to girls without ADHD. This meta-analysis is the first to examine sex differences in cognitive flexibility, working memory, and planning."

December 11, 2023

Immediate and Long-term Effects of Exercise on ADHD Symptoms and Cognition

Immediate and Longer-term Effects of Exercise on ADHD Symptoms and Cognition

A team of Spanish researchers has published a systematic review of 16 studies with a total of 728 participants exploring the effects of physical exercise on children and adolescents with ADHD. Fourteen studies were judged to be of high quality, and two of medium quality.

Seven studies looked at the acute effects of exercise on eight to twelve-year-old youths with ADHD. Acute means that the effects were measured immediately after periods of exercise lasting up to 30 minutes. Five studies used treadmills and two used stationary bicycles, for periods of five to 30 minutes. Three studies "showed a significant increase in the speed of reaction and precision of response after an intervention of 20-30 min, but at moderate intensity (50-75%)." Another study, however, found no improvement in mathematical problem-solving after 25 minutes using a stationary bicycle at low (40-50%) or moderate intensity (65-75%). The three others found improvements in executive functioning, planning, and organization in children after 20- to 30-minute exercise sessions.

Nine studies examined longer-term effects, following regular exercise over many weeks. One reported that twenty consecutive weekly yoga sessions improved attention. Another found that moderate to vigorous physical activity (MVPA) led to improved behavior beginning in the third week, and improved motor, emotional and attentional control, by the end of five weeks. A third study reported that eight weeks of starting the school day with 30 minutes of physical activity led to improvement in Connor's ADHD scores, oppositional scores, and response inhibition. Another study found that twelve weeks of aerobic activity led to declines in bad mood and inattention. Yet another reported that thrice-weekly 45-minute sessions of MVPA over ten weeks improved not only muscle strength and motor skills, but also attention, response inhibition, and information processing.

Two seventy-minute table tennis per week over twelve weeks improved executive functioning and planning, in addition to locomotor and object control skills.

Two studies found a significant increase in brain activity. One involved two hour-long sessions of rowing per week for eight weeks, the other three 90-minute land-based sessions per week for six weeks. Both studies measured higher activation of the right frontal and right temporal lobes in children, and lower theta/alpha ratios in male adolescents.

All 16 studies found positive effects on cognition. Five of the nine longer-term studies found positive effects on behavior. No study found any negative effects. The authors of the review concluded that physical activity "improves executive functions, increases attention, contributes to greater planning capacity and processing speed and working memory, improves the behavior of students with ADHD in the learning context, and consequently improves academic performance." Although the data are limited by a lack of appropriate controls, they suggest that, in addition to the well-known positive effects of physical activity, one may expect to see improvements in ADHD symptoms and associated features, especially for periods of sustained exercise.

July 18, 2021

High Dropout Rate in Six-Year Cohort Study of Medication Treatment for ADHD

High Dropout Rate in Six-Year Cohort Study of Medication Treatment for ADHD

Few studies have examined the safety and tolerability of ADHD medications (stimulants and atomoxetine) extending beyond six months, and none beyond a few years. A pair of Swedish neuroscientists at Uppsala University Hospital set out to explore longer-term outcomes. They conducted a six-year prospective study of 112 adults diagnosed with ADHD who were being treated with ADHD medications (primarily MPH, but also dexamphetamine and atomoxetine).

They found that at the end of that period, roughly half were still on medication, and half had discontinued treatment. There were no significant differences between the two groups in age, sex, ADHD severity, or comorbidity. The average ADHD score for the entire cohort declined to vary significantly, from a mean of 37 to a mean of 26, with less than one in a thousand odds of that being due to chance. There was also no sign of drug tolerance or a need to increase the dosage over time.
All 55 adults who discontinued treatment had taken MPH for at least part of the time. Eleven had also been treated with dexamphetamine(DEX) and 15 with atomoxetine (ATX). The average time on treatment was just under two years. Almost a third quit MPH because they perceived no beneficial effect. Since they were on average taking higher doses at discontinuation than initiation, that is unlikely to have been due to suboptimal dosage. Almost another third was discontinued for various adverse mental effects, including hyperactivity, elation, depressive moods, aggression, insomnia, fatigue, and lethargy. Another one in eleven quit when they lost contact with the prescribing physician. In the case of ATX, almost half quit because of what they perceived as adverse mental effects.

Among the 57 adults who remained on medication, four out of five reported a strong beneficial effect. Only two reported minimal or no effect. Compared with the group that discontinued, the group that remained on medication was far more likely to agree with the statements, "My quality of life has improved," and "My level of functioning has improved." Yet, as the authors caution, it is possible "that the subjects' subjective ratings contained a placebo-related mechanism in those who are compliant with the medication and pursue treatment over time." The authors reported that there were no significant differences in ADHD scores or ADHD severity between the group that quit and the group that remained on medication, even though, on average, the group that quit had been off medication for four years at follow-up.

We cannot explain why the patients who quit treatment showed similar levels of ADHD symptoms to those who continued treatment.  It is possible that some patients remit symptoms over time and do not require sustained treatment.  But we must keep in mind that there was a wide range of outcomes in both groups. Future work needs to find predictors of those who will do well after treatment withdrawal and those who do not.

Any decision on whether to maintain a course of medication should always weigh expected gains against adverse side effects. Short of hard evidence of continuing efficacy beyond two years, adverse events gain in relative importance. With that in mind, it is worth noting that this study reports that among those who remained on MPH, many reported side effects. More than a quarter complained of decreased appetite, one in four of dry mouth, one in five of anxiousness and increased heart rate, one in six of decreased sexual desire, one in nine of depressed mood, and one in eleven of insomnia.

This study breaks important ground in looking at the long-term effects of medication. It reaffirms findings elsewhere of the efficacy of ADHD medications. But contrary to the authors' conclusion, the data they present suggests the possibility that permanently medicating ADHD patients may not be more efficacious than discontinuation beyond a certain point, especially when balanced against adverse side effects.
But this is just one study with a relatively small sample size. This suggests a need for additional studies with larger sample sizes to pursue these questions with greater statistical reliability.

July 8, 2021

What is Sluggish Cognitive Tempo and is it Relevant to ADHD?

What is Sluggish Cognitive Tempo and is it Relevant to ADHD?

Over the past few decades, a consensus has emerged among psychopathologists that some patients exhibit a well-defined syndrome referred to as sluggish cognitive tempo or SCT.

There are no diagnostic criteria for SCT because it has not yet been accepted as a separate disorder by the American Psychiatric Association. People with SCT are slow-moving, indolent, and mentally muddled. They often appear to be lost in thoughts, daydreaming, drowsy or listless. In reviewing these symptoms and the literature, Barkley suggested that SCT be referred to as Concentration Deficit Disorder (CDD). This term is less pejorative, but is not yet commonly used.

Becker and colleagues recently evaluated the internal and external validity of SCT via a meta-analysis of 73 studies. Internal validity addresses the consistency of SCT symptoms as a measure of an underlying construct. Based on factor analytic studies using more than 19,000 participants, the authors concluded that the items purported to measure SCT are sufficiently correlated with one another to justify the idea that they measure the same underlying construct.

Further support for internal validity was found in studies reporting high test-retest and interrater reliability. As regards ADHD, the authors found that SCT correlated significantly with both inattentive (r = 0.72) and hyperactive-impulsive (r = 0.46) symptoms in adults. The greater correlation with inattentive symptoms makes sense given the nature of SCT symptoms. So these data confirm two key points about SCT: 1) it is associated with ADHD symptoms, and 2) it is a meaningful construct in its own right. Very little is known about the implications of SCT for the treatment of ADHD.  

In a naturalistic study of 88 children and adolescents with ADHD, Ludwig and colleagues examined the effect of SCT on the response of ADHD symptoms to methylphenidate. They found no significant differences in treatment response between subjects with and without SCT. McBurnett and colleagues tested the effects of atomoxetine on SCT in children with ADHD and dyslexia (ADHD+D) or dyslexia only. Atomoxetine treatment led to significant reductions in both ADHD symptoms and SCT outcomes.

Because controlling for changes in ADHD symptoms did not predict changes in SCT outcomes, the authors concluded that change in SCT in response to atomoxetine is mostly independent of change in ADHD. Although these data are preliminary and in need of replication, they do provide some guidance for clinicians dealing with ADHD patients who also have SCT.

March 26, 2021