A retrospective study of autism spectrum disorder (ASD) screening in primary care practices concluded that the practice is fraught with problems. The bottom line is that screening is not happening when and how it should. And even when it does occur, the tools used to screen have their own problems.
This is particularly distressing, given the overwhelming evidence that the earlier children with ASD are diagnosed and enrolled in services, the better the outcome.
Medscape spoke with Kate E. Wallis, MD, MPH, author of an accompanying editorial, co-investigator of a similar study published last year, and a developmental behavioral pediatrician at Children's Hospital of Philadelphia, about the key take-home messages from this research.
You note that the most recent studies of ASD screening using the most widely accepted tool, the Modified Checklist for Autism in Toddlers, Revised (M-CHAT-R), were conducted in "real world" settings, and therefore these disappointing results are more reflective of how screening really works. Can you briefly describe what you mean?
Wallis: When beginning research and developing a screening tool, it's important to test under ideal research circumstances to know whether it works. That typically means that the research is done with funding, which brings with it staff support and other services. In the case of screening tools, that means children who screened positive in the trials that developed those tools have the benefit of accessing free and rapid developmental assessments, which confirm or disconfirm the findings of the screen. That's how accuracy of the screening tool and the likelihood that the screening tool predicts that final diagnostic outcome is measured. But we know that research under those rigorous conditions doesn't always translate to real-world circumstances.
These two studies were done in clinical practice, with real doctors, real nurse practitioners, real families who use this screener as part of their standard practice. Clinicians weren't part of a research study, but were simply going about their daily practice. We were able to use that information — good, bad, and ugly as it may be — to see which children receive a follow-up interview when indicated, without the benefit of research assistants ensuring rigorous follow-up. And we were able to use that information to follow what happens with children who screen both positive and negative, to determine who ultimately was diagnosed with autism.
Unfortunately, both studies found that clinicians aren't really able to do that follow-up interview as reliably as they would have under those ideal research circumstances. And the accuracy of the tool in predicting autism was lower than previous studies, done under those ideal circumstances, had found.
This study found a high rate of false-negative screens with the M-CHAT. So even if a clinician is screening at 18 and 24 months of age using an accepted tool, there is no guarantee they will pick up kids with less significant impairment. Is that a system flaw or a tool flaw?
We don't entirely know. I suspect that it may be a little bit of both. I think parents have a tendency to answer those questionnaires on the basis of what they think others want to hear. They want to paint their child in the best light. It's a natural tendency. So they may underreport symptoms because they don't recognize that there might be something different about their child's development. Remember, these kids are fairly young. That's why it is important to screen at both 18 and 24 months of age. Parents have the benefit of looking back at their answers when their child was 18 months old to see what may have changed over time.
Our study found that there was some improvement in the accuracy of the tool when administered at both points.
This most recent study warned against the conclusion that screening, and the M-CHAT, don't work. Rather, the investigators note that children who were screened and found to be positive did end up with a diagnosis, on average, a year earlier, than children who screened negative or were not screened at all. They point to flaws in the system that prevent these screen-positive children from receiving follow-up diagnostic testing and services. Do you agree?
I absolutely agree with that. It is important to emphasize something that we found in our study, which is that many of the children — almost 4 out of 5 — who screened positive but did not meet criteria for a diagnosis of ASD did have some other developmental diagnosis. Kids who screened positive still need to be referred both for additional evaluation and intervention. Other developmental delays, and speech delays, still warrant attention. Although the M-CHAT screener wasn't developed as a general broadband developmental screening tool, it is important to take those results seriously when the screen identifies developmental risks.
I want to note results from a related study that my team and I conducted last year in follow-up to our screening study. We looked at follow-up after a positive autism screen and found that only 11% of children were referred for a comprehensive ASD evaluation. About one quarter of kids who screen positive were already receiving some type of early intervention services, and another third were referred for intervention. Both of these practices are part of the American Academy of Pediatrics guidelines for evaluation and management of children with ASD. So, even when a child screens positive, there's still a lot of work to be done to make sure those diagnostic and intervention services are accessed.
ASD screening was found, at least in this study, to be less sensitive in kids of color. Why is that?
We've known for a long time that children of color, especially those with milder symptom presentations, are diagnosed at lower rates. They either are missed entirely, or they're diagnosed at later ages. That means that they're not getting access to interventions that have the most impact when begun at a young age.
I think there are many reasons for that. One is that we are still living with a legacy of systemic racism. Historically, only White, middle-class children were identified as having autism, going back to recognition of ASD as a syndrome in the 1940s. The screening tool was developed and normed on a predominantly White middle-class sample, and so we may miss behaviors or aspects that present or are reported differently in non-White children.
Were there other factors that affected the sensitivity of the tool in predicting which children required more intervention?
I don't know that I have the full answer. We know that measures of sensitivity and even positive predictive value are somewhat dependent on the prevalence rate within the population. If there are differences in families access to, for example, diagnostic assessments, that's going to underestimate prevalence.
Families from lower socioeconomic backgrounds tend to have more challenges making it to appointments, and so children may not be screened at both recommended times. Parents accompanied by multiple children may have trouble focusing on the questions and thus not answer them fully or accurately. Parents who may not read English well can have difficulty with a written questionnaire. So in those cases, is it the tool that is the problem? Or is it systems issues making the tool less reliable?
The investigators found that pediatric clinicians, including advanced practice nurses, were more likely to screen for ASD than were family physicians. One of the reasons for that may be that the American Academy of Family Physicians (AAFP), echoing the US Preventive Services Task Force (USPSTF), does not call for universal ASD screening, noting less-than-ideal sensitivity and specificity, and instead calls for developmental surveillance. Is that a reasonable alternative to screening with an accepted tool?
To clarify, the American Academy of Pediatrics (AAP) guidelines published in January 2020 recommend a two-stage approach. In addition to screening with a validated tool at 18 and 24 months, AAP recommends developmental surveillance at all visits, both well-child and sick. That includes eliciting parent concerns, making observations, and documenting developmental milestones. The component of surveillance is important because it provides a backup method for those kids missed by a screening tool.
It's hard to identify autism in a brief encounter. In my field of developmental pediatrics, we spend up to 90 minutes with a new patient. We don't expect primary care clinicians, whether family physicians or pediatricians, to have the time to do that. Even with the time I typically am able to spend, families still feel sometimes that it is not enough to truly capture and understand what their child is going through.
As the USPSTF review of autism screening made clear, the evidence is really insufficient to recommend universal screening at this time. It doesn't mean that it's not effective. It just means that we need more evidence in order to support that recommendation.
I can understand why the AAFP hasn't come down on the side of screening. However, I believe in my heart of hearts, based on a lot of research, that it's important. I think these studies help to point out that screening can help aid in early identification of kids who warrant closer evaluation, because both studies found that children who screened positive were diagnosed with autism at an earlier age, which may allow them to access services earlier as well, when therapies have the most effect.
Obviously, clinicians are not seeing as many kids in their offices right now, and that will probably continue for the foreseeable future. Teachers and early intervention programs are also significantly affected by the pandemic. Many have voiced concerns that these important sentinel reporters may just not be in a position to spot at-risk kids. In addition to the concerns that raises both for identification of developmental disorders as well as intervention, we also know that these kids are inherently at higher risk for abuse. Are these concerns playing out in your practice?
I think there's several parts to this question that are important.
First, how are we going to identify kids if they're not presenting to primary care? We rely on our primary care colleagues to be the first to initiate a developmental evaluation. But unless parents — or others, such as teachers and daycare providers — can bring concerns to the attention of the pediatric provider, we certainly risk missing these kids early.
To compound that, another concern, I fear, is that some of these delays will be chalked up to the child having been out of school and other services. The argument would be that their skills are not as developed because of the lack of intervention, leading to a watch-and-wait period to see whether reinstituting services may lead to advancements.
Families are struggling and are going to have a really tough decision when determining the safest approach for their children returning to school and therapies while COVID-19 is still being spread in the community. Some children with autism and other developmental concerns need more hands-on care, some may be less able to comply with public health guidelines regarding masking and distancing (although many can be taught to do so), and a small portion may have more severe comorbid medical conditions that put them at higher risk for more serious complications from COVID if they were to be infected. They also may have a harder time participating in remote learning.
So families are going to have to come up with an individualized risk-benefit approach. They know their kids need to be in these services and be in school, but they might also have genuine concerns about added risks. We're going to have to take a very individualized approach and be creative about solutions. I know schools will try to provide more intensive therapy for the kids who have those needs, but that will take careful planning and additional funding.
Finally, as you've noted, is worry about the high risk for abuse. Kids are just not coming into contact with the people who would generally be the ones to identify things that might indicate abuse. Calls to abuse hotlines have gone down significantly, but we don't think that that tracks with actual lower cases of abuse. It's just that it's not being identified, which is really scary.
Kate E. Wallis, MD, MPH, is a board-certified developmental and behavioral pediatrician at the Children's Hospital of Philadelphia (CHOP). Her research focuses on the recognition, assessment, and management of children with ASD, with a goal of improving the process of screening to more equitably identify children at a younger age and help them access services as early as possible.
Follow the Children's Hospital of Philadelphia on Twitter
Follow Medscape on Facebook, Twitter, Instagram, and YouTube
© 2020 Children's Hospital of Philadelphia and Medscape
Cite this: Kate E. Wallis. We're Missing Kids With ASD: Strategies to Improve Screening - Medscape - Jul 29, 2020.
Comments