The NSCAW: Implications for Child Welfare Evaluations

The NSCAW: Implications for Child Welfare Evaluations


– CHILDREN INVOLVED WITH
THE CHILD WELFARE SYSTEM HAVE DISPROPORTIONATELY
HIGH RATES OF BEHAVIORAL, EMOTIONAL,
AND DEVELOPMENTAL NEEDS. VERY OFTEN, THESE NEEDS
CONTINUE OVER TIME AS A CHILD AGES. FEWER CHILDREN RECEIVE
EFFECTIVE SERVICES THAN THOSE WHO NEED THEM. CHILD WELFARE REFORM
EFFORTS ARE UNDERWAY TO ADDRESS THESE
WELL-BEING NEEDS. TODAY, I’D LIKE
TO SPEAK WITH YOU ABOUT THE NATIONAL SURVEYS OF
CHILD AND ADOLESCENT WELL-BEING, OR NSCAW. THESE SURVEYS ARE NATIONALLY
REPRESENTATIVE STUDIES OF CHILDREN INVOLVED IN
THE CHILD WELFARE SYSTEM AND ARE THE FIRST STUDIES
TO DIRECTLY GET INFORMATION ON SAFETY, PERMANENCY,
AND WELL-BEING DIRECTLY FROM THE CHILDREN
AND FAMILIES THEMSELVES. THESE STUDIES WERE
LANDMARK STUDIES IN THE CHILD WELFARE COMMUNITY AND HAVE DIRECT RELEVANCE
FOR CHILD WELFARE EVALUATIONS. THE FIRST NSCAW STUDY
INCLUDED 6,200 CHILDREN BETWEEN THE AGE OF BIRTH
AND 14 YEARS. THE TWO NSCAW STUDIES
ARE BOTH LONGITUDINAL STUDIES, WHICH MEANS THEY FOLLOW CHILDREN
OVER TIME. THE FIRST NSCAW COHORT INCLUDED CHILDREN REPORTED FOR
MALTREATMENT IN 1999 AND FOLLOWED THAT SAME
GROUP OF CHILDREN FOR A PERIOD OF 5 TO 7 YEARS. THE SECOND COHORT OF NSCAW
INCLUDED SLIGHTLY LESS THAN 6,000 CHILDREN BETWEEN BIRTH
AND 17 YEARS OF AGE. THIS STUDY FOLLOWED
THAT SECOND COHORT OF CHILDREN FOR 3 YEARS, AND THAT STUDY WAS
INITIATED IN 2008. THE NATIONAL SURVEY OF CHILD
AND ADOLESCENT WELL-BEING PUTS A LARGE EMPHASIS
ON THE MEASUREMENT OF VARIOUS
WELL-BEING CONSTRUCTS, INCLUDING BEHAVIORAL EMOTIONAL
HEALTH ISSUES. IN THIS GRAPH, YOU SEE RATES OF
BEHAVIORAL HEALTH PROBLEMS FOR CHILDREN AT THE POINT OF
AN INDEX MALTREATMENT REPORT AND THEN 18 MONTHS LATER
AND 36 MONTHS LATER. THE BLUE LINE REPRESENTS
PERCENTAGE OF BEHAVIORAL HEALTH PROBLEMS, THE ORANGE LINE IS DEPRESSION, AND THE GREEN LINE
POST TRAUMATIC STRESS SYMPTOMS. ONE OF THE THINGS THAT
YOU NOTICE IN THIS GRAPH IS THAT THE DEGREE AND THE
PREVALENCE OF THOSE BEHAVIORS GO DOWN AFTER 18 MONTHS
AND 36 MONTHS. WE KNOW THAT THE TIME
OF THE MALTREATMENT REPORT IN THE LIVES OF
CHILDREN AND FAMILIES IS A VERY STRESSFUL EVENT. SO, IT’S NOT SURPRISING
THAT RATES OF BEHAVIORAL HEALTH ISSUES
MIGHT BE ESPECIALLY HIGH AT THE TIME OF THAT
MALTREATMENT REPORT, BUT THE RATES OF THOSE THINGS
MAY GO DOWN OVER TIME THE FURTHER YOU GET AWAY
FROM THAT EVENT. IF A CHILD’S
WELFARE INTERVENTION WAS ATTEMPTING TO ADDRESS
ISSUES SUCH AS DEPRESSION OR BEHAVIORAL HEALTH PROBLEMS AND DID NOT INCLUDE
A COMPARISON GROUP, AN EVALUATOR MIGHT
WRONGLY CONCLUDE BETWEEN THE TIME OF THE INDEX
MALTREATMENT REPORT AND 36 MONTHS LATER THAT THEIR INTERVENTION HAD LED TO THAT IMPROVEMENT
IN BEHAVIORAL HEALTH SYMPTOMS WHEN IN FACT WHAT
THEY WERE SEEING WAS THE NATURAL CHANGE
IN BEHAVIORS OVER TIME. IT’S ESPECIALLY IMPORTANT
TO MEASURE WELL-BEING OUTCOMES BOTH IN THE POPULATION
THAT YOU ARE ADDRESSING WITH AN INNOVATION OR
A NEW INTERVENTION AS WELL AS
A COMPARABLE POPULATION NOT RECEIVING THAT NOVEL IDEA
OR INTERVENTION. WE ALSO NEED TO THINK
VERY CAREFULLY ABOUT THE INSTRUMENTS
THAT WE CHOOSE AS OUTCOME EVALUATION TOOLS. MEASURING A CHILD’S
STRENGTHS AND NEEDS IS A CRITICAL COMPONENT IN
THE SERVICE DELIVERY PROCESS. THAT’S HOW CASEWORKERS IDENTIFY
WHAT A PARTICULAR CHILD NEEDS. BUT UNFORTUNATELY, THOSE
INSTRUMENTS THAT WE USE TO ASSESS STRENGTHS AND NEEDS MAY NOT BE THE BEST TOOLS FOR USE IN
AN OUTCOME EVALUATION, AND THIS IS FOR
A COUPLE OF REASONS. FIRST, A LOT OF TIMES,
CLINICAL NEEDS ASSESSMENT TOOLS ASK QUESTIONS DIRECTLY
OF A CASEWORKER. IN NSCAW, WE FIND
THAT CASEWORKERS OFTEN REPORT LESS NEEDS,
FEWER NEEDS THAN WHEN YOU ASK
A PARENT DIRECTLY OR A CAREGIVER DIRECTLY, OR,
EVEN BETTER, A YOUTH THEMSELVES. A SECOND ISSUE WITH
CLINICAL ASSESSMENT TOOLS IS VERY OFTEN THEY’RE
ASKING CASEWORKERS TO ADDRESS NEEDS IN
A YES OR A NO FASHION. DOES A PARTICULAR CHILD
HAVE A DEVELOPMENTAL DELAY? YES OR NO? IN THE CONTEXT OF
AN OUTCOME EVALUATION, IT’S VERY RARE TO SEE
THAT A CHILD GO FROM THE POINT BEFORE
AN INTERVENTION STARTS AT HAVING A PROBLEM AND THEN MOVING AFTER
A 6-MONTH INTERVENTION OR A ONE-YEAR INTERVENTION
TO CLEARLY NO, NOT HAVING A PROBLEM. WHAT IS MUCH MORE COMMON
IS TO SEE VARIATIONS IN THE DEGREE OR SEVERITY
OF THOSE ISSUES OVER TIME. TO TRULY CAPTURE THAT VARIATION
AND BE POSITIONED TO STUDY CHANGES OVER TIME, AN EVALUATOR NEEDS TO CHOOSE AN INSTRUMENT THAT HAS
A VARIATION IN SCORES. OFTEN, THAT MEANS
THE CHOICE AND SELECTION OF A STANDARDIZED TOOL. NSCAW INCLUDED SEVERAL
STANDARDIZED ASSESSMENTS OF CHILD AND ADOLESCENT
WELL-BEING. BY STANDARDIZED MEASURE,
I MEAN MEASURES THAT WERE DESIGNED
WITH NORMATIVE DATA IN MIND. NORMATIVE DATA INDICATES
HOW WOULD A TYPICAL U.S. CHILD OF THAT AGE PERFORM
ON THIS INSTRUMENT. SO, IN THIS TABLE, YOU SEE
A HANDFUL OF INSTRUMENTS THAT NSCAW USED
TO ASSESS WELL-BEING. ON THE FIRST ROW OF THIS TABLE, YOU SEE THE INSTRUMENT
THAT NSCAW USED TO ASSESS EARLY LANGUAGE SKILLS AMONG VERY YOUNG
INFANTS AND TODDLERS– THE PRESCHOOL LANGUAGE SCALE-3. IN NSCAW, APPROXIMATELY
18% OF PRESCHOOLERS SCORED IN THE CLINICAL RANGE,
INDICATING A NEED FOR SERVICES ON THE PRESCHOOL LANGUAGE SCALE. MEANWHILE, ACCORDING TO THE
NORMATIVE DATA FROM THAT SCALE, WE WOULD EXPECT THAT
APPROXIMATELY 2% OF INFANTS AND TODDLERS
WOULD SHOW ABOUT THE SAME LEVEL OF NEED. NOW, LOOKING AT THE REST
OF THE ROWS IN THIS TABLE, WE COULD MAKE THOSE SAME TYPES
OF COMPARISONS ON THE STANDARDIZED
ASSESSMENT TOOL THAT ASSESSED INTELLIGENCE OR TRAUMA
OR BEHAVIOR. SOMETIMES, WE’RE
NOT ONLY INTERESTED IN COMPARING
THE CHILDREN INVOLVED IN OUR CHILD WELFARE SYSTEM TO CHILDREN ACROSS
THE ENTIRE COUNTRY. WE KNOW THAT OUR
CHILDREN ARE DIFFERENT. WHAT MAY BE MORE RELEVANT
IS COMPARING THE CHILDREN IN OUR SYSTEM
TO CHILDREN SERVED BY OTHER LOCATIONS
OR OTHER STATES. HERE, AGAIN, COMES
THE RELEVANCE OF NSCAW. THE STATE OF ILLINOIS INCLUDED SEVERAL MEASURES
COMPARABLE TO NSCAW IN THEIR STATE DATA COLLECTION. SO, THIS ALLOWS THEM TO MAKE EXACTLY THOSE TYPES
OF COMPARISONS. LET’S LOOK AT A HEALTH OUTCOME–
CHILDHOOD OBESITY. IN THIS GRAPH, YOU SEE
RATES OF CHILDHOOD OBESITY FOR CHILDREN IN ILLINOIS REPORTED FOR MALTREATMENT
IN BLUE, NSCAW CHILDREN IN RED,
AND WHAT YOU WOULD EXPECT FOR U.S. RATES OF
CHILDHOOD OBESITY IN GREEN. WHAT WE CAN SEE
FROM THIS GRAPH IS THAT CHILDHOOD OBESITY
AMONG CHILDREN INVOLVED WITH THE CHILD WELFARE
SYSTEM IN ILLINOIS ARE HIGHER THAN WHAT WE WOULD
EXPECT FOR CHILDREN NATIONALLY, BUT RATES OF CHILDHOOD OBESITY
FOR CHILDREN INVOLVED IN THE SYSTEM IN ILLINOIS
ARE ACTUALLY LOWER THAN WHAT NSCAW DEMONSTRATES
FOR CHILDREN INVOLVED IN THE CHILD WELFARE
SYSTEM NATIONALLY. SO, BY MEASURING
CHILDHOOD OBESITY IN THE STATE OF ILLINOIS
IN A WAY COMPARABLE TO NSCAW, ILLINOIS IS ACTUALLY
ABLE TO UNDERSTAND THEIR CHILDREN’S OUTCOMES
COMPARED TO A COUPLE OF DIFFERENT TYPES OF CHILDREN– THOSE IN THE CHILD WELFARE
SYSTEM AND THOSE NATIONALLY. MEASURING AND CONNECTING
SAFETY, PERMANENCY, AND WELL-BEING OUTCOMES
ARE COMPLICATED, BUT NSCAW OFFERS A NATIONALLY
REPRESENTATIVE PLATFORM TO DESCRIBE OUTCOMES
FOR THE CHILDREN SERVED BY THE CHILD
WELFARE SYSTEM. THIS OFFERS
A NATURAL COMPARATOR, A SUMMARY OF THE STATUS QUO FOR CHILDREN SERVED BY
THE CHILD WELFARE SYSTEM. HOW DO OUTCOMES LOOK
FOR THE SYSTEM AS IT’S CURRENTLY OPERATING? NSCAW, BY VIRTUE OF COLLECTING OUTCOMES RELATED TO CHILD
AND FAMILY WELL-BEING, ALSO REFLECT POSSIBLE POINTS
FOR INTERVENTION TARGETS. AND FINALLY, BY VIRTUE OF
THE RICH CADRE OF INSTRUMENTS INVOLVED IN THE NSCAW SURVEY, THIS OFFERS TO
THE CHILD WELFARE COMMUNITY AN EVALUATOR’S CHOICES OF
POTENTIAL COMPARABLE MEASURES THAT THEY COULD INTEGRATE
INTO THEIR LOCAL EVALUATIONS. AS YOU’RE MAKING CHOICES ABOUT A CHILD WELFARE
EVALUATION DESIGN OR THE MEASURES
THAT MIGHT BE USED IN AN IMPLEMENTATION EFFORT, I HOPE YOU’LL CONSIDER
LESSONS LEARNED FROM THE NATIONAL SURVEY OF
CHILD AND ADOLESCENT WELL-BEING.

Leave a Reply

Your email address will not be published. Required fields are marked *