The ZERO TO THREE Court Teams for Maltreated Infants and

Fostering a Permanent Home: A Mixed Methods Evaluation of the ZERO TO THREE Court Teams for Maltreated Infants and Toddlers Initiative

Kimberly L. McCombs-Thornton, PhD University of North Carolina at Chapel Hill December 10 , 2011

Outline • Background and significance • ZTT Court Teams program

• Quantitative study • Qualitative study

• Limitations and Discussion

2

Background and Significance • Infants experience highest rate of victimization; toddlers second – Infants (< age 1): – Toddlers (age 1): – Toddlers (age 2):

20.6 per 1000 11.9 per 1000 11.3 per 1000

• Developmental needs of young children are time sensitive • The caregiver is important for the child’s healthy development • Infants and toddlers in child welfare are at risk for – Attachment disorders – Poor physical health 3

Background and Significance • The Adoption and Safe Families Act (ASFA) of 1997 seeks to expedite the time to reach a permanent, stable home for children in child welfare • A permanent home may include: – family reunification – legal guardianship – or traditional adoption

• More children appear to be reaching permanency since ASFA’s enactment, though the need is still great – Adoptions rose 64.5% in the three years following ASFA’s enactment – Represents only 28% of children eligible for adoption during these years

4

ZTT Court Teams Project TARGET GROUP: CHILDREN ENTERING FOSTER CARE BEFORE AGE 3

LOCAL ACTORS

Judge

Community Coordinator

ACTIVITIES Monthly Case Reviews

LONG-TERM OUTCOMES Decrease in Time to Permanency

Referral to Childfocused Services Child-Parent Psychotherapy

Reduction in Recurrence of Maltreatment

Court Team -DHS case worker -CASA -Attorneys -Providers -Etc.

Evidence-Based Parent Education

Improved Child Well-being

ZTT National Office Activities -Training and TA -Resource materials -M and E 5

Previous Evaluation • JBA completed evaluation in 2009 – Process oriented – 95% of closed cases reached permanency – 55% of those reached permanency within one year

• No comparison group

Problem of causal inference 6

Study Aims Aim 1: Determine the effect of the ZTT Court Teams project on time to permanency. Aim 2: Assess the influence of the ZTT Court Teams program on how children exit the foster care system. Aim 3: Examine successful and unsuccessful cases to understand how program components and/or client characteristics contribute to time to permanency. 7

Causal Inference Q: How do we know outcomes are due to the program? A: Need a comparison group

Basic Pre/post Test

Advanced Secondary Data as Comparison

Matched Comparison Group

Randomized Experiment (w/ control group)

8

Quantitative Data ZTT “Treatment” Group Dataset:

Court Teams MIS

Data Source:

CPS documents, observations in court, etc.

What collected:

Child background, service needs and usage, visitation, placements, case status

Study sample:

All children in first 4 ZTT sites who entered by end of 12/2009

Number in study: 298 children Follow up period: One year or more for 94% of cases 9

Quantitative Data ZTT “Treatment” Group

NSCAW Comparison Group

Dataset:

Court Teams MIS

National Survey of Child and Adolescent Well-Being (NSCAW)

Data Source:

CPS documents, observations in court, etc.

Interview with CPS case worker

What collected:

Child background, service needs Family risks, child living and usage, visitation, environments, services needed placements, case status and received, child behavior, etc.

Study sample:

All children in first 4 ZTT sites who entered by end of 12/2009

Children who entered child welfare supervised out of home placement before the age of 3

Number in study: 298 children

511 children

Follow up period: One year or more for 94% of cases

One year or more for all cases 10

Case Characteristics Child Characteristics Age of child at first out of home placement Mean (months) Infant Child gender Male Race/Ethnicity African American Caucasian Latino/a Parent Characteristics Substance Abuse Severe Mental Health Needs Poverty Lack of Employment

ZTT (n=298)

NSCAW (n=511)

9.3 (1.0) 67% (.04)

11.0 (.99) 57% (.04)

50% (.02)

46% (.04)

37% (.10) 29% (.10) 14% (.10)

24% (.04) 41% (.05) 18% (.05)

72% (.06) 17% (.03) 95% (.02) 35% (.15)

59% (.05) 21% (.04) 81% (.03) 47% (.05)

11

Propensity Score Propensity score = association between covariates and group membership

12

Selection into ZTT Court Teams How child selected within a site: - Child under age of 3 at time of entry into child welfare system - Either all young children assigned to Court Teams judge or randomly assigned

13

Predictors of Time to Permanency

Child Characteristics

Reasons for Removal

Parent Characteristics

Community Characteristics

14

Aim 1 Determine the effect of the ZTT Court Teams project on time to permanency.

15

Aim 1 2 Outcome Measures: – Time to “move in” permanency – Time to “official” permanency

Methods: – Time to event analysis with propensity score weights – Test for proportional hazards – Diagnostics

16

Aim 1 Time to “move in” permanency Permanency Move In

ZTT

NSCAW

(n=298)

(n=511)

Median (SE)

168 (10.5)

210 (25.6)

Mean (SE)

343 (32.1)

398 (23.4)

ZTT cases move into what becomes the permanent home 1.5 to 2 months faster on average

17

Aim 1 Time to “move in” permanency Kaplan Meier Kaplan-Meier survival estimates: Time to Move in Permanency

0.00

0.25

0.50

0.75

1.00

-

0

500

1000

1500

2000

Days until move into what becomes the permanent home

ZTT Court Teams

-

NSCAW

Time to Event Analysis Parametric model (Gompertz): HR = 1.06 (CI .67, 1.65) p=.81 18

Aim 1 Time to “official” permanency Permanency Official

ZTT

NSCAW

(n=298)

(n=511)

Median (SE)

383 (10.8)

762 (26.8)

Mean (SE)

466 (20.5)

825 (18.5)

ZTT cases exit foster care one year earlier on average

19

Aim 1 Time to “official” permanency Kaplan Meier

0.25

0.50

0.75

1.00

Kaplan-Meier survival estimates: Time to Official Permanency

0.00

-

0

500

1000

1500

2000

Days until officially discharged from foster care

ZTT Court Teams

NSCAW

20

Aim 1 Time to “official” permanency - Time to Event Analysis Parametric model (Weibull): HR = 2.67 (CI 1.65, 4.31) p=.000

ZTT cases exit foster care nearly three times as fast as the comparison group

21

Aim 2 Assess the influence of the ZTT Court Teams program on how children exit the foster care system.

22

Aim 2 4 ways to exit foster care: - Reunification

fastest

- Adoption

slowest

- Relative custodianship - Non-relative legal guardian

23

Aim 2 Type of Foster Care Exit

ZTT Court Teams

NSCAW sample

(n=298)

(n=511)

pct

se

pct

se

Reunification

37.6%

(.047)

29.3%

(.042)

Adoption

15.4%

(.059)

40.7%

(.045)

Relative custodian

24.8%

(.085)

8.4%

(.033)

Non-relative guardian

3.0%

(.017)

1.6%

(.007)

Still in foster care at end of study period

19.1%

(.068)

20.1%

(.036)

24

Aim 2 Days until Exit Foster Care, by Type of Exit Type of Exit from Foster Care Reunification

ZTT

NSCAW

(n=298)

(n=511)

Median Mean

309 340

547 587

8 months faster on average

Adoption

Median Mean

464 496

764 800

10 months faster on average

Relative custodian

Median Mean

351 363

450 487

3-4 months faster on average

Non-relative guardian

Median Mean

481 467

878 780

10-13 months faster on average

Patterns hold when use propensity scores in a competing risks analysis

25

Aim 3 Examine successful and unsuccessful cases to understand how program components and/or client characteristics contribute to time to permanency.

26

Aim 3 Examine successful and unsuccessful cases to understand how program components and/or client characteristics contribute to time to permanency.

27

Aim 3 Methods • One-on-one open-ended phone interviews with each community coordinator; 2 interviews each • Approach orientated to unique cases– discuss top and bottom decile in each site (Patton, 1990) • Community coordinator’s perception of how each component of the program model contributed time to permanency for each case • Record and transcribed calls • Coded using Atlas.ti 28

Aim 3 Parents comply with service plan?

Yes

Reunification

“[The mother] is absolutely amazing that, what is she, 20 years old, she had two children with special needs that really require a lot of doctors visits. So she was able to show all the professionals in the system that she was able to follow through with all of these things. And at the same time, she was dealing with her own victimization issues and poor relationship choices…..The children were [ultimately] reunified with the mother.” 29

Aim 3 Parents comply with service plan?

Yes

Reunification

No

Terminate/Surrender/Suspend Parental Rights

Adoption

Legal Guardian

“The mother would come into court every time positive for cocaine, refused to get any help….. She didn’t approach it [the service plan]. She just didn’t do anything….This child was 30 only placed in one foster home… [who] adopted the child.”

Aim 3 Parents comply with service plan?

Yes

Reunification

Somewhat

No

Terminate/Surrender/Suspend Parental Rights

Adoption

Legal Guardian

“An inpatient drug treatment program was recommended for her. She went to one and she completed it. And then when she came out of course they recommended outpatient care and AA meetings and so forth. She started taking those and then slacked off. So that was a discussion in every hearing. Was she or was she not in compliance with the aftercare 31 recommendation….”

Aim 3 Parent attitude/willingness Parent behavior : - substance abuse issues - mental health issues - relationship issues Parent personal resources: - education/job skills - poverty - transportation

Parents comply with service plan?

Yes

Reunification

Somewhat

Judge: - monitor case progress - verbal support when parents comply - verbal warning when parents not complying - timekeeper

Department of Social Services: - design service plan - assistance seeking services - visitation supervision

No

Service providers: - attempt to meet needs, teach skills, stabilize condition

Terminate/Surrender/Suspend Parental Rights

Adoption

Legal Guardian

SOCIAL SUPPORT INFLUENCES Family, friends, foster parents, and others to provide: transportation, housing, visitation supervision, financial support, care for children, emotional support, etc.

32

Aim 3 Role of Judge in Time to Permanency: • Motivate the parents “[The judge] was very supportive and really wanted the children with their mother. And you know basically would encourage her and would actually praise her and tell her she was doing a good job and tell her to keep it up…assuring her we were going in the right direction… [The judge] is very good about praising when you’ve made progress on your service plan and you’re doing what you’re supposed to do.”

“[The judge] became increasingly frustrated… and really confronted the mom on how the professionals were working harder than she was and her inability to take ownership for her part that she played in this. And confronting mom on not understanding the impact it had on the children.” 33

Aim 3 Role of Judge in Time to Permanency: • Motivate social support network • Motivate the case workers • Role as timekeeper

34

Aim 3 Role of Monthly Case Review in Time to Permanency: •

Keeping all on task “Everybody stayed on task because they knew we were gonna be staffing and we were going to be in court. So there was no room for making, for example, making a referral a week before we go to court because we were always going to court.” “Usually CPS cases, the hearings are every 90 days. But with this, the parents know that they have to be in court every month. It gets them motivated to get on the ball so they don’t have to go to the judge in 30 days to explain to the judge why they haven’t done what they are supposed to have done 30 days prior. “

• Monitor parental compliance 35

Aim 3 Fast v. Slow case comparison • Program is consistent across the cases - All have monthly case reviews - Judge has equal contact with cases

36

Aim 3 Activity to Strengthen Results

Reliability Validity

Within a Site Used same questions 5 to 6 times in one interview re: fastest cases



Used same questions 5 to 6 times in a second interview on a different day re: slowest cases



Used same questions for fast and slow cases



Validation interviews – with one other professional in each site



Negative case analysis to search for examples contrary to findings



Across Sites Asked same questions across the sites a total of 46 times



Compared findings to the James Bell Associates Court Teams evaluation



Mixed methods – quantitative analysis



37

Conclusion • The ZTT Court Teams program reduces time to official permanency, but not time to move in permanency • More kids are exiting foster care through reunification and less through adoption • Though – ZTT Court Teams cases are exiting faster regardless of type of exit • The program operates consistently for both “fast” and “slow” cases • The judge and the monthly case review components appear to be most linked to time to permanency 38

Limitations and Next Steps Quantitative study • Propensity score subject to omitted variable bias • Only considers child’s first episode in child welfare

• Assess effect of number of court hearings • Are all types of exits equal? • Next step – cost effectiveness study

39

Limitations and Next Steps Qualitative study • Only one person coded and analyzed data • Sorted by time to move in permanency

• Collected before completed quantitative analysis • Site differences – fidelity to the model?

40

Reflections Judge Constance Cohen Polk County, Iowa

Judge Ernestine Gray Orleans Parish, Louisiana

Judge Michael McPhail Forrest County, Mississippi 41