CiMH Palette of Measures Evaluation Training: Eyberg Child Behavior Inventory™ (ECBI) Cricket Mitchell, PhD CIMH Evaluation Consultant
Palette of Measures Evaluation: What You Will Need (slide 1 of 2) • Palette of Measures Data Entry Shell v2 (Excel file) – Developed by CiMH and customized for each participating agency – Holds all data for clients served thru the Palette of Measures project – Demographics, service delivery information, pre- and post- outcome measure data
• Palette of Measures Data Dictionary v2 (Word document)
– A guide for using the associated data entry shell – Defines each column in the excel file 2
Palette of Measures Evaluation: What You Will Need (slide 2 of 2) • Outcome measures from the two-pronged approach – General Outcome Measure – Target-Specific Outcome Measure(s) – For example…
• Eyberg Child Behavior Inventory™ (ECBI) – Trade-marked and published by Psychological Assessment Resources, Inc. (PAR) • Must be purchased by interested Palette of Measures project participating agencies
3
Overview of Training • Brief Overview of Palette of Measures evaluation protocol • Eyberg Child Behavior Inventory (ECBI): Target-specific measure when the focus of treatment is disruptive behaviors + Sutter-Eyberg Student Behavior Inventory (SESBI) – Administration – Scoring – Clinical Utility
• Instructions for Palette of Measures data entry and data submissions – Data entry: ECBI & SESBI – Data entry: Demographics & Services – Data submissions to CiMH
4
Brief Overview of Palette of Measures Evaluation Protocol
5
Outcome Assessment • Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning • Pre- and Post- a “dose” of treatment / an intervention interval – General measure of youth mental health functioning (e.g., YOQ/YOQ-SR, CANS, Ohio Scales) – Target-specific measure linked to focus of treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS)
• Providers may choose to administer mid-course assessments as well – e.g., at 3-month intervals in usual care 6
A note about the use of standardized assessment measures… (slide 1 of 2) • Assessment is the beginning of developing a relationship with the child and family – Demonstrates a desire to know what the child and family are experiencing – By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced
7
A note about the use of standardized assessment measures… (slide 2 of 2) • Using standardized assessment measures of functioning… – Assists in initial clinical impressions – Provides valuable information to guide treatment/interventions – Assesses sufficiency of treatment delivered – Demonstrates treatment-related improvements in child functioning 8
Eyberg Child Behavior InventoryTM (ECBI™) and Sutter-Eyberg Student Behavior InventoryTM (SESBI-RTM) 9
CiMH ECBI Training • Information on the administration, scoring, and clinical utility of the ECBI & SESBI was obtained from the measures’ Professional Manual, written by Sheila Eyberg and Donna Pincus, published by Psychological Assessment Resources, Inc. (PAR)
10
ECBI™ and SESBI-R™ Description • Target-specific measure when the treatment focus is disruptive behavior • Measures the current frequency and severity of child disruptive behavior problems – Parent/caregiver report for children ages 2-16 – Teacher report for children ages 2-16
• Available in English and Spanish 11
ECBI™ and SESBI-R™ Description • • • •
Written at a 6th grade reading level 5-10 minutes to complete Valid and reliable Trade-marked and published by Psychological Assessment Resources, Inc. (PAR) – Must be purchased by interested Palette of Measures project participating agencies – After purchase of the manual ($52), the cost per use ~$1.60 – http://www4.parinc.com/Products/Product.aspx?Prod uctID=ECBI
12
ECBI™ and SESBI-R™ Description • ECBI: 36 items • SESBI-R: 38 items • 7-point Likert scale response options – – – – –
Never (1) Seldom (2-3) Sometimes (4) Often (5-6) Always (7)
• Two Scales – Intensity • Frequency with which the child exhibits the behaviors
– Problem • Extent to which the parent or teacher considers the behaviors to be a problem for him/herself
13
Example: Items from the ECBI™ • • • • • • •
Refuses to do chores when asked Argues with parents about rules Has temper tantrums Hits parents Lies Physically fights with friends own age Is overactive or restless 14
Example: Items from the SESBI-R™ • • • • • • •
Teases or provokes other students Does not obey school rules on his/her own Demands teacher attention Acts bossy with other students Sasses teacher Physically fights with other students Has difficulty staying seated 15
ECBI™ and SESBI-R™ Administration • Administer pre- and post- a dose of treatment, or an intervention interval – ECBI completed by mother/mother figure or father/father figure (when available) • Both can be completed for clinical purposes; however, only one should be entered for outcome tracking purposes – Select parent/caregiver who spends most time with client
– SESBI can be completed by a teacher or child care worker • Residential treatment settings can have staff complete this measure 16
Let’s take a look at the ECBI... (not distributed due to copyright laws)
17
ECBI™ and SESBI-R™ Administration • Approximately 5 minutes to complete • Parents and/or teachers complete independently – Written at a 6th grade reading level – Can be read aloud if necessary/desired
• There are no “right” or “wrong” answers; encourage honest responses – Informants can easily be influenced by the attitude of the person administering the scale – Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall
• Ask parents (and teachers) to complete all items 18
ECBI™ and SESBI-R™ Scoring • Calculate the Intensity scale Page 1 subtotal by summing the marked scores and recording this sum in the space provided • Tally the number of Problem responses for which a “Yes” rating was provided, and record this total in the Page 1 subtotal for the Problem scale • Transfer Page 1 subtotals to appropriate row on Page 2 • Calculate Page 2 subtotals in the same manner as Page 1 • Each scale total raw score is obtained by summing Page 1 and Page 2 subtotals
19
ECBI™ and SESBI-R™ Scoring: Missing Data • If more than four items missing from Intensity scale, it is invalid and cannot be scored – With three or fewer items missing, circle 1 (Never) and calculate raw score
• If more than four items missing from Problem scale, it is invalid and cannot be scored – With three or fewer items missing, circle the “No” response and calculate raw score
20
ECBI™ and SESBI-R™ Scoring • ECBI Raw Scores (36 items): – Intensity Scale Raw Scores Range from 36-252 – Problem Scale Raw Scores Range from 0-36
• SESBI-R Raw Scores (38 items): – Intensity Scale Raw Scores Range from 38-266 – Problem Scale Raw Scores Range from 0-38
21
ECBI™ and SESBI-R™ Scoring • Convert Raw scores to T scores – T scores are standardized scores with a mean of 50 and a standard deviation of 10 – Intensity scale T score conversions are in Appendix C (ECBI) and Appendix E (SESBI-R) • Pg 53 and pg 57 of the ECBI/SESBI-R Professional Manual
– Problem scale T score conversions are in Appendix D (ECBI) and Appendix F (SEBI-R) • Pg 55 and pg 59 of the ECBI/SESBI-R Professional Manual
22
Clinical Utility of the ECBI™ and SESBI-R™ • Higher scores indicate a greater level, or greater frequency, of conduct-disordered behavior (Intensity) and a greater impact on the parent or teacher (Problem)
23
Clinical Utility of the ECBI™ and SESBI-R™ • T scores > 60 are clinically significant – ECBI Intensity Raw Score > 131 – ECBI Problem Raw Score > 15 – SESBI-R Intensity Raw Score > 151 – SESBI-R Problem Raw Score > 19
• T scores < 60 are within the normal range 24
Clinical Utility of the ECBI™ • Parenting education workshops and parent training interventions can be helpful even to those who do not score in the clinical range – Behavior problems of “normal” children are often annoying to parents and others – ECBI is sensitive to change even when initial administration is not in clinical range 25
Clinical Utility of the ECBI™ • Discrepancies between scores on the ECBI Intensity and Problem scales can occur; however, it happens relatively infrequently…
26
Clinical Utility of the ECBI™ • …When the Intensity score is low and the Problem score is high, interpret cautiously
– Parent could have a low tolerance for normal misbehaviors of child – Parent may have unrealistically high expectations for child conduct and an authoritarian parenting style – Parent may have limited understanding of child behavior and the kinds of behavior problems that are developmentally appropriate – Parent may be attempting to cope with chaotic and difficult circumstances and be overwhelmed by the stressors of child rearing 27
Clinical Utility of the ECBI™ • …When the Intensity score is high and the Problem score is low – Parent may have a high tolerance for misbehavior – Parent may hold well-intentioned beliefs about permissive parenting that are mismatched to the temperament of his/her child – Parent may be detached from the child either because of the child’s problems or because of characteristics in the parent’s own personality
28
Clinical Utility of the SESBI-R™ • The SESBI-R is a newer measure with much less empirical study than the ECBI – While cutoff scores are provided to estimate the severity of child behavior problems at school, there is a wide variability both within and between geographic locations – Use caution in interpretation, and consider prevailing local customs and norms as needed
29
Clinical Utility of the ECBI™ and SESBI-R™ • Comparisons of pre/post scores reveal specific areas of treatment-related improvement in child disruptive behavior – e.g., • Does the frequency of disruptive child behavior decrease substantially? • Does the extent to which disruptive child behaviors are problematic to parents and/or teachers improve? 30
Palette of Measures Data Entry and Data Submissions
31
Palette of Measures Data Entry Shell
• There is a separate spreadsheet in the excel workbook (aka database) for each type of data: – – – – –
Demographics & Services Pre- General outcome measure Post- General outcome measure Pre- Target-specific measure(s) Post- Target-specific outcome measure(s) • Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants
32
Palette of Measures Data Entry Shell • In addition to the spreadsheets that hold data... – There is an Instructions spreadsheet • Basic data entry instructions • Contact information for T.A. (Cricket Mitchell)
– There is a Data Lists spreadsheet at the end of the workbook that you will not use • Data Lists populate the pull-down menus in other spreadsheets 33
Palette of Measures Data Entry: ECBI TEACHER REPORT (SESBI): IF PARENT REPORT (ECBI): AGES 2-16 AVAILABLE
Pre-ECBI Date of
Intensity Scale Problem Scale Intensity Scale Problem Scale EEEESSSSClient ID# Assessment IntRaw1 IntT1 ProbRaw1 ProbT1 IntRaw1 IntT1 ProbRaw1 ProbT1
•There is a separate spreadsheet for Pre-ECBI&SESBI data, Post-ECBI&SESBI data, and Mid-ECBI&SESBI data •In each spreadsheet, there is a separate field for Intensity and Problem Scales, Raw and T Scores, for each measure •In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields. 34
Palette of Measures Data Entry: ECBI PARENT REPORT (ECBI): AGES 2- TEACHER REPORT (SESBI): IF 16 AVAILABLE
Mid-ECBI Date of
Client ID# Assessment
Assessment Intensity Scale Problem Scale Intensity Scale Problem Scale
Interval
EEEESIntRaw IntT ProbRaw ProbT IntRaw M M M M M
SSSIntT ProbRaw ProbT M M M
•For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-ECBI&SESBI spreadsheet by selecting from the available pull-down menu •e.g., 1st mid-treatment assessment, 2nd mid-treatment assessment 35
But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information...
36
Palette of Measures Data Entry: Demographics & Services (1 of 2) Client Information: Primary (DSM-IV code) (DSM-IV code) Client ID#
DOB
Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID
•Use a unique identifier for Client ID# •Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language) •Dates should be entered as xx/xx/xxxx •Axis I diagnoses s/b the numeric DSM-IV code •Therapist ID is optional 37
Palette of Measures Data Entry: Demographics & Services (2 of 2) Focus 1 of Treatment
Focus1
Date of First Session
Date of Total # Completed Last of Session Sessions Focus 1?
(if Focus 1 not completed)
Completed
(if Services not completed)
Reason
Services?
Reason
•Select Focus from pull-down menu (e.g., anxiety, depression) •The shell will hold data for up to 4 foci, or treatment targets •Enter Date of First Session •The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions) • 2 levels of “Completed?” • Treatment targeting this particular focus • Overall service delivery 38
Palette of Measures Data Submissions (slide 1 of 3) • Data submissions to CiMH will occur twice a year throughout the duration of the project – The end of each May (reflecting all clients served from the initiation of the project through the end of that April) – The end of each December (reflecting all clients served from the initiation of the project through the end of that November) – Note that this is the anticipated schedule; actual data submission dates may vary slightly
• An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline 39
Data Submissions (slide 2 of 3) • Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH: – Use YouSendIt, or another secure web-based transfer site, to submit data electronically • YouSendIt (www.YouSendIt.com) is a vendor that supports the secure transfer of electronic data (encrypted and passwordprotected)
– Mail a password-protected CD to CiMH and submit the password separately (via email or phone) – Email an encrypted, password-protected file(s) to CiMH and submit the password separately (via email or phone)
40
Data Submissions (slide 3 of 3) • After data are submitted, sites continue to enter new data into the same database – Always reflects an ongoing, historical record of clients served through the Palette of Measures project
• Every effort is made to distribute reports within two months of each data submission – Aggregate and site/agency-specific reports
41
Questions
42
The End Contact Information •Cricket Mitchell, PhD •Email:
[email protected] •Cell phone: 858-220-6355
43
