The NYSPCC conducts ongoing evaluations of all programs to ensure consistency, quality, and client satisfaction and to monitor client outcomes. Program-specific evaluation plans are designed using a multi-dimensional, mixed-methods approach. Both process and outcome evaluations are conducted when possible, to monitor consistency of service delivery in addition to client improvement. Data are collected from multiple informants, including parents, children, teachers, and clinicians, using a combination of quantitative and narrative formats. Standardized assessment tools are used when available. Evaluation data is collected at multiple time points throughout service delivery to track client change over time.
Please read on for more detailed information on evaluation methods for each of our programs:
The NYSPCC uses a combination of standardized assessments, client interviews,and clinical observations to measure client outcomes for the Therapeutic Supervised Visitation Program. The Adult Adolescent Parenting Inventory (AAPI) a standardized measure developed by Bavolek and Keene, is administered to visiting parents at the beginning of services and again after the completion of Parent Education (corresponding to the completion of 12 visits). The AAPI assesses parenting strengths and weaknesses in the domains of developmentally appropriate expectations of children, empathy for children’s needs and feelings, beliefs related to the reversal of typical parent and child roles within the family, beliefs related to the value and use of corporal punishment, and oppression of children’s power and independence.
Before each visit, clinicians work collaboratively with visiting parents to determine the parenting skills to be focused on during the visit with their children. The parenting skills reflect six different domains that address a holistic approach to effective parenting. Goals include: re-engagement, safety, expectations of children, communication, structure, and healing. Clinicians rate parents on their independent use of these six domains of parenting skills following the 2nd and 12th visits and at closing. At case closure, interviews are conducted with custodial and visiting parents to assess child support arrangements, reason for case closure, services utilized, and experience of safety during visitation. Follow-up interviews are conducted with custodial and visiting parents three months after case closure to assess the current visitation arrangement.
To assess consistency of service delivery, clinicians complete a rating scale after each visit indicating visit activities and interventions made by the clinician. This form also includes information on parent/child interaction, indicators of parent/child comfort/discomfort during the visit, areas of strength and need for improvement, and a parenting skills checklist. Data are monitored at regular intervals to assess consistency of service delivery and client outcomes.
Learn more about the current outcomes from Therapeutic Supervised Visitation Program.
The NYSPCC uses standardized assessments and clinical observations to measure client outcomes for the Trauma Recovery Program. Clients are administered two sets of standardized assessment measures at intake and again every three months until termination of treatment. The first, developed by John Briere, Ph.D. (Trauma Symptom Checklist for Children; Trauma Symptom Checklist for Young Children; Trauma Symptom Inventory)
, is aimed at identifying acute symptoms and reactions that are directly related to experiencing trauma. The second set of measures, developed by Thomas M. Achenbach, Ph.D. (Child Behavior Checklist; Youth Self-Report; Teacher Report Form)
, assesses the presence of emotional and behavioral symptoms.
In addition to the standardized assessments, The NYSPCC’s Trauma Recovery clinicians work with clients to collaboratively complete a checklist of child symptoms, adaptive coping skills, and caregiver attunement behaviors as a way of monitoring client progress and creating a treatment plan with defined goals. These treatment plans are completed at intake and again every three months until termination of treatment. Data from the standardized assessments and treatment plans are analyzed at regular intervals to monitor program effectiveness. Consistency of treatment delivery is assessed via regular chart reviews. The NYSPCC is currently in the process of developing a quantitative scale for measuring treatment delivery to allow for a more fine-grained assessment of treatment process.
Learn more about the current outcomes from Trauma Recovery Program.
The NYSPCC developed a brief evaluation survey that is completed by all participants in Crisis Debriefing sessions. The survey was designed to elicit participants’ feedback in the following areas based on their experiences in the session: ability to identify their personal stress reactions, degree of safety that they experienced regarding expressing their feelings in the session, future utilization of stress management techniques taught in the session, encouraging fellow staff members to attend a future session, and the helpfulness of The NYSPCC clinicians in addressing their particular stress concerns. Participants also have the opportunity to provide narrative feedback in two sections of the survey.
Learn more about the current outcomes from Crisis Debriefing Program.
Evaluation surveys assessing knowledge gains and client satisfaction are distributed to participants following all training workshops. Survey results are compiled and examined regularly by the Director of the Training Institute to determine possible areas for improvement. Additionally, when feasible, evaluation results are aggregated and returned to the workshop requesters for review.
The NYSPCC is dedicated to ensuring that all clients have a safe, positive experience while receiving services at the agency. In order to gain insight from clients about their experiences, The NYSPCC holds a Client Satisfaction Week twice a year. During this week, clients are encouraged to complete a Client Satisfaction Survey through which they can provide confidential feedback on the extent to which they felt listened to, respected, and safe, as well as whether they felt like they were helped by the services at The NYSPCC. Data from the Client Satisfaction Surveys are compiled and reviewed with all agency staff twice yearly, and improvements are made where possible based on client feedback.
Below are some of the results from The NYSPCC’s Client Satisfaction Surveys completed by clients in 2013:
- 88% felt listened to when in sessions with the clinician
- 80% felt their culture was respected at The NYSPCC
- 73% felt their clinician helped them reach their goals
- 73% would come back to The NYSPCC if the need arose
- 72% felt the agency helped them overall
- 64% felt their child’s safety was a priority
- 62% felt their child’s privacy and confidentiality were protected