Evaluation
Evaluation of family activities should be incorporated into your project’s overall evaluation plan. The following sections provide examples of the types of measures—both formative and summative—to include.
Formative Evaluation
Formative evaluation involves collecting output data such as the number of participants who attended a workshop or the number of packets of information produced and disseminated. It typically applies to universal TA and some types of targeted TA, such as events. This Family TA Tracking Form for Targeted TA can be adapted to track your project’s events and major dissemination activities. Use one tracking form for a single year.
Formative data can be used to check to make sure you are doing what you said you would do, examine patterns in your TA, and inform adjustments to ongoing work. For example, if you find that only families from a couple of regions attend a family learning event, you may want to implement strategies to assist those from other regions to attend in the future.
Summative evaluation primarily refers to changes in family member knowledge or skills, such as improvement in the way a family uses high-quality practices with their child. It involves collecting outcome data and requires that you have identified specific outcomes you want to achieve, a way to measure them, and a process for collecting the data. Outcomes should be practical and written in a way that they can apply to a range of families. This prevents the need to create individualized outcomes for each family and allows you to aggregate data from multiple families. Summative data helps you determine whether you have achieved your proposed outcomes and are being effective as a project.
In addition to making it possible to evaluate the overall effectiveness of your TA, global outcome measures help you determine whether you have the capacity to help an individual family. If the desired outcomes for an individual family don’t align with your global outcomes, it’s unlikely that your current services will meet their needs. It’s hard to say no to any family, but your project will have the most positive impact by creating and sticking to a plan based on your determination of the highest priority needs of families in your state and your project's capacity.
While the ultimate goal of technical assistance is to promote positive outcomes for children, outcome measures for family technical assistance should focus primarily on improving family members’ knowledge and skills. Here are some examples:
- Increased knowledge of deaf-blindness and its impact on learning
- Increased skill using high-quality instructional practices
- Increased knowledge of the IEP process, child's legal rights, and conflict prevention and resolution procedures
- Increased ability to explain and advocate for child's needs within the IEP process
- IEP supports and services reflect child’s needs
- Increased knowledge of service delivery systems that benefit child
- Success in accessing services related to employment, community living, and recreation
- Increased ability to advocate for changes or improvements to service delivery systems
- Increased engagement in project activities (e.g., increase in number of activities attended over a two-year period)
- Increased connections to other family members of children and youth with deaf-blindness to share resources and support
- Increased ability to collaborate with service providers (early intervention, education, or transition) to positively impact child's services
- Increased use of high-quality practices that promote a child's independence and active engagement at home, at school, and in the community
- Increased ability to promote self-advocacy and self-determination for child who is career- or college-bound
- Increased access to paid or unpaid community services (e.g., the Arc, Easter Seals, Lion’s club, religious organizations, Special Olympics ) that benefit child
- Increased knowledge of best practices for transitioning to adult life
- Child’s transition plans reflect family input
Fidelity of TA
It’s also important to periodically (at least annually) conduct a fidelity check on a sample of your family TA activities to evaluate how well you are:
- Adhering to your project’s universal, targeted, and intensive TA activities for families
- Completing all defined family activities for the year
- Successfully implementing individual family TA action plans
- Capturing outcomes from your activities and TA
- Making adjustments, as needed, to accommodate changes in staff time, resources, and ways to measure effectiveness
Example Fidelity Checklist - adapt as needed to match your project’s family TA process.
Additional Evaluation Resources
For information on developing an overall evaluation plan for your project, see:
- Developing an Evaluation Plan in Implementing Evidence-Based Practices for Children Who Are Deaf-Blind: A TA Reference Guide
- TA Evaluation Purposes and Methods