EIC Lessons Learned

Fidelity with Relevant Adaptations

Fidelity with relevant adaptations is essential for achieving desired outcomes, as students cannot benefit from programs they don't receive (Lyon, 2017; Fixsen et al., 2015). Ongoing measurement of how well an innovation is delivered provides crucial information for supporting those delivering it and ensures students receive it as intended. Clearly defining core components and monitoring fidelity through tools like observation protocols and self-assessments ensures consistent delivery across contexts (Hill & Erickson, 2021; Ysseldyke et al., 2003; Fixsen et al., 2015). This also offers a standardized framework for assessing and improving implementation and scaling, applicable across various sectors (Ryan et al., 2024).

Measuring fidelity within the EIC was challenging due to difficulties in identifying or developing reliable and valid measures that could be used consistently. While a lack of fidelity measures for leadership and organizational support is not unique to education, valid, practical, and reliable measures are emerging in education and early learning fields, such as the Illustrative Mathematics reflection tool (IM; 2024). The EIC project addressed this by purveyors and partners using a range of observation tools, both standardized and co-created, to inform professional development and ongoing improvement cycles.

When scaling math practices and curricula, maintaining fidelity ensures that the core components responsible for the intervention's effectiveness are preserved, reducing the risk of "voltage drop," where effectiveness diminishes as programs expand (Fixsen et al., 2005). When interventions are scaled, fidelity measurement serves as a diagnostic tool to assess implementation quality, identify necessary supports, and guide continuous improvement (Fixsen et al., 2005). Moreover, tracking fidelity helps distinguish between implementation failure (the intervention was not delivered properly) and program failure (the intervention is ineffective even when implemented correctly), which is essential for making informed decisions about scaling.

Two critical lessons learned are:

1) the purpose of the fidelity measurement needs to be clarified early. For example, if non-evaluative classroom observations or "learning walks" (Aarons et al., 2011) conducted by a principal served as a fidelity indicator, it could be perceived as being used for individual teacher evaluation—potentially linked to teacher pay—rather than collectively evaluating overall implementation and informing ongoing training and coaching;

2) it is critical to go beyond simple fidelity evaluations. Research also emphasizes the need for adaptive fidelity, wherein interventions are adapted to fit local contexts while retaining their essential features (Century et al., 2010). This balance between adherence and adaptation is essential to facilitate local ownership and long-term sustainability (Metz & Bartley, 2012).

For more information, visit the EIC website.