Good evaluation starts with good planning: Pathway modeling is first step to successful character project measurement
For three days last fall, teams from ECI grantee schools filled four conference rooms with colorful sticky notes and arrows, mapping out how their projects would create changes that form individual and institutional character. They had come together for the ECI’s Program Assessment for Character Education (PACE) Workshop, ready to learn from each other and build something lasting through a process called Relational Systems Evaluation, which includes pathway modeling. Within the first year of modeling their efforts at the workshop, many are already seeing how the groundwork they laid there is helping their projects move forward.
“Prior to the workshop, I was really pushing our team to do a maximal, super-cutting-edge, high-tech approach to evaluation. I thought that good evaluation meant using the newest, shiniest methods,” reflected Chris Napolitano, Associate Professor and Associate Chair for Educational Psychology, and principal investigator for the ECI Institutional Impact Grant at the University of Illinois at Urbana-Champaign. At the time, the team was just starting their initial Capacity-Building Grant — during the workshop, he and his teammates recognized that they were still in the early stages of developing their project’s concept and needed to focus not on the splashiness of the method but on the foundations of their theory of change.
“We realized we needed to start slow, iterate, learn from our participants, and respect that there will be changes to our project as we evolve. This kind of thinking was liberating, and it helped us see the gaps in our thinking and in our model.” The team — whose project promotes the virtue of neighborliness through dozens of new service-learning courses and campus-level initiatives — has since revised the scientific component of their project and believes their work will be considerably stronger in the long run.
“Sometimes people are surprised when they come to one of our workshops and hear us say that we’re not going to even start talking about measurement until day three,” reflected Jennifer Urban, the workshop’s lead facilitator and principal investigator on a 2024 Institutional Impact grant at Montclair State University. “Good evaluation starts with thoughtful program planning. We spend a lot of time having each team build a pathway model for their initiative.”
“Good evaluation starts with thoughtful program planning. We spend a lot of time having each team build a pathway model for their initiative.”
Jennifer Urban, Montclair State University
Urban, professor at Montclair State and co-director of the Institute for Research on Youth Thriving and Evaluation, helped develop Relational Systems Evaluation. This framework brings evaluators and program leaders together as partners. She started this work more than twenty years ago at Cornell, alongside colleagues Thomas Archibald, Jane Buckley, and Monica Hargraves, all of whom joined us as facilitators at the PACE Workshop. Pathway modeling, a key part of this approach, helps teams visually map out how their activities connect to the outcomes they hope to see, both in the short and long term.
“Oftentimes, different people who are working on developing or delivering a program or initiative hold implicit assumptions about what they are doing and why. It’s not until the group is challenged to get that thinking down on paper that gaps in reasoning emerge or disagreements between stakeholders are identified and discussed. Pathway modeling provides a structured way to collaboratively surface assumptions and articulate the visual theory of change for their initiative,” explained Urban.
Pathway modeling helps teams visually map out how their activities connect to the outcomes they hope to see, both in the short and long term.
Carol Moye is the director of assessment and program learning for Howard University’s ECI grants, initially for the school’s Capacity-Building Grant and currently for their newly awarded Institutional Impact Grant. While many experts and specialists across the university would work together toward the project’s goal of implementing a research-informed, HBCU-based framework for character education, Moye was tapped to operationalize the goal and clarify the process for the rest of the team.
By setting up a pathway model, Moye and the team’s research graduate assistant, Tatianna Duperier, clearly identified, organized, and aligned inputs, activities, outputs, and outcomes.
“The visual impact of identifying the intersections throughout was invaluable to how we understood and reimagined the operationalization process,” Moye reflected. “This visual conceptualization highlighted the actions needed from each partner, the timeline for deliverables, and integration of each to bring the goal to fruition.”
The team at Howard has continued to use their pathway model as a starting point and sees it as a living document. As such, Moye noted, “it has allowed the team to have a more focused and aligned conversation around what the operationalization of the ECI grant’s goal should look like and produce.”
Change in character can be difficult to quantify and, consequently, measure. But before practitioners even reach the evaluation stage, they need to be able to identify how they intend their actions to affect students’ character. “Part of what makes measuring character challenging is defining what it is we’re trying to do, and therefore what it is we’re trying to measure,” explained Urban. “Clarifying our character constructs of interest and articulating the stepping stones to achieving them helps get us closer to good measurement strategies.”
Bellarmine University’s Conor Picken had this realization at the workshop. Picken, professor of English and Faculty Director. for the school’s Institutional Impact Grant, said that the pathway modeling process helped his team approach their project more tactically. “We have been able to articulate what we want to do, but that nebulous and unquantifiable piece—how to do it—remained elusive until this exercise. Where the model was most helpful to us was where we broke down into component parts the short-, middle-, and long-term goals of our project,” Picken reflected. “More importantly, pathway modeling challenged us to be concrete about the means by which we would begin the project and, sometime in the near future, achieve our long-term outcomes.”
The team chose to set shorter, more realistic goals. Before the workshop, they planned for each cohort to create a community-engaged course syllabus. After the workshop, they added a new layer: the syllabus would now include a learning objective focused on understanding how character education, community engagement, and dialogue across difference appear in the work of Thomas Merton, a key figure in the university’s mission.
“The modeling exercise laid bare to us that we need our cohorts not just to develop a course but to do so after having spent meaningful time immersed in the work of Bellarmine’s intellectual-spiritual figurehead. A community-engaged course syllabus now reflects something particularly Bellarmine because we have baked in assessable short-term outcomes that measure character, community, and collaboration. Pathway modeling helped us do this,” Picken explained.
The team at Austin Community College (ACC) came to a similar conclusion while building their pathway model. For their 2025 Capacity-Building Grant project, the ACC team developed what is, to our knowledge, the first working model for educating character at a community college. Their new Institutional Impact Grant project is to fully integrate the model, which entails, among other goals, redesigning up to 30 courses across disciplines. “Creating the pathway model reinforced the need to design our framework as an evolving ecosystem with a built-in feedback loop for continuous revision and growth,” reflected Arun John, Professor of Composition and Literary Studies and Associate Dean for Liberal Education Advancement and Development. “The pathway model will help us identify opportunities to evaluate the impact of redesigned courses on supporting academic success while helping students find meaning and connection in their core classes and with developing values and habits-of-mind that contribute to a good life and thriving community.”
Urban hopes the participants gained insight into their initiatives and have a clearer idea of what they want to do and why. “That is what lays the groundwork for clear articulation of character constructs, which is what you need in order to measure character well,” she reflected. “I also hope the workshop energized them and made them feel as though evaluation is not a box that needs to be checked, but it can be something valuable, useful, and a source of insight.”
