EMERGING DIALOGUES IN ASSESSMENT

Ho‘oulu: An Equity-Minded Analysis of a Professional Disposition Instrument for Teacher Education

April 8, 2025

• Jamie Simpson Steele, PhD, Director of Assessment, Accreditation, and Accountability, University of Hawai‘i at Mānoa, College of Education
• ‘Alohilani Okamura, PhD, Assistant Professor of World Language, University of Hawai‘i at Mānoa, College of Education
• Jerae Kelly, PhD, Assistant Professor of Special Education, University of Hawai‘i at Mānoa, College of Education

As an institution accredited through AAQEP (Association for Advancing Quality in Educator Preparation), the College of Education (COE) at the University of Hawaiʻi at Mānoa (UHM) is poised to collect data for our self-study and site visit in 2028.  A working group of seven faculty members, including the three authors, set out to improve the quality of our assessments by updating them to align with new frameworks for teacher education while attending to bias mitigation. Early in this process, we committed to developing culturally responsive assessments to reflect a Native Hawaiian Place of Learning (NHPoL) for our college (Native Hawaiian Place of Learning Advancement Office, 2019). Meanwhile, our larger institution was simultaneously developing recommendations for equity-minded assessment for UHM (University of Hawaiʻi at Mānoa, 2024).

The purpose of this article is to engage in equity-minded analysis of one of the COE’s renovated instruments as a case study for our larger assessment system. Ho‘oulu (to grow) is an assessment through which teacher candidates demonstrate actions, behaviors, and values according to the norms of the profession. Professional disposition assessments such as this have the potential to reify norms and practices that reflect entrenched power within society, thereby marginalizing those outside such power structures (Saltis et al., 2020). One year after embarking on this journey, we asked, “How well does Ho’oulu hold up to UHM’s standards for equity?” Below, we examine Ho’oulu: Growing as a Professional against the ten ‘Recommendations’ defined in UHM’s Guidelines for Equity Minded Learning Assessment to center justice in our accreditation practices.

Equity Minded Analysis

Recommendation #1: Transparency and Accessibility 

The COE historically maintains public-facing websites with our assessment instruments and results, which also appear in handbooks for faculty, teacher candidates, and mentors who are practicing PK-12 classroom teachers. This transparent distribution is critical for providing access and understanding to more than 600 teacher candidates in 21 license fields at the undergraduate level,16 at the post-baccalaureate level, 14 at the graduate level, and nine add-a-field options for teachers who already hold a license. Prior to updating communications and links across platforms, we found that Ho‘oulu still needs a rigorous accessibility review, especially around several unique features that we included to support equity. For example, visual images require descriptions, and the continuum of growth needs to be more user-friendly.

Recommendation #2. Clear Explanations and Intentions

We articulated the expectations and purpose behind the instrument with jargon free language. In addition, we included a “Resource Toolkit” with explanations and examples. While the intention of this assessment is to nurture growth through co-intentional reflection, faculty used the prior version to monitor and correct undesired behaviors. We propose that an online training module with further explanations, examples, non-examples, and scenarios would orient both faculty and teacher candidates to the instrument while supporting data reliability.

Recommendation #3: Multiple and Diverse Perspectives

We intentionally recruited faculty from all teacher licensure programs at the COE to serve on the workgroup. Faculty from Special Education, Early Childhood, Elementary, and Secondary teach a variety of content specializations: World Language, Literacy, STEM, Health/PE, and Fine Arts. We surveyed all faculty from all programs before making any changes to the original instrument, then brought proposed revisions to program meetings to garner feedback through discussion. We invited faculty to pilot a version of the instrument then followed up with a focus group to learn about their experiences. One cohort of teacher candidates took a deep dive into identifying specific examples of educators who go above and beyond (Hōpoe) in the profession; however, we have more work to do to elicit candidate perspectives. We plan to conduct focus groups to ensure feedback reflects the rich diversity of our teacher candidate population.

Recommendation #4: Critical Reflection & Bias Mitigation

Over time, the workgroup engaged in critical conversations that addressed positionality and bias. These conversations were not initially planned or structured but evolved as we built trust and respect within the group. We acknowledged the characteristics of the socio-cultural landscapes of the places we are from and how they shaped our values, juxtaposing these perspectives with the diverse student populations, languages, cultures, and histories of Hawai‘i. To engage all teacher preparation faculty in similar reflections, we plan to conduct further professional development: for example, in an upcoming “Assessment Symposium,” facilitators will support our critical conversations by posing difficult questions about who we are as faculty members, who candidates are as future teachers, and who their PK-12 students are in the community we all ultimately serve.

Recommendation #5: Student Learning Outcomes Related to Equity, Diversity, and Inclusion

When surveyed, faculty expressed that our former tool was not “authentic” to the work of teacher education. Competencies within Ho‘oulu now reference the Nā Hopena Aʻo Framework (HĀ) (Hawai‘i State Department of Education, n.d.) and the Charlotte Danielson Framework (CDF), (The Danielson Group, 2025) both currently used in the Hawaiʻi Department of Education for P-12 learners.  We made several significant shifts, such as replacing the domain of “Diversity” with “Aloha ‘Āina” (love of the land and its people). However, focus group participants reported candidates experienced some confusion about the meaning of Aloha ʻĀina, which is not a phrase that stands up to translation but rather a set of complex relationships. Moving forward, we plan to develop a training module to communicate Hawaiian concepts to faculty so they can effectively build these concepts into their curricula.

Recommendation #6: Culturally Responsive Language

Ho‘oulu embraces the challenge to be “reflective of Hawaiian language, history, and culture” (Pukui & Elbert, 1986) for all teacher candidates. The culturally responsive language of the instrument now incorporates ‘ōlelo Hawaiʻi (Hawaiian language) and Hawaiian concepts. For example, the metaphor of the native ʻōhi‘a lehua flower represents the teacher candidates as they develop. This shrub is one of the first to grow after a lava flow and can resist harsh conditions before flowering. To promote shared usage and understanding, the proposed training module will define Hawaiian terms and explore the kaona (layered meanings). As faculty begin to use the language with more agility, they will be modeling engagement with the host culture for teacher candidates.

Recommendation #7: Options for Demonstration of Learning

We conceptualized Ho‘olulu as a dynamic assessment which focuses on evaluating a student's capacity for learning and potential for growth (Diez, 2006). A dynamic assessment process examines how students respond to scaffolding and guidance rather than merely measuring their current performance. This is particularly advantageous for culturally minoritized groups who may be constrained by traditional static assessments due to cultural and linguistic differences (Villegas, 2007). To enhance the dynamic assessment model, we have constructed a protocol tool with question stems that guide discussion and reflection. We have developed a second tool to suggest a variety of modalities for reflection -- written, oral, quantitative, and artistic -- to support candidate engagement and understanding.

Recommendation #8: Investigating Equity Gaps

While we collect data on teacher candidate characteristics and data on program assessments, we have not disaggregated data to investigate possible relationships between the two. In order to be responsive to Native Hawaiian well-being by recognizing areas of strengths and struggles, we are committed to disaggregating this data in an upcoming “Data Party.” This process may also shed light on other underrepresented groups of educators, such as Filipinos and Pacific Islanders. The new approach to data analysis will provide us with the opportunity to make program improvements to support the success of historically marginalized populations.

Recommendation #9: Eliminating Deficit-Based Interpretations

Our original assessment tool did not fully articulate priorities and values that reflected a holistic approach to dispositions. Due to this limitation, faculty often breezed through the checkboxes because they did not value the criteria or used them as the basis for disciplinary action. In contrast, Ho'oulu emphasizes growth --  change over time. To ensure a strengths-based approach, we have developed a discussion protocol to guide meaningful and positive conversations between evaluators and candidates. The intention behind the cyclical dialogue is to highlight diverse generational or cultural norms while focusing on the candidate’s needs (not deficits). The aggregate data that results from Ho’oulu will provide faculty with evidence to examine our practices, suggest innovations, and develop equitable solutions.

Recommendation #10: Assessment to Advance Equity

We view Ho'oulu as a living document that will change in response to the continuously evolving landscape of education. By using Ho‘oulu as a formative, dynamic assessment we can use the results to advance cultural responsiveness, equity, and inclusiveness in teaching. We plan to meet with faculty focus groups regularly to discuss the evidence that will guide our practices, innovations, and solutions.

Conclusion

Applying the UHM equity-minded assessment framework to evaluate Ho‘oulu provided us with a chart of actionable steps for further developing our assessment practices with a commitment to equity. Although we applied institutional recommendations to a single instrument, we see powerful applications across all our shared assessments. In conclusion, we suggest institutions of higher education adopt an equity framework that resonates with their mission, and conduct similar evaluations to articulate opportunities for advancing equity in the field of assessment.