EMERGING DIALOGUES IN ASSESSMENT

March 2, 2018

Natasha Jankowski
Director of the National Institute for Learning Outcomes Assessment

Assessment as a field of practice has well established principles of what "good" assessment entails that have stood the test of time. In 1992, nine principles were put forward via the American Association of Higher Education (AAHE) Assessment Forum, building on prior efforts by campuses to lay out guidelines for the meaningful practice of assessment by synthesizing the work already undertaken, as well as to invite additional statements about responsible and effective assessment efforts. With an illustrious list of authors including Alexander Astin, Trudy Banta, Pat Cross, Peter Ewell, Pat Hutchings, Ted Marchese, Margaret Miller, and others, the Principles of Good Practice for Assessing Student Learning provide a compelling counterpoint to the current narratives swirling around the disillusionment with assessment of higher education as an industrial and bureaucratic enterprise (see Gilbert and Worthen pieces). The principles open with a discussion of assessment that builds upon the ongoing work of faculty looking at student learning within their courses, to move towards examining student learning across courses for a more cumulative picture of how our collective pieces integrate and fit together for our learners. The core framing of the document is that the connection between assessment and improvement not be lost in the accountability discussion, asserting that the importance of the work is on improving student learning. The vision for the principles is coupled with a vision of the purposes of education, one that is as timely and relevant today as it was in 1992:

Implicit in the principles that follow is a vision of education that entails high expectations for all students, active forms of learning, coherent curricula, and effective out-of-class opportunities; to these ends, we need assessment—systematic, usable information about student learning—that helps us fulfill our responsibilities to the students who come to us for an education and to the publics whose trust supports our work (AAHE, 1992).

In addition to the AAHE principles, guidelines for meaningful and intentional engagement with assessment continued to be developed in the form of the Principles and Profiles of Good Practice (Banta, Jones, & Black, 2009), New Leadership Alliance’s Committing to Quality: Guidelines for Assessment and Accountability in Higher Education (2012), the Principles for Effective Assessment of Student Achievement (WSCUC, 2013) which was endorsed by higher education associations and regional accreditors, Guidelines for Judging the Effectiveness of Assessing Student Learning (Braskamp & Engberg, 2014), and five principles of practice informed by research from the National Institute for Learning Outcomes Assessment (2016). All to say, as a field, we have been thinking meaningfully about our work and what drives us as it relates to improving student learning.  

Not surprisingly, none of these principles or guidelines supports or endorses the ‘bane to faculty existence of assessment as administrative burden’ picture painted by Gilbert, Worthen, and others. So why do these tensions exist between administrative burden and teaching and learning approaches to assessment? Why have we yet to fully realize the principles of good assessment practice developed from the field? And how are assessment practices unfolding that provide a way forward to address the tensions and realize more fully our shared principles?

When we looked across our case studies of assessment practice, we found the AAHE principles in use but still under development (Kinzie, Jankowski, & Provezis, 2014). The principles lay out the vision to guide our work, but there are various tensions that provide road blocks to progress – namely a compliance mentality driven by external accountability efforts. In the National Institute for Learning Outcomes Assessment (NILOA) surveys of the field, regional accreditation remains the top driver of assessment efforts (Kuh & Ikenberry, 2009; Kuh, Jankowski, Ikenberry, & Kinzie, 2014; Jankowski, Timmer, Kinzie, & Kuh, 2018). This is not new, in 2009, Peter Ewell wrote about the tensions between external accountability drivers to assessment practices and internal improvement building on a paper addressing the same topic from the 1980s. While it is easy to say, “focus on improvement and you will have what you need to address external accountability,” if the requirements for what counts as evidence for improvement and accountability don’t match or if administrative support is not on the same page regarding why assessment is undertaken, then realizing the connection between the two becomes increasingly difficult.

Elsewhere, I wrote about different mentalities, or lenses, that are brought to bear on our assessment efforts—that of measurement, compliance, or teaching and learning (Jankowski, 2017). The point of that piece was to help tease out the different reasons we engage in assessment work the ways that we do. However, it is important to note that we need a bit of each to have a successful, meaningful assessment process. We can’t ignore issues of measurement or compliance, neither can we ignore teaching and learning. The question is, how do we create a balanced approach to accommodating the different pieces. While that may seem an incredibly difficult task to undertake, there is a lot of work unfolding that provides ways forward.

In 2015, the NILOA senior scholars released a book examining the shift from a compliance driven mentality and organizational structure of assessment to one of meaningful use (Kuh, et al, 2015). To make this shift happen, we need to critically examine and reflect upon several elements: what have we designed and organized our assessment processes to do and how do we communicate about the value and importance of engaging in meaningful assessment practice? Have our approaches fostered a means to generate reports at the expense of engaging in faculty-driven questions about their teaching practices? Has a compliance mentality shifted our focus and thinking away from realizing the AAHE principles of assessment? Yes, we do have reports to complete, and we do have metrics and mandates to respond to—but how often do we educate external audiences on meaningful ways to engage with assessment or the types of evidence that should be collected and discussed? How often do we push back on potentially limiting discussions of comparability? And when high-level administrators come knocking on our door to inform us that the regional accreditation visit occurs in two years so we need to get the documentation ready so we can pass, how do we navigate a dialogue on the impact of framing assessment as solely about external mandates for reporting? In essence, we become the students in our classes asking how long the paper is, how many citations it needs to have, and our focus is upon getting it done and documenting it—losing sight of our core principles and values.

However, there is hope. While there are a lot of compliance-driven approaches to assessment still in operation, we are seeing a paradigm shift unfolding in the way assessment and teaching and learning are viewed as integrated and mutually supportive (Jankowski & Marshall, 2017). Projects are unfolding nationally that focus on embedded assignments through faculty-driven approaches such as the VALUE Institute, and 62% of provosts report they are currently facilitating faculty work on the design of assignments while 77% are currently mapping curriculum to foster more coherent and integrated learning experiences for students (Jankowski, Timmer, Kinzie, & Kuh, 2018). There is growth in organizations such as AALHE that provide space for assessment professionals to engage in reflective dialogue around issues of measurement and meaningful assessment practices, the relationship between equity and assessment is being explored (Montenegro & Jankowski, 2017), and the topic of learning improvement is taking increasing prominence within the field (Fulcher, Good, Coleman, and Smith, 2014).

The Gilbert and Worthen pieces wouldn’t have hit a nerve if we weren’t in the midst of an upheaval of paradigms. What they wrote isn’t a new argument for either of them. A year prior, Worthen wrote about how lecture is better than applied learning and Gilbert had the same complaints about assessment. Yet, a year ago, it didn’t have the same reaction, impact, or collective response from the field. The field is moving, and yes, change is scary and hard. George Kuh likes to say that change moves at the speed of trust, and yes, there are faculty who have lived through the reporting regime with mandated approaches to assess student learning that were quickly developed because an accreditor was coming. They have every right to be skeptical. But as a field and a profession, assessment is finding ways to move forward and navigate the tensions, push back, and realize the principles that inform and guide our practice. It is our job to communicate the differences between the paradigms that are driving our work and remind people of the principles that we have consistently operated under. To engage in a paradigm shift during the time when the usual, or comfortable and accepted ways of doing things are changing, requires clear communication about the value and importance of our work, what it is and isn’t, and what the approaches we support and endorse position us to do and explore. I take heart in the Gilbert and Worthen pieces because I see them as signals that the change is upon us and the dialogues are unfolding.

References

American Association for Higher Education (AAHE). 1992. Nine Principles of Good Practice for Assessing Student Learning. North Kansas City, MO: AAHE.

Banta, T. W., Jones, E. A., and Black, K. E. 2009. Designing Effective Assessment: Principles and Profiles of Good Practice. San Francisco, CA: Jossey-Bass.

Braskamp, L.A, & Engberg, M.E. (2014). Guidelines for judging the effectiveness of assessing student learning. Chicago, IL: Loyola University.

Ewell, P. (2009). Assessment, accountability, and improvement: Revisiting the tension. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

Fulcher, K. H., Good, M. R., Coleman, C. M., & Smith, K. L. (2014, December). A simple model for learning improvement: Weigh pig, feed pig, weigh pig.(Occasional Paper No. 23). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.

Jankowski, N. A. (2017). Moving toward a philosophy of assessment. Assessment Update, 29(3), 10-11.

Jankowski, N. A., & Marshall, D. W. (2017). Degrees that matter: Moving higher education to a learning systems paradigm. Sterling, VA: Stylus Publishing, LLC.

Jankowski, N. A., Timmer, J. D., Kinzie, J., & Kuh, G. D. (2018). Assessment that matters: Trending toward practices that document authentic student learning. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

Kinzie, J., Jankowski, N., & Provezis, S. (2014). Do good assessment practices measure up to the principles of assessment? Assessment Update, 26(3), 1-2, 14-16.

Kuh, G. D., & Ikenberry, S. O. (2009). More than you think, less than we need: Learning outcomes assessment in American higher education. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

Kuh, G. D., Ikenberry, S. O., Jankowski, N. A., Cain, T. R., Ewell, P., Hutchings, P., & Kinzie, K. (2015). Using Evidence of Student Learning to Improve Higher Education. San Francisco, CA: Jossey-Bass.

Kuh, G. D., Jankowski, N., Ikenberry, S. O., & Kinzie, J. (2014). Knowing what students know and can do: The current state of student learning outcomes assessment in US colleges and universities. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

Montenegro, E., & Jankowski, N. A. (2017, January). Equity and assessment: Moving towards culturally responsive assessment. (Occasional Paper No. 29). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

National Institute for Learning Outcomes Assessment. (2016, May). Higher education quality: Why documenting learning matters. Urbana, IL: University of Illinois and Indiana University, Author.

New Leadership Alliance for Student Learning and Accountability. (2012). Committing to quality: Guidelines for Assessment and Accountability in Higher Education. Washington, DC: Author http://www.chea.org/userfiles/PDFs/alliance-committing-to-quality.pdf

Western Association for Schools and Colleges, Senior College and University Commission [WSCUC]. (2013). Principles for Effective Assessment of Student Achievement. Alameda, CA: Author. https://www.wscuc.org/content/principles-effective-assessment-student-achievement