EMERGING DIALOGUES IN ASSESSMENT

Introduction

The Lincoln Memorial University’s DeBusk College of Osteopathic Medicine (LMU-DCOM) Office of Academic Assessment (OAA) utilizes student assessment teams (SAT) and student focus groups (SFG) for course evaluation (CE) after attempts to refine the former course evaluation survey process continued to yield ambiguous course data. According to Field (2020), enhanced opportunity for gathering data also brings danger of over-use, and with it, the onset of survey fatigue. The ubiquitous use of course evaluation surveys can impact students’ survey fatigue and data samples suffer from low response rates (Field, 2020). Mary Bolton, Director of Examination Services at LMU-DCOM from 2007-present, said assessment professionals initially assessed courses using a standard post-course electronic survey issued via the learning management system. The evaluation was used for gathering feedback on the course as a whole and the team of faculty and guest speakers teaching the courses. Quantitative Likert data and numerous pages of open-ended responses including not applicable or other less meaningful responses made up the course evaluation reports (M. Bolton, personal communication, March 8, 2022). Faculty struggled to identify meaningful improvements and students had minimal input to identify needed course or teaching improvements. Serving on SATs and SFGs, students gather opinions of their peers, author a professional report including suggestions for improvement, and present report findings to course directors or clinical deans. Using these students led teams and focus groups at LMU-DCOM has improved the evaluation process, but more importantly, has led to improved quality of instruction.

Course Evaluation Change

Since 2008, CE surveys at LMU-DCOM were revised annually, but the revisions did not help the initially low response rates and non-specific data. The OAA found the only way to get students to complete the required CE survey prior to the final course exam was to enforce mandatory survey completion. Even with that mandate and designated time built into the schedule for the evaluation, data remained ineffective with little usable open-ended feedback since students rushed to complete before taking the final course examination. Mandatory enforcement conclusively did not guarantee usable, quality data. 

The OAA and LMU-DCOM began a redesign the CE system in 2012 to include student-centered and student-led assessment teams and focus groups. Assessment professionals piloted use of an assessment team in Essentials of Patient Care, a complex course with lecture, lab, and simulation components, resulting in much more condensed, professional, and useful feedback. The LMU-DOCM Curriculum Committee then voted to implement in all first- and second-year courses beginning in Fall 2013 (M. Bolton, personal communication, March 8, 2022). Students in the SAT group found the team approach seemed to provide more realistic and professional recommendations. Students noted meeting with course directors to present information as opposed to using post-course surveys as a method of venting frustration avoided the possibility of course directors not “hearing” responses (M. Bolton, personal communication, March 8, 2022). 

Student Assessment Teams

Assessment professionals from LMU-DCOM’s OAA randomly assemble student assessment teams in both first- and second-year pre-clerkship student cohorts. Student team members discuss lecture and learning activity experiences, any feedback gathered, and form an assessment report together. Team leaders receive feedback from the OAA regarding content clarity and correct use of report template, then present the finalized course evaluation to course directors and/or senior administration. In transitioning from electronic surveys to assessment teams, course directors and administration find data and feedback are richer and more useful as students vet and condense information from their own experiences as well as those from the larger class. Professionally written, reports elevate observations and offer succinct recommendations based on student’s experiences in the course and with teaching faculty. Upon receiving her first team report, one professor stated (M. Bolton, personal communication, March 8, 2022), “This is the first time I have not needed a bottle of wine and box of Kleenex to read end-of-course feedback!” Course Directors appreciate having condensed, professionally written feedback as a resource for drafting their own individual End-of-Course reports. Faculty have consistently reported to the OAA that they find assessment teams provide concise information; much more useful than reading 40-60 pages of short answers to find usable nuggets of information. They also found round table style meetings with student team leaders a collaborative, culminating exercise.

Student Focus Groups

Each spring, the OAA collaborates with the LMU-DCOM Clinical Education department to assemble focus groups consisting of LMU-DCOM third- and fourth-year student volunteers from across clinical rotation sites. The clinical curriculum is multifaceted, and the focus groups provide the opportunity to gather student perceptions from a wide variety of rotation sites (C. Ruff, personal communication, March 24, 2022). Assessment professionals meet with the focus group to ask course-assessing questions and elicit student opinions of the course. The OAA provides students with a transcription of the focus group session and an Assessment Report template with guiding prompts. Students in the SFG collaborate virtually to draft the assessment report, scheduling addition group discussion meetings as necessary to ensure the reports are all-encompassing and all-inclusionary, then present the report to the Clinical Deans. According to Chloe Ruff, Director of Academic Assessment at LMU-DCOM, the organization of the focus groups changed in the spring of 2021: moving the focus groups to a Zoom setting, bringing a wider variety of students together within the Zoom meeting, and asking students to create a report following the meeting roughly mirroring the pre-clinical Assessment team reports (C. Ruff, personal communication, March 24, 2022). Students collaborate via Google documents or Microsoft Teams to complete a report, receive feedback regarding report clarity and tone from assessment professionals, and present course evaluations to the Clinical Deans and other members of senior administration.