Inspiring Greater Use of Assessment Results
The University of Saint Mary (USM), in Leavenworth, Kansas, participates in various external and internal surveys to gather direct and indirect evidence of educational effectiveness at many levels of the university. About three years ago the campus revamped their assessment efforts, in response to feedback from their accreditor, the Higher Learning Commission (HLC), which encouraged them to make their campus assessment efforts more data-driven. HLC wanted to see more data-based and informed decision making.
In response to the feedback for expanded assessment efforts, USM faculty and staff examined the measures they had in place and solicited further feedback. The faculty unanimously echoed the desire to create a combination of internal and external assessment measures. As such, they realigned their University Learning Outcomes (ULOs) to three assessments. First, each semester faculty report student achievement of ULOs. Second, first-year students and seniors complete a direct assessment of achievement by participating in the CLA+ (CAE’s collegiate level assessment). Third, first-year students and seniors report on their behaviors and time on task through annual participation in NSSE. Combined, the campus is able to look across faculty reports of student learning, students’ performance, and students’ reports about their behavior and engagement. Additionally, for comparisons, the campus can look at national data, usually in percentile ranks.
Upon receiving their 2014 NSSE results, USM distributed copies of their NSSE Snapshot to senior administration, vice presidents, and the campuswide assessment committee. Additionally, a series of presentations focusing on specific NSSE items and groups of students were presented to different interest groups across campus that included the faculty and student life.
NSSE results and other assessment data are also regularly discussed at USM’s semi-annual faculty institute, an all-faculty meeting held at the start of each semester. During one such institute, faculty reflected on the results from the assessment metrics in place. Looking at USM’s results from the CLA+, faculty saw that the students did not perform as well as they would have expected or wanted them to. To dig deeper, they looked at their NSSE 2013 results regarding students’ reports of time on task, specifically, how much time they spent studying and preparing for class and the number of papers they wrote over the course of the year. The faculty were very concerned by their NSSE results. The students reported low amounts of time on task or studying and preparing for class in comparison to their NSSE comparison groups. Additionally, the students reported having written an average of 30 pages over the course of the year, far below the faculty expectation for first-year students and seniors.
In response to these results, the faculty had conversations about how to increase the time students spend studying and preparing for class and the amount of writing they do. Several efforts were also implemented across the curriculum that focused on strengthening students’ critical thinking skills including critiquing and making an argument. For their NSSE 2015 administration, after debating which Topical Module to select—either Learning with Technology or Experiences with Information Literacy—the campus decided to administer Learning with Technology for two reasons. First, the campus was in the process of applying for an external grant relevant to increasing technology in the classroom for students at risk of dropping out, thus, creating the necessity for a baseline measurement of student competency in technology. Second, the questions in the technology module captured more points of interest to current USM initiatives than did the other Topical Modules.
In addition to the NSSE core survey and the technology Topical Module, USM also participates in the Catholic Colleges Consortium, which administers its own customized question set, appended to the core survey. USM has large student involvement in their campus ministry program, and, with the consortium questions, USM is able to see how they compare to other Catholic institutions. To promote campus conversations about outcomes from their first-year experience course and campus ministry program, USM has shared these results with faculty and student life.
