Using NSSE Results to Study “Sophomore Slump” Pace University has participated annually in NSSE since 2002. Results have been shared extensively with the Board of Trustees, Presidents’ Council, and senior administrative councils. The Provost’s office has placed special emphasis on sharing results with faculty and the entire university community. NSSE results have not only been shared, but have been acted on and incorporated into various institutional assessments. Very early on, the Office of Planning, Assessment, and Institutional Research along with the University Assessment Committee teamed up with the Pforzheimer Center for Faculty Development and the Center for Teaching, Learning, and Technology to present Faculty Development Days to review NSSE results. The programs prompted discussion among faculty concerning NSSE Benchmarks of Effective Educational Practice such as Academic Challenge, Active and Collaborative Learning, and Student-Faculty Interaction. Best practices were also shared.
In addition to sharing NSSE data with the various administrative councils, individual deans and department heads requested presentations on the results for their department faculty and staff. Each year, interest in “how we are doing” grows within the institution. Several NSSE items helped assess Pace’s progress in achieving specific goals of its strategic plan in which a special emphasis was placed upon the goal of “student-centeredness.” NSSE items also were easily adapted to the goals and objectives of specific programs and initiatives such as: measurement of progress in service-learning, development of capstone experiences, participation in study abroad, reaffirmation of Pace’s commitment to a diverse learning environment, and increase in positive student self-reports in mastering the learning objectives of the 2003 Core Curriculum. Some specific examples follow that further illustrate how NSSE results have been used by Pace for institutional improvement.
Improving the Sophomore Experience
Pace University had long provided coordinated programs for first-year students to promote their success. These efforts seemed effective as evidenced by a stabilized first-year retention rate of 76–77%, beginning with fall 2000 cohort. However, no special initiatives or programs addressed the needs of students in their sophomore year and there was growing concern over a retention rate that after two years dropped off by more than 9%. Motivated by this persistence data and the success of the first-year experience, the “Sophomore Working Group,” comprised of faculty, academic administrators, and student affairs professionals, began to focus on developing a special program or “experience” for sophomores.
In reviewing NSSE 2004 first-year results, the Working Group sought to better understand areas where Pace was doing well and those that needed improvement in students’ relationships with faculty, other students, administrators, and staff. Schreiner and Pattengale’s (2000) Visible Solutions for Invisible Students: Helping Sophomores Succeed provided the group with additional insights into the sophomore year. They found that the phenomenon of “sophomore slump” corresponded with a number of NSSE questions, so the Working Group incorporated these items into a short survey that was administered to sophomores to assess the extent to which students might be experiencing this phenomenon. Findings from student responses to this survey revealed, for example, that relationships with faculty played a critical role in students’ assessment of their educational experiences and achievements, and that specific bureaucratic procedures for registration, financial aid, and payment of fees were a source of frustration for students.
Sophomore focus groups were also conducted to further contextualize NSSE responses to the Pace environment. Focus group findings were consistent with previous focus groups conducted among a larger sample of the general Pace student population, and indicated sources of students’ satisfaction and their key reasons for attending and remaining at Pace. Specific actions and programs resulted from the findings of the Sophomore Working Group, including the development of comprehensive transition and support programs for sophomores such as the “Pace Plan”, a comprehensive advisement model for both academic and career advisement, and the expansion of faculty mentoring opportunities to increase quality interactions with faculty, and restructuring the registrar, bursar, and financial aid office.
Digging Deeper: Examining Variation Within and Over Time
In an effort to provide usable results to each school or college, Pace conducted a local NSSE administration in 2005. This provided larger samples for each of its schools and resulted in a more insightful profile of their students’ engagement experience. The Office of Planning, Assessment, and Institutional Research used NSSE results to determine whether there were significant differences in the engagement experience of students across schools and colleges.
Pace used NSSE data to carry out additional studies on the experiences of transfer students compared to native students; commuters compared to resident students and first-generation students. In addition, NSSE results have been used by professional schools at Pace in their accreditation efforts with AACSB, ABET, CCNE, and NCATE. Pace also incorporated NSSE results in its Middle States Self-Study in preparation for a spring 2009 reaccreditation visit. Pace also looked over time at two satisfaction questions on NSSE to identify relationships between engagement practices and membership in one of the two extreme satisfaction groups, “Low Satisfaction” defined by “poor or fair” rating and “definite or probable would not repeat experience” versus “High Satisfaction” defined by “good or excellent” rating and “probable or definite would repeat experience.” Although results indicated that the trend in most areas was one of improvement, the percentage of unambiguously satisfied students (i.e., those who found the experience satisfactory and would attend the same institution) hovered steadily between 65% and 70% over a five-year period, compared to Pace’s Carnegie peers who consistently demonstrated higher scores.
In all, 37 engagement activities correlated positively with student satisfaction and perceptions of the Pace experience. The analysis demonstrated that the engagement activity most strongly correlated with student satisfaction was the quality of academic advising. This was followed by “provided the support to help you succeed academically,” “quality of your relationships with faculty members,” “course work contributed to acquiring a broad general education,” and “quality of your relationships with administrative personnel and offices.”
The University Assessment Committee disseminated the findings of the five-year student satisfaction analysis as widely as possible, beginning with the University’s leadership—all members of the President’s Council which included the Vice President of Student Affairs, and all members of the University’s management team. Results from the five-year study and a report highlighting Pace’s NSSE results were also shared with the Board of Trustees. Faculty members were a prime audience for the satisfaction results since many of the activities identified were within their control. Because faculty members are often faced with reports of what is wrong, the Assessment Committee thought it was especially important for them to see what was “right.”
The Assessment Committee published a newsletter reporting on the five-year study of NSSE results that was sent through the provost’s Listserv to all Pace faculty in late March 2007. To follow up, a workshop on the study was presented at the annual Faculty Institute in May.
Results Influence Revision of Freshman Seminar
The influence of the five-year satisfaction results fed directly into an issue getting a great deal of attention and concern at Pace: a proposed revision in the Freshman Seminar, UNV 101. One of the most important changes proposed was to have full-time faculty from each of the schools and the college teach the UNV 101 course. In the past, professional staff and long-time adjunct faculty taught the seminar along with a handful of full-time faculty. The NSSE student satisfaction results provided additional evidence for the associate provost to convince deans and full-time faculty that the assignment of fulltime faculty to UNV 101 would have a significant impact on the first-year experience. As the instructor of the seminar also served as the student’s advisor, a second change extended the advisory role of the faculty member from a one-semester to a year-long relationship with the student. First-year students would be assigned to seminar sections based upon their professional school or college selection. As a result, first-year students would come into early contact with a full-time faculty member from their school or college in a meaningful advisory relationship. With the help of NSSE evidence to strengthen the proposal, fall 2007 UNV101 sections benefitted from the expertise of 57 full-time faculty members.
Results Inform Reorganization of Student Services
The satisfaction study, which identified that “quality of your relationships with administrative personnel and offices” contributed to student satisfaction, and the sophomore survey results that revealed the need for improvement in student services, particularly the Registrar, Bursar and Financial Aid, made a strong case for the creation of “one-stop services.” In 2007, these offices were restructured and renamed the Office of Student Assistance. A new administrator was hired to oversee the operation and a new series of assessments was performed to identify the most pressing problem areas. Pace’s president was keen on using engagement results for improvements and made student satisfaction a high priority. He extended Pace’s commitment to the improvement of service delivery and has supported formal programs to empower Pace staff to take greater responsibility for resolving student problems. Student engagement data provided Pace University leaders with empirical evidence of areas where action and change was needed.
