The University of Hawai’i at Mānoa exemplifies how investing in student buy-in to raise response rates and creating innovative tools to inform and engage users enable an institution to get the most out of its National Survey of Student Engagement (NSSE) data. The Mānoa Institutional Research Office (MIRO) shifted from its past supporting role in producing NSSE reports to a proactive role in leading campus efforts to improve the NSSE response rate. Its focus now is on efforts to use NSSE data to support improvements in key areas. As part of MIRO’s follow-up research and the creation of an action plan, a cross-functional team will attend the second annual National Institute for Teaching and Learning, where participating campus teams will develop evidence-based action plans aimed at improving instructional practices, student engagement, and student learning and success. In an attempt to get NSSE data into the hands of those who can use the information to inform decision making, MIRO has reached out to campus units through strategies such as customized reports, online interactive data reporting tools, video tutorials, and face-to-face discussions and training.
For NSSE 2015, MIRO carried out a comprehensive marketing strategy that included several key steps to promote survey participation among first-year and senior students. First, the office coordinated campus-wide advertisements for the survey on dozens of banners and boards as well as hundreds of flyers in first-year and senior residence halls. Second, based on research on the relationship between the use of incentives and increases in response rates, rewards for participants were offered through a drawing for prizes such as an iPad Air 2 and 20 bookstore gift cards. Third, student resources with information about the survey were provided, including a landing page on the MIRO website featuring frequently asked questions such as: “What is the National Survey of Student Engagement (NSSE)?” and “Why should I take part?” The office also organized visits to some of the largest first-year classes to present information about the survey and to encourage participation.
On days when the survey was being administered, information tables were staffed around campus—a service coordinated by student members of the American Marketing Association. With the currency of social exchange as the guiding principle, students were offered snacks and pens with NSSE information notes as they gained awareness about the survey. Prior to the survey administration, MIRO presented its marketing plans to the academic deans, who in turn supported the effort by advertising the survey in their buildings, hosting survey administration parties, doing their own tabling for the survey, and encouraging faculty to promote the survey. Finally, during administration, advertisements were updated to include the end date of the survey period, thereby reminding students to complete the survey before the deadline. To better understand the effectiveness of those promotion strategies, MIRO entered survey response rate data on a daily basis and used the NSSE interface to track changes in response rates.
It was clear that the efforts put forth by MIRO paid off. Compared to the 2011 administration of NSSE at the University of Hawai’i at Mānoa, response rates for NSSE 2015 doubled from 16% to 32%. Closing the loop on this project, MIRO posted an online video showing the steps taken to improve survey participation and the university’s favorable response rate compared with those of other institutions. MIRO also compared NSSE responses with enrollment data to demonstrate that the survey sample adequately represented the overall student population along the characteristics of class standing, gender and race. This final comparison can (a) persuade skeptics of the representativeness of information derived from NSSE and (b) provide strong evidence of the success of campus partners in promoting the survey. These efforts complement other efforts of MIRO to expand access to NSSE data.
MIRO has also created innovative ways to disseminate NSSE findings to different academic units and offices on campus to enhance their capacity for data-based decision making. Outreach efforts include developing interactive data tools to help departments and academic units access NSSE data and conduct data mining in ways that answer specific questions about student engagement. Central to the design and functionality of the web apps that MIRO developed for NSSE data is the ability to “slice and dice” the data based on one or more variables (e.g., gender, race/ethnicity, college, department and many others). The visually appealing report designs enable users to quickly identify data trends. The office also created customized presentations and video reports for student affairs and academic affairs units to focus on three aspects of student engagement: supportive environment, diverse perspectives and student accountability. In addition, MIRO hosted face-to-face training sessions on how to use NSSE data (eight sessions in one semester) and developed virtual tools that include video tutorials, scenarios for use, and follow-up surveys. These tools and data sharing strategies have garnered positive feedback from various offices on campus. By placing data into users’ hands, creative ways of using data to drive decisions have become possible.
To gain a better understanding of one of the areas identified for improvement, MIRO administered a follow-up survey, in July 2016, consisting of five open-ended questions looking at different perspectives of the University of Hawai’i at Mānoa’s supportive environment. Nearly 1,800 students responded, generating nearly 9,000 total responses. MIRO created an interactive online reporting tool allowing decision makers to quickly locate students’ responses from different student populations on specific issues and campus services. These qualitative results provided critical and meaningful information from student voices. To generate real campus change using NSSE results, in August 2016, MIRO’s director led a cross-functional Mānoa team at the National Institute for Teaching and Learning, where they used data from NSSE and the supportive environment survey to develop an action plan to enhance the university’s supportive environment for student success.
All of these efforts to put NSSE data into users’ hands and to link data with program improvements provides the Mānoa community with a better understanding and appreciation of the importance and usefulness of NSSE results. With increased awareness, the University of Hawai’i at Mānoa is likely to enjoy an even more desirable NSSE response rate in the next administration period, which will bring more NSSE data to use for campus decision makers. This healthy and sustainable process works and can be replicated at other institutions.
While recognizing that individual units are in charge of making changes in their educational practice and policy, the institutional research office at the University of Hawai’i at Mānoa serves as an excellent resource in assuring these decisions are informed by NSSE data. Its investment in both the participation and the data use aspects of survey research provides a blueprint for how users can maximize NSSE data to better serve their students.
