Building a Culture of Curiosity: Integrating Chat GPT into Assessment, Teaching, and Learning at Delaware State University
By: Alethia Russell, NSSE Institute Project Associate
Artificial Intelligence (AI) tools, such as ChatGPT and Gemini, have entered the research scene in recent years, eliciting a mixed array of reactions. Some are negative, with concerns about disrupting or hindering students’ learning, while others are excited about how it can enhance learning opportunities.
Like many institutional research offices that regularly participate in NSSE administrations, Delaware State University (DSU) found itself with a wealth of qualitative data waiting for analysis, but with limited time and personnel to complete the task.
“We have a lot of the data collected, and we can’t get much done with it except the main reports. We have high-level benchmark reports that we share with faculty and staff. But really, all of the qualitative data sometimes is useful and helpful, and I didn’t realize the value of it until I did my dissertation, and it was on qualitative data,” said Dr. Bina Daniel, Director of Assessment for Delaware State University.
Daniel, who has been working for DSU for over 20 years, initially took an interest in the responses to the final item on the NSSE’s core survey, a customizable open-ended prompt. In the prompt, institutions can select one of four pre-written questions or write their own.
“You know how that last question is: if you were to start again at this institution, would you choose DSU? And many times, the responses were really low at certain points, and then eventually they started improving a little bit as we improved our administrative services,” Daniel said. “I wanted to know a little bit of answers to why, without actually doing an entire focus group on all those students who responded. Since we had the qualitative data, I thought it [Chat GPT] would give some ideas and clues about it, and it did.”
Daniel completed a manual thematic analysis in Excel, familiarized herself with the data, and then tested a small batch of de-identified open-ended comments in Chat GPT to get started.
“I was like, okay, let me just try it. So, I tried it, just a little batch of it. And then it seemed to give similar results as I would evaluate it.” She said. “Of course, it’s not the only reliable thing to use. I think it’s still important for you to look at your data, become familiar with it, analyze it yourself, review it, and have some themes that you code already with Excel. But it did a pretty good job,” Daniel said.
Its usability in the classroom and in DSU’s assessment office became even better as team members learned the interface and became more proficient at prompting.
“At first, I was just like, ‘Do qualitative analysis on this data.’ And then it just kind of gave me the answers. But then I even said that ‘create themes’, ‘suitable themes ’, ‘overarching themes’, ‘key themes’, and then, under those themes, put all of the quotation marks of what the students have responded under those themes. So that was really helpful. That piece, adding the extra prompts in there, gives me much better responses, and it corroborated with what students were coming up with, and they found it very valuable, too, so they could use something like that for their dissertation research.”
Since then, she has found opportunities to build a culture of curiosity about how ChatGPT can be used in teaching, learning, and assessment workflows. She has used Chat GPT to corroborate other survey data, worked individual faculty members, and even incorporated Chat GPT into qualitative mixed-methods courses for doctoral-level students.
All in all, whether for immediate research insights in a campus-wide survey or to assist with member checking qualitative analysis for larger projects, Daniel has found ways to build a culture of curiosity around integrating AI tools into teaching and assessment work using NSSE Data.
Daniel offered a few high-level learnings for institutions looking to incorporate Chat GPT into their assessment work:
1. Put Privacy First
Always de-identify data before inputting it into AI platforms. Start by securing appropriate paid licenses and subscriptions to AI platforms. Clean identifiable information from open-ended comments, such as faculty names or staff names, and exclude identifiable information, including Student IDs, from any part of the data file.
2. Foster mutually beneficial relationships to mitigate bottlenecks in the assessment process.
One example is partnering with faculty to utilize students in a mutually beneficial way.
“I think it’s very easy to incorporate and get our students to partner with us,” Daniel said. “If you have graduate student assistance, if you have classes that you’re teaching, where can you implement this? Because one of the faculty members was also teaching a data analysis class. He asked me that. ‘Oh, do you have any workshop survey data that our students can practice on?’ And it would benefit both people. It would benefit me in getting my data analyzed, and then they would have real, live data to work with. It would give them practical experience of how the survey was written. They can give me feedback. Were these questions helpful? Were these questions not so easy to analyze? It works both ways. It gives me an objective view on my survey data as well.”
3. Consult with university officials for clarity on your university’s policies on ChatGPT utilization. Institutions may have differing policies across departments or different norms for students and staff usage.
4. Build your toolkit and share the knowledge: Working across departments and empowering other faculty and students to become comfortable with AI tools and use them responsibly is key.
5. Trust but verify. In Daniel’s experience ChatGPT analyzes open-ended comments best when there are 500 or less at a time. As you take the time to de-identify the data, make sure you are familiar enough with the data that you can make a sound judgement call on ChatGPT’s outputs.
