Exploring Generative AI Tools in the Management Information Systems Class for Management Undergraduates 

Generative AI has taken the world by storm. Businesses are adopting it rapidly, and using ChatGPT, copilots, and assistants is quickly becoming a critical skill—for faculty and staff who are already in the workforce and for students who are about to enter it. With a two-decade-long career in technology, I was curious to explore ways to incorporate Generative AI into the curriculum. This past summer, the Teaching and Learning Center offered a six-week-long Teaching Circle during which several faculty members prepared ways to incorporate the technology in the classroom. Artificial Intelligence is a natural fit in the Introduction to Management Information Systems course (MIS300) for undergraduates at West Chester University College of Business and Public Management. From the concept this summer to the implementation this fall, I am sharing my learnings and observations in this post and welcome your questions and feedback. 

Objectives

Generative AI makes the latest AI technology more accessible than it was before. Hence, the primary goal was to enhance students’ understanding of AI by providing practical examples and exercises. The goal was to make AI concepts more accessible and engaging, thereby improving students’ ability to apply them in real-world scenarios—in the classroom and beyond. Two two-and-a-half-hour lessons, one week apart, offered the opportunity to learn, apply, discuss, and refine students’ understanding of technology and tools such as OpenAI ChatGPT and Anthropic Claude. Here are the outlines of both lessons.

Implementation

The content of week 1 focused on an introduction to AI and its key concepts as the foundation for getting hands-on with publicly available versions of ChatGPT and Claude. Incorporating several AI tools helped show students the potential of this technology in the workplace and explain foundational concepts. The following tools and websites were helpful resources for students to explore these concepts first-hand:

• Google NotebookLM,
• Google Teachable Machine,
• ThisPersonDoesNotExist.com,
• ChatGPT, and
• Claude.

Students appreciated the variety of examples and the opportunity to experiment with these tools themselves.  

Interactive questions and knowledge checks via a student response system (Mentimeter) helped increase student engagement in class and gauge understanding as a pulse check. In future iterations, after introducing AI concepts and examples to separate information from reflection, additional questions will be added as knowledge checks. 

Initially, students used ChatGPT and Claude in pair work and compared their outputs. However, this approach faced challenges due to the variability in outputs, making it difficult to view and discuss specific results. Instead, I decided to share my screen, and we discussed prompts, iterations, and results collectively, which proved more effective. Students noticed that the output varies between tools and even when submitting the same prompt multiple times, which was an important active learning aspect of this exercise. Beyond using the tools to create emails, students also used them as coaches for interview preparation. This included generating common interview questions, coaching on how to answer them, and providing feedback on responses. This practical application was well-received by students and helped them see the real-world relevance of AI tools. 

In addition to in-class activities, I assigned students a homework assignment which encouraged students to interact with AI tools and apply them to practical scenarios. The assignment was graded on a pass/fail basis to encourage participation without adding pressure for those students who used these tools for the first time. For example, students used ChatGPT to explore career paths, prepare for interviews, and strengthen their resumes. This approach resonated well with students and provided valuable insights for future class iterations, such as structuring exercises (e.g., role-plays, exploration, ‘a day in the life of’ scenarios) based on these areas of interest.  

Students shared their scenario and approach at the beginning of the second lesson one week later. This enabled the class to learn from each other and receive inspiration about additional scenarios and approaches that they had not yet considered, building upon each other. For example: 

• Student 1 – Explore a career path in supply chain management 

• Student 2 – How to find internship opportunities 

• Student 3 – How to strengthen their accomplishments on their resume 

• Student 4 – Typical interview questions and how to prepare 

• Student 5 – Role-play interview questions plus feedback on improving individual responses 

94% of students in this class completed the homework assignment. Students could explain why they chose certain scenarios and found hearing from other students about their approaches beneficial. This feedback was crucial for refining the class structure and ensuring that future iterations are even more effective.  

The second lesson continued with applying advanced prompting techniques and sharing the findings with the class. It included a section on the responsible use of AI including Generative AI’s current limitations based on a combination of interactive online elements, recent news coverage, and pair work.  

Students shared several aspects they had learned during the two lessons… 

• We use AI everyday
• I learned about how to properly use ChatGPT and refine the answers it gives me to fit what I need.
• I learned the different types of ai and how it can be used to help with jobs or interviews
• AI is in our lives all the time
• I learned how different ai can provide different answers, but be alike in more ways than we think
• AI is everywhere
• How many different things you can do with AI
• How chat gpt can put itself into someone else’s shoes and have a great conversation about certain subjects
• I have learned how AI can help me better prepare for an interview
• How helpful AI really is
• different ways AI can be used in our daily lives
• ai can help do many things and helps with unexpected things

…as well as what has surprised them: 

• how often AI is used
• how much ai is in our daily lives
• How different ai can give different yet similar outputs to the same prompt
• That you can attach links to the conversation
• to have to use ChatGPT for a college assignment
• Ai can help with job application and interview processes
• AI is moving faster than I thought
• How useful chatgpt is in daily life
• How often people use AI everyday without knowing it
• How we’ll chat gpat acts I thought the questions would be more botty and not as detailed as they were
• how quick AI learns
• how different responses can become when you give it different information

Observations

When asked about their observations, students noticed that ChatGPT’s answers can be quite verbose. They also shared that ChatGPT’s answers were helpful. However, compared to similar evaluations in trainings for professionals who can be quite critical about the generated output, students could not accurately assess the quality and reliability of the AI-generated output due to their relative lack of experience and knowledge in specific disciplines. Providing a framework or rubric to evaluate responses could help address this issue in a future semester. 

Students were free to choose a topic of their own or select from a list of three topics. The intent was to have students spend more time using the tool rather than a lot of time defining a scenario. When asked about the quality and helpfulness of the AI-generated output for their selected topic, students predominantly shared that they found it to be ‘helpful.’ However, students selecting a topic was most likely a rather artificial scenario. Outside the class setting, under real-world conditions, students might choose a different topic to research and explore and, thereby, be even more invested in generating relevant output and assessing its quality and utility. Ensuring that assignments are closely aligned with students’ interests and goals will be crucial for future success. 

Conclusion

Integrating Generative AI tools in the classroom offers new opportunities for enhancing student engagement and understanding of AI concepts. From gaining a foundational understanding of AI and its everyday use to leveraging the latest Generative AI tools hands-on, students have shared that this lab has increased their awareness of AI and further accelerated their professional development. While there are challenges to address, the insights gained provide a valuable foundation for future improvements. Hands-on experience with Generative AI tools provides highly relevant and marketable skills that prepare students for a rapidly evolving environment when entering the workplace.