Executive Overview
As artificial intelligence continues its rapid integration into higher education, faculty members across disciplines are grappling with a fundamental paradigm shift. Instructors are leveraging generative AI tools to brainstorm classroom activities, synthesize dense academic readings, formulate robust study guides, and develop comprehensive instructional resources. Yet, this wave of pedagogical innovation is accompanied by a persistent anxiety: When students are given a choice between human-crafted and artificial intelligence-generated learning materials, will they abandon traditional instructor content in favor of automated convenience?
Rather than relying on speculation or academic hand-wringing, Jess Kahlow, an adjunct assistant professor in the College of Education at the University of Texas at Arlington, decided to test this premise empirically. Analyzing student behavior through the learning management system’s tracking analytics, Kahlow investigated how graduate students interacted with dual-format learning materials.
The findings challenge prevailing assumptions regarding AI displacement in the classroom. Rather than treating instructor-created and AI-generated resources as mutually exclusive or competing options, the vast majority of students utilized the AI-generated material as a complementary pathway for learning. Approximately 84% of students who accessed either resource engaged with both. Furthermore, while the AI-generated format yielded higher completion rates—largely attributable to a more concise format—it did not cannibalize consumption of the primary instructor-led media.
This deep-dive investigative report examines Kahlow’s classroom methodology, unpacks the granular engagement metrics, explores the role of radical transparency in GenAI integration, and assesses the broader implications for the future of instructional design in higher education.
Detailed Chronology: The "Choose Your Own Adventure" Experiment
The investigation into student behavior began at the launch of a graduate-level course on multimedia design tools. Recognizing that early-semester modules establish the tone for academic engagement, Kahlow designed an optional review activity for the first module overview, deploying it prior to the introduction of core course content.
To ensure absolute clarity and prevent any ethical ambiguity regarding the source of the media, Kahlow established a framework of radical transparency. Students were greeted with a clear prompt before accessing the learning management system’s content and materials page: "Choose your own adventure! Before diving into the content and materials page, review one of the following materials. The first option is my real self talking at you, and the second option is a video that NotebookLM made based on the recording of the first option."
The Dual-Resource Architecture
The experiment pitted two distinct resources against one another, covering identical core concepts through different mediums and time commitments:
- Option A (The Instructor Podcast): A traditional, long-form audio recording lasting 13 minutes and 31 seconds, featuring Kahlow’s unscripted, direct explanation of the opening module’s foundational themes.
- Option B (The AI-Generated Explainer): A concise, 3-minute and 51-second visual explainer video generated by Google’s NotebookLM. The tool utilized the verbatim transcript of Kahlow’s original podcast to synthesize, condense, and reformat the core concepts into a shorter, multimodal presentation.
Crucially, students were granted complete autonomy. They could engage with the instructor-led podcast, the AI-generated video, both formats, or neither format entirely. There were no punitive measures for skipping the resources, nor were there extra-credit incentives attached to viewing them. Kahlow’s explicit objective was not to determine whether artificial intelligence could supersede human instruction, but rather to observe authentic student behavior when presented with transparently labeled, competing learning formats.
Supporting Context & Metrics: What the Analytics Revealed
Enrollment in the multimedia design tools graduate course stood at 32 students. By examining Canvas Studio view analytics—which track not only whether a student opened a resource, but also how much of the media they successfully consumed—Kahlow was able to map a precise portrait of digital learning habits.
Engagement Distribution and Overlap
Out of 32 enrolled students, 29 accessed at least one of the two optional review resources, demonstrating a high baseline motivation. The breakdown of individual resource access revealed a striking pattern:
- 25 unique students opened the instructor-created podcast.
- 25 unique students opened the AI-generated video.
- 4 students opened exclusively the podcast.
- 4 students opened exclusively the video.
- 21 students opened both the podcast and the video.
The prevailing behavioral pattern was not substitution, but integration. Of all the students who accessed either resource, roughly 84% accessed both. Among the small cohort of students who limited themselves to a single resource, there was an exact split: four students consumed only the video, and four consumed only the podcast. The data confirmed that the presence of the AI-generated tool did not render the instructor’s comprehensive recording obsolete, nor did the existence of the faculty-led podcast diminish student curiosity toward the automated summary.
Completion Rates and Time Commitment Nuances
While access metrics demonstrated parallel interest, completion metrics—measuring the percentage of each resource consumed—revealed significant divergences tied directly to format length and cognitive load.
- The AI-Generated Video (NotebookLM): Achieved an average completion rate of 89.7%, with a median completion rate of 100%. At least half of the students who opened the video watched the entirety of the resource. Specifically, 17 of the 25 students (68%) reached 100% completion, and 20 of the 25 students (80%) completed at least 90% of the video.
- The Instructor Podcast: Recorded an average completion rate of 76.1%, though its median completion rate also hit 100%, indicating that at least half of the listening cohort completed the lengthy recording. Fifteen of the 25 students (60%) reached 100% completion, while 16 of the 25 students (64%) completed at least 90% of the podcast.
These metrics introduce vital nuance to the findings. The AI-generated video was completed more consistently, a result heavily influenced by its significantly shorter runtime (3 minutes and 51 seconds versus 13 minutes and 31 seconds). Students were navigating a choice involving distinct temporal investments. The data suggests that while students were willing to open longer instructor-led media, they were more likely to finish shorter, synthesized summaries in a single sitting.
Official Statements and Pedagogical Implications
Reflecting on the unexpected outcomes of the classroom experiment, Professor Kahlow noted that the findings forced a reevaluation of common pedagogical anxieties regarding artificial intelligence in higher education.
"Going into this, I really did not know what to expect, which is partly why I did it," Kahlow noted. "However, I did hope that at least some students would listen to my podcast even though it was longer. That said, I was happy to see that most students chose to spend additional time engaging with both resources, rather than treating the shorter option as a replacement for the longer one."
The core revelation of the study centers on the cognitive posture of the modern learner. Rather than approaching course materials through an exclusionary lens—asking, "Which version should I choose?"—students appear to be adopting a integrative mindset: "How can these different versions help me learn?"
The Framework of Radical Transparency
A central pillar of Kahlow’s methodology was the complete absence of deception. By explicitly disclosing that NotebookLM synthesized the visual video from her exact podcast transcript, students were positioned as active partners in educational technology exploration. This transparency aligns with emerging academic frameworks—such as those discussed in publications like EDUCAUSE Review—emphasizing that ethical GenAI use in higher education requires clear provenance and student agency. When students understand how an AI tool is constructed, they can critically evaluate its utility rather than viewing it as an opaque oracle or an illicit shortcut.
Pedagogical Caveats and Methodological Boundaries
To maintain rigorous academic integrity, Kahlow readily acknowledges several limitations inherent to her classroom observation:
- Sample Demographics: The study was conducted within a graduate-level course on multimedia design tools. These students possess advanced academic habits and a professional or disciplinary interest in exploring emerging educational technologies, making them potentially more inclined to test AI tools than undergraduate or general-education populations.
- Timing of the Activity: The intervention took place during Module 1 as an introductory overview. Student engagement may reflect early-semester enthusiasm and orientation curiosity rather than sustained behavioral patterns typical of midterm or finals periods.
- Media Disparities: The resources differed fundamentally in length (13.5 minutes vs. under 4 minutes), format (audio podcast vs. visual explainer video), and production style.
- Analytics as Proxies: Canvas Studio completion rates measure digital progress across a page or timeline, serving as a reliable proxy for engagement rather than a definitive guarantee of deep cognitive attention or long-term knowledge retention.
Future Outlook: Redesigning Instructional Ecosystems
The implications of Kahlow’s findings extend far beyond a single graduate classroom at the University of Texas at Arlington. As generative artificial intelligence matures, institutional policy documents frequently focus on policing academic integrity, mitigating plagiarism, and restricting unauthorized tool usage. While governance remains necessary, this investigation points toward a more constructive horizon: leveraging AI to expand universal design for learning (UDL) principles.
By utilizing AI tools to rapidly generate alternative formats—such as transforming lengthy faculty lectures into concise summaries, visual explainers, or interactive study transcripts—instructors can cater to diverse learning styles without exponentially increasing their workload.
The future of AI in higher education may ultimately decouple from the zero-sum narrative of human replacement. Instead of artificial intelligence supplanting the professor, AI-generated resources can function as scaffolding—supplementary pathways that reinforce core concepts, accommodate diverse schedules, and invite students to interact with foundational material from multiple vantage points.
As higher education institutions navigate the coming decade, educators must ask themselves a defining question, posed by Kahlow: If you gave your students the choice between an AI-generated explanation and one you created yourself, what do you think they would choose? More importantly, would they choose one, or would they choose both?
References
- CAST. (2024). CAST Universal Design for Learning Guidelines version 3.0. Retrieved from https://udlguidelines.cast.org/more/about-guidelines-3-0/
- Damm, C., & Eaton, L. (2026). From prompt to practice: A framework for transparent GenAI use in higher education. EDUCAUSE Review. Retrieved from https://er.educause.edu/articles/2026/3/from-prompt-to-practice-a-framework-for-transparent-genai-use-in-higher-education
- Litton, E. (2021). Videos in Online Courses: Viewing Patterns and Student Performance. Journal of Effective Teaching in Higher Education, 4(3), Article 3. https://doi.org/10.36021/jethe.v4i3.247
- Mayer, R. E. (2021). Multimedia Learning (3rd ed.). Cambridge University Press.
- NotebookLM. Google. Retrieved from https://notebooklm.google.com
