EXECUTIVE OVERVIEW
As artificial intelligence continues to reshape the landscape of higher education, faculty members across disciplines are racing to integrate generative tools into their teaching workflows. Educators routinely employ AI to brainstorm lesson plans, synthesize lengthy academic readings, design study guides, and curate supplementary multimedia resources. Yet, this rapid technological adoption has given rise to an urgent pedagogical anxiety: When given an explicit choice between instructor-crafted educational content and an AI-generated alternative, will students bypass their professors in favor of synthetic convenience?
Rather than surrendering to pure speculation, Dr. Jess Kahlow, an Adjunct Assistant Professor in the College of Education at the University of Texas at Arlington, decided to test this hypothesis empirically within her own graduate-level classroom. Utilizing the learning management system’s built-in tracking tools, Kahlow analyzed student engagement patterns when both human-authored and AI-generated study materials were presented side-by-side with complete transparency.
The findings challenge prevailing fears regarding the obsolescence of human educators in the age of generative AI. Rather than treating instructor-created resources and AI-generated assets as competing, mutually exclusive options, an overwhelming majority of students utilized the synthetic material as a complementary scaffold. Approximately 84% of students who engaged with either resource accessed both. Furthermore, while the AI-generated material yielded higher overall completion rates—largely attributable to its concise format—it functioned not as a replacement for the professor’s voice, but as an alternative, low-friction pathway for processing complex cognitive concepts.
This deep-dive investigation examines Kahlow’s classroom methodology, analyzes the quantitative metrics harvested from Canvas Studio analytics, explores the vital role of pedagogical transparency, and assesses the broader implications of these findings for the future of instructional design in higher education.
The Experimental Design: A "Choose Your Own Adventure" Framework
To investigate how students interact with artificial intelligence when it is placed on equal footing with human instruction, Kahlow structured a controlled, transparent experiment for the opening module of a graduate-level course centered on multimedia design tools.
Before diving into the foundational readings and core deliverables of the semester, students enrolled in the course encountered an optional review activity designed to summarize the module’s core objectives. Crucially, Kahlow rejected any form of deception, believing that pedagogical transparency is non-negotiable when introducing generative AI into academic settings. Students were explicitly informed about the nature of both resources before making their selection through a prompt titled: "Choose your own adventure!"
The interface presented students with two distinct pathways to review the exact same foundational material:
- Option One (The Instructor-Created Resource): A traditional, 13-minute and 31-second audio podcast recorded entirely by Kahlow, in which she verbally unpacked the objectives, expectations, and core themes of the module.
- Option Two (The AI-Generated Resource): A 3-minute and 51-second explainer video generated by Google’s NotebookLM. This synthetic tool utilized the verbatim transcript of Kahlow’s original podcast as its source material, transforming a lengthy auditory lecture into a shorter, highly condensed visual presentation highlighting the core concepts.
Students possessed total agency over their learning experience. They could elect to engage with the professor’s podcast, watch the AI-generated video, consume both assets sequentially, or bypass both resources entirely without facing any academic penalties or receiving extra-credit incentives. The objective was clear: rather than determining whether artificial intelligence could render human-led instruction obsolete, Kahlow sought to observe organic student behavior when AI is deployed transparently as a supplementary study aid.
Quantitative Metrics: Decoding Canvas Studio Analytics
With an enrollment of 32 graduate students in the course, Kahlow pulled granular interaction data from Canvas Studio to track student behavior beyond self-reported surveys. The analytics yielded precise insights regarding both resource access and completion rates.
Access Patterns: Complementary, Not Competitive
Out of the 32 enrolled students, 29 accessed at least one of the two offered resources, reflecting high baseline motivation. The engagement breakdown revealed a distinct pattern that defied binary assumptions of student preference:
- 25 unique students opened the instructor-created podcast.
- 25 unique students opened the AI-generated video.
- 4 students opened only the podcast.
- 4 students opened only the video.
- 21 students opened both the podcast and the video.
Among the cohort of students who accessed learning materials, roughly 84% accessed both formats. Far from viewing the AI-generated explainer video as an efficient substitute that allowed them to skip the professor’s lecture, students overwhelmingly treated the two assets as mutually reinforcing tools. Among the minority who chose just one format, preferences were evenly split, with four students opting exclusively for the human-led audio and four relying solely on the synthetic video.
Completion Rates: The Impact of Conciseness
While raw access metrics demonstrated a strong appetite for multi-format engagement, completion tracking—measuring how much of each resource a student actually consumed—revealed significant divergences tied directly to format and time commitment.
- The AI-Generated Video: NotebookLM’s 3-minute and 51-second video achieved an average completion rate of 89.7%. The median completion rate sat at 100%, indicating that at least half of the students who opened the video watched it from start to finish. Specifically, 17 of the 25 students (68%) reached 100% completion, while 20 students (80%) finished at least 90% of the video.
- The Instructor-Created Podcast: Kahlow’s 13-minute and 31-second audio recording yielded a lower average completion rate of 76.1%. While the median completion rate also hit 100%—meaning half of the listening cohort finished the entire lecture—only 15 of the 25 students (60%) reached full completion, and 16 students (64%) crossed the 90% threshold.
These completion metrics underscore a well-established principle in educational psychology: cognitive load and time commitments heavily influence sustained attention. Shorter content is inherently easier to finish in a single sitting. However, the data introduces vital nuance to the access findings. Students did not merely open both resources out of idle curiosity; when committing to a single-source deep dive, they were demonstrably more consistent in finishing the condensed, AI-generated multimedia presentation.
Pedagogical Implications: Redefining the Role of GenAI in Higher Education
The empirical results observed in Kahlow’s multimedia design course dismantle several prevalent anxieties circulating within academic faculty lounges regarding generative artificial intelligence.
From Replacement to Reinforcement
The primary takeaway from the data is that students do not conceptualize AI-generated course materials and instructor-created lectures as zero-sum adversaries. Instead of asking the reductive question, "Which version should I choose to minimize my workload?" students appeared to be operating under a framework of cognitive enrichment: "How can these different formats collectively help me master this material?"
By leveraging the AI-generated summary alongside the long-form instructor podcast, students utilized synthetic media as an iterative processing tool. The AI asset functioned as a high-level cognitive roadmap or preview, enabling learners to anchor core concepts quickly before or after engaging with the nuanced, conversational context provided by their professor.
The Power of Radical Transparency
A critical catalyst for these positive outcomes was the absolute transparency maintained throughout the activity. By explicitly detailing how NotebookLM synthesized the professor’s original recording into a compressed visual format, Kahlow removed the ambiguity that often breeds academic distrust.
When institutions or instructors integrate AI covertly—or fail to contextualize its purpose—students may feel manipulated or uncertain about the academic value of the output. Transparent implementation empowers learners to critically evaluate the strengths and limitations of synthetic media, fostering digital literacy alongside subject-matter mastery.
Methodological Caveats and Contextual Boundaries
To maintain rigorous academic integrity, Kahlow readily acknowledges several structural limitations inherent to her classroom activity. The findings should be interpreted as localized observations rather than universal pedagogical laws:
- Length and Format Disparities: The comparison was not entirely symmetrical. The instructor resource was a nearly 14-minute audio-only podcast, whereas the AI resource was a sub-4-minute visual explainer. The observed discrepancies in completion rates are likely heavily influenced by duration rather than an inherent superiority of synthetic content over human instruction.
- Derivative vs. Independent AI Generation: The NotebookLM video was built directly upon a human-authored transcript. It did not represent an independent AI hallucination or autonomous syllabus curation, but rather a compressed reflection of the instructor’s own expertise.
- Cohort Characteristics: The participants were graduate students enrolled in an advanced multimedia design program. These learners possess a professional and academic interest in educational technology, making them uniquely predisposed to experiment with generative AI tools out of disciplinary curiosity. Furthermore, graduate students historically exhibit higher compliance and completion rates for optional resources than general education or introductory undergraduate cohorts.
- Timing within the Semester: The activity took place during the opening module of the term. High initial engagement rates may reflect early-semester enthusiasm that could wane as academic burnout sets in later in the academic calendar.
Future Outlook: A Collaborative Educational Horizon
As higher education institutions navigate the uncharted waters of the generative AI era, faculty development programs must move past reactionary prohibitions and embrace thoughtful pedagogical integration.
Kahlow’s classroom experiment offers an inspiring blueprint for educators seeking to harness AI without undermining their own authority or classroom presence. The future of teaching and learning is unlikely to be defined by a dystopian binary where artificial intelligence supplants human professors. Instead, generative tools stand poised to vastly expand the multi-modal pathways through which students access, review, and synthesize complex academic concepts.
Ultimately, the experiment poses a compelling challenge to educators across the globe: If presented with the transparent choice between an AI-generated distillation of a core concept and an exhaustive human-crafted lecture, what will your students choose? More importantly, as Kahlow’s analytics prove, the right question may not be whether they choose one over the other—but whether they possess the agency to choose both.
References
- CAST. (2024). CAST Universal Design for Learning Guidelines version 3.0. Retrieved from https://udlguidelines.cast.org/more/about-guidelines-3-0/
- Damm, C., & Eaton, L. (2026). From prompt to practice: A framework for transparent GenAI use in higher education. EDUCAUSE Review. Retrieved from https://er.educause.edu/articles/2026/3/from-prompt-to-practice-a-framework-for-transparent-genai-use-in-higher-education
- Google. NotebookLM. Retrieved from https://notebooklm.google.com
- Litton, E. (2021). Videos in Online Courses: Viewing Patterns and Student Performance. Journal of Effective Teaching in Higher Education, 4(3), Article 3. DOI: https://doi.org/10.36021/jethe.v4i3.247
- Mayer, R. E. (2021). Multimedia Learning (3rd ed.). Cambridge University Press.
