By the Education Policy & Technology Desk
Published in partnership with the AI Policy Hub (Bellwether and PIE Network)
Executive Overview
As artificial intelligence (AI) continues its rapid and often disruptive march into K-12 classrooms, school systems find themselves grappling with a paradox. On one hand, the market is flooded with a dizzying array of generative and adaptive AI technologies promising to revolutionize teaching and learning. On the other hand, a rising wave of skepticism among parents, educators, and policymakers highlights a growing fatigue with flashy, unproven ed-tech trends that fail to deliver in the real world.
While safety-oriented regulatory frameworks—focusing heavily on data privacy, algorithmic bias, and student surveillance—have rightly taken center stage, a critical gap remains: How do we evaluate the actual efficacy and pedagogical quality of AI tools used by students and teachers?
Enter the "Signals of Quality" series, an initiative developed through the AI Policy Hub (a collaborative venture between Bellwether and the Public Innovation and Public Education [PIE] Network). The series seeks to cut through the marketing noise of the ed-tech sector by establishing a rigorous baseline for what genuinely effective AI-powered learning tools look like.
In this final installment of the series, the focus shifts from theoretical frameworks to practical application. Specifically, it examines how education advocates, state leaders, and district policymakers can weaponize procurement—the process of sourcing, negotiating, and acquiring educational tools—to ensure that taxpayer dollars and classroom time are dedicated strictly to tools that drive measurable student growth. By aligning procurement practices with five distinct "Signals of Quality," stakeholders can transform the K-12 ed-tech market from the inside out, demanding products that put student learning outcomes ahead of technological novelty.
Detailed Chronology: The Evolution of AI in Ed-Tech Procurement
To understand why current procurement practices are failing to filter out substandard AI tools, it is necessary to examine the historical trajectory of educational technology adoption over the past two decades.
Phase 1: The Wild West of Digital Disruption (2010–2018)
During the initial digital transition—characterized by the widespread adoption of 1:1 laptop initiatives and tablet rollouts—procurement was largely decentralized. School districts rushed to digitize textbooks and implement basic learning management systems (LMS). Ed-tech companies sold products based on feature lists—boasting cloud storage, gamification, and sleek user interfaces—with little to no empirical evidence demonstrating their impact on academic achievement. Districts lacked the evaluation infrastructure to assess whether these tools improved foundational skills, leading to what educational researchers term "digital bloat."
Phase 2: The Pandemic Acceleration and the Emergency Procurement Era (2020–2022)
The onset of the COVID-19 pandemic forced an overnight pivot to remote learning, triggering an unprecedented influx of federal relief funds, notably through the Elementary and Secondary School Emergency Relief (ESSER) funds. During this window, school districts engaged in emergency procurement, often fast-tracking software purchases to solve immediate logistical hurdles. Vetting processes were compressed, and long-term sustainability was sacrificed for short-term fixes. Ed-tech vendors capitalized on this urgency, saturating the market with adaptive software algorithms that promised to "close learning gaps" without offering transparency into how their algorithms functioned or whether they aligned with evidence-based instructional practices.
Phase 3: The Generative AI Boom and the Regulatory Wake-Up Call (2023–Present)
The public release of advanced generative AI models in late 2022 fundamentally altered the educational landscape. Overnight, students and teachers gained access to tools capable of writing essays, solving complex calculus problems, and generating lesson plans in seconds.
Initially, the institutional response was defensive. Many districts implemented sweeping bans on tools like ChatGPT out of valid concerns regarding academic integrity, data privacy, and misinformation. However, as the realization set in that AI is here to stay, school systems shifted from outright bans to reactive policy formulation.
State departments of education began issuing guidance documents. However, these documents overwhelmingly focused on guardrails: preventing cheating, securing student data, and managing copyright issues. While essential, these safety frameworks left a gaping void regarding quality control. State leaders and advocates realized that procurement was the ultimate lever to shape the market, yet the existing procurement infrastructure remained tethered to a slower-moving, pre-AI era.
This realization catalyzed the creation of the AI Policy Hub and the "Signals of Quality" series, designed to give advocates and state leaders the vocabulary, frameworks, and tactical steps needed to demand rigorous pedagogical standards from ed-tech developers.
Supporting Context & Metrics: The State of Ed-Tech Procurement and AI Efficacy
To appreciate the urgency of reforming AI procurement, one must examine the systemic inefficiencies plaguing the current K-12 education market.
The Evidence Gap
According to recent analyses by educational research organizations, fewer than 10% of ed-tech products currently on the market possess rigorous, independent third-party evidence of efficacy (such as randomized controlled trials or quasi-experimental studies meeting What Works Clearinghouse standards). The vast majority of products rely on internal case studies, testimonials, or vague claims of "machine-learning personalization."
The Procurement Bottleneck
School districts spend billions of dollars annually on educational software. Yet, a survey of district procurement officers reveals several structural challenges:
- Fragmented Vetting: Small and mid-sized districts rarely have the specialized staff required to evaluate complex AI algorithms, often relying on vendor claims rather than independent audits.
- Misaligned Incentives: Ed-tech venture capital historically rewards rapid user acquisition and feature expansion over long-term educational impact, leading companies to prioritize flashy conversational interfaces over sound pedagogical design.
- The "Black Box" Problem: Many generative AI tools are built on foundational models whose underlying logic cannot be easily audited by educators, creating risks around algorithmic bias, hallucinations, and developmentally inappropriate outputs.
The Five Signals of Quality
To counter these systemic flaws, the AI Policy Hub establishes five core signals that define high-quality AI tools in education. Aligning advocacy and procurement around these signals is essential for market correction:
- Emphasis on Learning Outcomes, Not Technology Features: High-quality AI tools are judged strictly by their ability to accelerate student learning and mastery of concepts, rather than by how many cutting-edge AI features (e.g., voice cloning, avatar generation) they incorporate.
- Productive Struggle as a Primary Pathway for Learning: Effective educational tools do not simply do the work for the student; rather, they scaffold learning, foster critical thinking, and facilitate healthy, productive struggle that builds cognitive resilience.
- Sound Pedagogy and Coherence with Existing Instructional Practice: AI tools must align with established learning sciences and integrate smoothly into existing curricula and teacher-led instruction, rather than operating as isolated gimmicks.
- Technical Configurations Designed to Maximize Quality: This involves architecture that minimizes hallucinations, ensures age-appropriate data isolation, and maintains strict guardrails to protect students while optimizing response accuracy.
- Attention to Market Sustainability and Long-Term Planning: Tools must be economically viable for school districts beyond initial grant funding cycles (such as fading federal relief funds) and transparent about long-term data governance and pricing models.
Official Statements & Expert Perspectives
The transition from passive observation to active procurement advocacy requires coordinated leadership across the educational ecosystem. Experts and coalition leaders have emphasized the urgency of this mission.
Dr. Sarah Lin, Senior Fellow at Bellwether, noted the unique position advocates hold in this landscape:
"For too long, the conversation around educational technology has been dictated by what software companies can build, rather than what students and teachers actually need. Procurement is not merely an administrative back-office function; it is a profound values statement. When state leaders and advocates use procurement as a quality filter, they send a clear, unmistakable signal to the market: if your AI tool does not demonstrably improve student learning, it does not belong in our classrooms."
Marcus Vance, director of the PIE Network’s AI Policy Hub initiatives, underscored the collaborative nature of the work:
"State policy and local procurement do not operate in a vacuum. Advocates are the bridge between state-level policy intent and district-level reality. By translating high-level quality signals into actionable procurement checklists, advocates can empower school boards and superintendents to ask the hard questions—challenging vendors to prove efficacy, protect student agency, and respect the irreplaceable role of the classroom teacher."
Education technology researchers have similarly echoed the need for proactive curation. As artificial intelligence becomes deeply embedded in daily school operations, regulatory bodies are recognizing that safety frameworks alone are insufficient. Without a concurrent focus on pedagogical efficacy, schools risk investing heavily in sophisticated software that may ultimately stunt student engagement or substitute critical thinking with automated compliance.
Three Strategic Entry Points for Advocacy Organizations
Because the field of educational AI is still in its infancy—and because traditional procurement infrastructure was built for a slower-moving, textbook-driven era—advocacy organizations may wonder where to begin. The appropriate starting point depends heavily on an organization’s local context, existing political relationships, and strategic priorities.
The AI Policy Hub outlines three distinct entry points for advocacy groups looking to champion high-quality AI procurement:
1. The Policy and Guidance Entry Point (State-Level Focus)
For organizations with strong ties to state departments of education and state boards of education, the entry point lies in shaping regulatory guidance and approved vendor lists.
- Actionable Steps: Advocate for the inclusion of the "Signals of Quality" in state-level ed-tech guidance documents. Push state agencies to develop evaluation rubrics that go beyond basic data privacy and cybersecurity compliance, explicitly requiring vendors to provide independent evidence of student learning gains.
2. The District Procurement and Purchasing Entry Point (Local-Level Focus)
For organizations working directly with school districts, superintendents, and local school boards, the focus shifts to local purchasing policies and Requests for Proposals (RFPs).
- Actionable Steps: Partner with progressive school districts to pilot new RFP language that demands algorithmic transparency and pedagogical coherence. Train local education stakeholders to evaluate ed-tech vendor pitches through the lens of student outcomes and "productive struggle," ensuring that software purchases are subjected to rigorous vetting before contracts are signed.
3. The Capacity-Building and Coalition Entry Point (Ecosystem Focus)
For statewide advocacy coalitions looking to build long-term momentum, the priority is stakeholder education and cross-sector collaboration.
- Actionable Steps: Host informational briefings, workshops, and working sessions that bring together educators, parents, technologists, and policymakers. Develop user-friendly toolkits that translate complex technical AI concepts into practical checklists for school administrators, empowering grassroots voices to demand accountability in ed-tech spending.
Future Outlook: The Road Ahead for AI in K-12 Classrooms
The integration of artificial intelligence into American education is no longer a speculative future scenario; it is an active, ongoing transformation. However, the trajectory of this transformation remains unwritten.
If left unchecked, the ed-tech market risks repeating past mistakes—flooding classrooms with expensive, superficial technologies that distract from core educational goals and widen equity gaps. Conversely, if state leaders, education policymakers, and dedicated advocates seize the moment, procurement can become a powerful instrument of quality control.
As state guidance and procurement practices continue to evolve in the coming years, advocates will play an indispensable role. By anchoring their efforts in the five "Signals of Quality"—prioritizing learning outcomes over tech features, preserving productive struggle, demanding sound pedagogy, ensuring robust technical configurations, and planning for long-term market sustainability—advocates can ensure that artificial intelligence fulfills its highest promise: serving as a genuine catalyst for student growth, equity, and success in the 21st century.
