The Hidden Curriculum of AI: Unmasking ChatGPT's SEO-Informed Writing Biases

How can we encourage writing as a creative endeavor when we constrain our students with rigid forms and genres?

As the new school year kicks off, I find myself at the exciting threshold of transitioning to teaching seniors. This year, our class will lean heavily into integrating and implementing AI into student work cycles, all while maintaining a focus on cultivating and maintaining voice—conceptualized at the intersection of agency, expression, and creativity. For a deeper dive into how we approach voice, see my previous post. This week, with the help of my friend and colleague

Bechem Ayuk

, I've ventured into building my first website and started unraveling the complexities of SEO. This exploration has led to some profound discoveries about AI that carry potentially serious implications for our students.

---

Quick plug: There is still time to sign up for “From Policy to Practice” Seminar. Seating is limited. Registration link here: https://lnkd.in/etJNkviZ

---

Section One: The Data Behind AI

As we all know by now, artificial intelligence, particularly in the realm of text generation like ChatGPT, is trained on vast datasets scraped from the internet. These datasets encompass a wide variety of texts, from peer-reviewed academic papers and revered classic literature to the vast ocean of blog posts and social media updates that populate the web. While the diversity of this content might suggest a rich training ground, the reality is somewhat different. A significant portion of this data is tailored with a very specific goal in mind—ranking well on Google. This means that many of the texts feeding into AI models are optimized for search engine algorithms, designed more to capture eyeballs than to enlighten or educate.

---

---

Section Two: The Rise of SEO as an Art and a Science

SEO, or Search Engine Optimization, has profoundly transformed how “content” (ugh…hate this word!!!) is created and consumed online. It walks a fine line between art and science, engineered to increase a website's visibility in search engine results, drive more traffic, and ultimately boost engagement and conversion rates. However, the rise of SEO has also introduced practices that, while effective in achieving these goals, can distort the original intent of content and subtly alter how information is conveyed.

To illustrate the impact of SEO, consider a simple paragraph, progressively modified to meet SEO standards:

Example 1: Original Paragraph (No SEO Intervention)
"Many cities across the world are facing increasing pressures from growing populations and urban sprawl. This leads to challenges in sustainability and quality of life. Effective urban planning is crucial to address these issues."

Example 2: Initial SEO Intervention
Headline Added: "Urban Planning Challenges in Growing Cities"
Keywords Identified: urban planning, sustainability, quality of life
Modified Paragraph:
"Cities worldwide are grappling with the pressures of increased populations and extensive urban sprawl. These factors pose significant challenges to sustainability and quality of life. Effective urban planning is essential to mitigate these issues."

Example 3: Final SEO Product
"Urban Planning Challenges in Growing Cities: Ensuring Sustainability and Quality of Life"
"Urban sprawl and booming populations are putting cities at risk. Discover how effective urban planning is critical in maintaining sustainability and enhancing the quality of life. Explore strategies that can make a difference."

Terminology Explanation:

• Keywords: These are terms and phrases specifically chosen to improve search engine visibility. In this case, "urban planning," "sustainability," and "quality of life" are targeted to attract readers interested in city development and environmental issues.

• Headline: Designed to catch both the reader's eye and search engine algorithms, the headline incorporates keywords and is structured to be engaging and informative.

• SEO-Optimized Content: This content is adjusted to include high-impact keywords naturally within the text, crafted to provide answers and draw in readers, hence improving search rankings and engagement.

This progression from a straightforward, informative paragraph to a highly optimized SEO product demonstrates the significant influence of SEO on content creation. With each stage of revision, elements are introduced to enhance visibility and engagement, but these changes also steer the content away from pure information delivery toward a focus on maximizing clicks and traffic.

---

Section Three: The Impact of SEO on Writing and Reading

The SEO process has fundamentally transformed both writing and reading, often not for the better. The emphasis on keywords and search engine performance has led to a homogenization of content, prioritizing algorithmic appeal over creativity, agency, and authentic expression.

In my previous post on voice, I discussed how true voice encompasses agency (the power and autonomy of the author in shaping the content), expression (the unique style and perspective conveyed by the author), and creativity (the originality and inventiveness that make the content distinct). SEO-driven writing often collapses these dimensions, reducing the writer's agency and creativity to serve algorithmic and commercial interests.

This reduction manifests in various ways. SEO-driven content tends to rely on repetitive language to meet keyword density requirements, which can flatten the nuances of a writer's voice. The need to include specific keywords multiple times within a piece can result in a monotonous tone and a lack of depth, as the focus shifts from the substance of the message to the mechanical inclusion of these terms. This compromises the richness of expression and stifles creativity, leading to content that feels formulaic and disconnected from the writer’s true intent.

Additionally, the prioritization of SEO-friendly structures, such as bullet points and subheadings (yes, I too am guilty!!! Mea culpa!!!), further diminishes the complexity and flow of writing. While these structures are useful for improving readability and search engine performance, they often reduce content to a series of digestible chunks, stripping away the layers of meaning that are typically conveyed through more nuanced writing. As a result, both the writing and reading experiences are diminished, with readers engaging more with the structure of the content than its substance.

---

Section Four: AI’s SEO-Informed Writing Preferences

AI systems, trained on vast datasets derived from the internet, inherently carry SEO-informed writing preferences. While techno-optimists like

Ethan Mollick

emphasize the protean flexibility of AI tools through exercises like AI role-playing, it is clear that these systems have built-in biases and values shaped by their training data. When tasked with editing an essay, AI systems frequently produce condensed text with keyword-heavy headlines and bullet-pointed exposition, reflecting a structure aligned with SEO practices.

Even though AI tools do not conduct keyword research with the precision of specialized SEO tools, they still follow the general principles of SEO, favoring content structure and keyword density over deeper engagement and creativity. This means that AI-driven writing often gravitates toward the same formulaic tendencies seen in human-generated SEO content. The result is a type of writing that is optimized for search engines but lacks the depth, originality, and personal voice that make writing compelling and meaningful.

The implications of these SEO-informed preferences are particularly concerning in educational settings. When students use AI tools to write or edit their work, they may unconsciously absorb these preferences, leading them to prioritize keyword placement and SEO-friendly structures over the development of their own voice and creative expression. This not only undermines the educational goal of fostering independent, critical thinkers but also risks creating a generation of writers who are more attuned to the needs of algorithms than to the art of writing itself.

---

Section Five: The Implications for Our Students

The integration of AI tools into the classroom poses a significant risk of inadvertently turning students into SEO professionals.

Terry underwood

on his Substack Learning to Read, Reading to Learn highlights the collision between the Common Core’s privileging of the five-paragraph essay and the widespread accessibility of AI systems. The five-paragraph essay, with its clear structure and focus, is a perfect AI prompt, easily manipulated by AI tools to produce optimized content.

Reflections on Portfolio Thinking

In a similar vein, if students come to rely heavily on AI for writing and editing, they risk absorbing the values and norms of SEO and internet writing, which prioritize algorithmic success over original thought and genuine expression. This could lead to a generation of students who write for the machine and like the machine rather than for human connection and intellectual growth.

The danger here is that students will CONTINUE to view writing as a process of filling in predetermined templates rather than as a creative and intellectual endeavor. How do we interrupt this continuation? Dare I say that is the crux of the matter from a pedagogical perspective. This shift could have profound implications for how students think and express themselves, as they may come to see writing as merely a tool for achieving technical correctness and algorithmic approval, rather than as a medium for exploring ideas, expressing individuality, and engaging with the world around them.

Furthermore, the widespread use of AI tools that prioritize SEO-friendly writing structures could reinforce a narrow, reductive view of what good writing looks like. Students might come to equate good writing with the ability to produce content that meets the technical requirements of search engines, rather than with the ability to craft nuanced, thoughtful, and original pieces that reflect their own voices and perspectives.

---

Where Does This Leave Us?

As the new school year approaches, this situation demands immediate attention. Teachers need time, space, and training to understand and integrate AI responsibly into their classrooms. Last year saw many schools largely ignoring AI, but the risk of inaction grows exponentially as AI systems increasingly shape the implicit values driving writing, reading, and thinking.

If schools fail to engage in this conversation and address these issues head-on, the process will unfold beneath the surface, off-campus, on platforms like TikTok, where students are ironically engaging with SEO-driven or AI-composed content about AI integration and implementation. The time to take action is now—before the subtle yet pervasive influence of AI-driven SEO norms becomes the default mode of student expression and thought.

Teachers and administrators need to collaborate to develop strategies that prioritize human creativity and critical thinking over algorithmic optimization. This could involve rethinking the types of writing assignments given to students, encouraging more open-ended and exploratory tasks that resist the constraints of SEO-driven content creation. It also means providing students with the tools and knowledge they need to recognize and push back against the pressures of writing for algorithms, helping them to maintain their agency and creative expression in an increasingly automated world.

Nick Potkalitsky, Ph.D.

---

Check out some of my favorite Substacks:

Terry Underwood’s Learning to Read, Reading to Learn: The most penetrating investigation of the intersections between compositional theory, literacy studies, and AI on the internet!!!

Suzi’s When Life Gives You AI: An cutting-edge exploration of the intersection among computer science, neuroscience, and philosophy

Alejandro Piad Morffis’s Mostly Harmless Ideas: Unmatched investigations into coding, machine learning, computational theory, and practical AI applications

Michael Woudenberg’s Polymathic Being: Polymathic wisdom brought to you every Sunday morning with your first cup of coffee

Rob Nelson’s AI Log: Incredibly deep and insightful essay about AI’s impact on higher ed, society, and culture.

Michael Spencer’s AI Supremacy: The most comprehensive and current analysis of AI news and trends, featuring numerous intriguing guest posts

Daniel Bashir’s The Gradient Podcast: The top interviews with leading AI experts, researchers, developers, and linguists.

Daniel Nest’s Why Try AI?: The most amazing updates on AI tools and techniques

Riccardo Vocca’s The Intelligent Friend: An intriguing examination of the diverse ways AI is transforming our lives and the world around us.

Jason Gulya’s The AI Edventure: An important exploration of cutting edge innovations in AI-responsive curriculum and pedagogy.