The Paradox of AI Agency: How Siri-GPT Undermines User Control
Part 3 of Our Series on AI Tools and AI Agents
Greetings, Dear Readers,
Before I begin, I want to thank my readers who have decided to support my Substack via paid subscriptions. I appreciate this vote of confidence. Your contributions allow me to dedicate more time to research, writing, and building Educating AI's network of contributors, resources, and materials.
Apple and OpenAI Corner the Market on Everyday AI Agency
In recent years, the landscape of artificial intelligence has dramatically shifted, driven by advancements in generative models and their integration into everyday tools and applications. This transformation reached a new milestone with the recent partnership between Apple and OpenAI, announced at the 2024 Worldwide Developers Conference (WWDC) in Cupertino, California this week. This deal, which embeds ChatGPT capabilities into Apple's Siri and other core applications, is set to revolutionize the AI landscape and solidify the role of generative AI in our daily lives.
Apple's WWDC keynote, led by CEO Tim Cook and Senior Vice President of Software Engineering Craig Federighi, showcased a range of new AI features collectively branded as “Apple Intelligence.” The highlight of the event was the integration of ChatGPT-4o into Siri, transforming it from a basic voice assistant into a more sophisticated, context-aware AI companion. Federighi emphasized that these new capabilities would make Siri "more natural, relevant and personally contextual," enhancing the user experience by leveraging AI's advanced understanding and response capabilities (De Vynck, 2024). While this sounds impressive, it also raises significant concerns about privacy and control.
According to the announcement, users will be able to use Siri to perform a variety of tasks more efficiently. This includes composing emails, generating images, and accessing sophisticated responses to complex queries—all powered by ChatGPT. Apple also stressed its commitment to privacy, noting that most AI functions will run on-device to ensure data security. However, when necessary, more complex processes will be handled in Apple’s private data centers, with stringent measures to protect user information (Guo, 2024). Despite these assurances, the lack of explicit user consent in these upgrades is troubling.
The integration of ChatGPT into Apple’s ecosystem is a significant validation for OpenAI's technology, placing it in front of millions of Apple users who might not have previously engaged with generative AI. This partnership underscores the tech industry’s broader shift towards embedding AI deeply into consumer products, aiming to enhance functionality while maintaining a user-friendly experience (De Vynck, 2024). However, this also means users have little say in the adoption of these pervasive technologies.
Something Old, Something New
While this integration might seem like a natural progression given the increasing presence of AI in various domains, it also represents a fundamentally new and potentially disturbing development. ChatGPT is already the foundational architecture for a variety of educational tools and applications, such as Khanmigo, SchoolAI, and Verdebloom, and is an architecture option for tools like PowerNotes. These applications use AI to assist with tasks like lesson planning, personalized learning, and content creation, showing how embedded AI has become in educational contexts (Spencer, 2024).
However, Siri is a different kind of surface application. It is thoroughly embedded into “the way we live in the world.” Siri answers questions and helps us make decisions about mundane, everyday tasks as well as extremely important aspects of our lives. For instance, Siri can help with:
Setting reminders and alarms
Managing schedules and appointments
Navigating and providing directions
Sending texts and making calls
Searching for information online
Controlling smart home devices
Providing weather updates and news briefings
As already noted, this upgrade to a much more capable AI with ChatGPT's integration preempts consumer choice. One day, Siri users are dealing with the old, clunky AI, and the next, they are interacting with a new, much more sophisticated AI. This upgraded AI has emerging capabilities to access, analyze, and synthesize personal data, mobilize and utilize financial information and resources, among other things.
In essence, Apple and OpenAI have decided for us that more AI is in our best interest without including us in the dialogue (Guo, 2024). This imposition of advanced AI without consumer consent raises serious ethical concerns. While we can sit back and say that this is just what big tech does, to do so is to admit defeat very early on in the push towards greater AI agency—a positioning that could have devastating consequences for our colleagues, students, and loved ones.
Tools vs. Agents Dichotomy
In my previous two articles, I examined the traditional differentiation of AI as tools versus agents, focusing on degrees of autonomy and independence. I argued that all AI systems, regardless of their level of sophistication, are fundamentally tools designed to augment human capabilities. Even the most advanced AI should be viewed as tools, with their perceived agency being a byproduct of their functionality and integration into human activities. AI agency is influenced by technical independence, human relationality, and perceptions, existing on a spectrum of autonomy and human interaction.
The integration of ChatGPT into Siri brings this dichotomy into sharp focus. Traditionally, AI tools augment human capabilities without asserting their own agency. However, as AI systems like Siri with ChatGPT integration start handling tasks requiring contextual understanding and decision-making, they exhibit characteristics of agents, influencing user actions and decisions. This deeper integration makes AI operate seamlessly and invisibly, subtly influencing our behavior and decisions without our active awareness. Here, the invisibility of AI’s tool nature encourages users to perceive Siri-GPT, for example, as an AI agent.
This shift in perception from tool to agent status is transposes the locus of control from the user to the AI. Siri-GPT can cross-reference personal schedules, suggest actions, provide tailored recommendations based on past behavior, and integrate financial management tasks such as tracking expenses and making payments. It can also manage schedules and appointments, navigate and provide directions, send texts and make calls, search for information online, control smart home devices, and provide weather updates and news briefings.
These capabilities embed AI deeper into everyday life and make it more challenging to maintain its status as a tool. Critically, these agentic properties are not intrinsic but result from functionalities that encourage a relationship, leading to the perception of AI as an agent. This nuanced evolution impacts our autonomy and decision-making processes, as the relational dynamics foster a perception of AI that goes beyond its functional utility (Guo, 2024). The forced adoption of these functionalities below the level of consumer choice further complicates this dynamic.
All AI Agents are Tools: Regardless of their sophistication, AI systems are extensions of human intention and action.
Spectrum of Agency: AI agency is not binary but exists on a spectrum influenced by autonomy, independence, relationality, positionality, and human perception.
Perception and Relationship: The perception of AI as an agent often arises from the relationship and context in which the AI operates. For instance, an AI that adapts to teaching styles and student emotions may appear more agent-like due to its contextual integration and relational dynamics.
Influence Through Perception: The more the functionalities of an AI tool encourage users to perceive it as an AI agent, the more opportunities AI companies have to influence users. This allows them to introduce foundational changes in the way we interact with AI, often beneath the level of awareness and volition.
Conclusion
The integration of ChatGPT into Apple's ecosystem marks a pivotal moment in the evolution of AI. This partnership represents the completion of the shift towards GPT influencers, embedding AI deeply into the fabric of everyday tools and applications. As AI processes become more invisible and AI transitions from tool status to agent status, it is essential to navigate these changes with awareness and caution. The future of AI lies not just in its capabilities but in how we perceive and interact with it, shaping the role it will play in our lives. The question we must ask ourselves is whether we are ready to relinquish so much control to these AI systems and what implications this has for our autonomy and decision-making power.
Sources:
De Vynck, G. (2024). "Apple jumps into the AI arms race with OpenAI deal." The Washington Post.
Guo, C. (2024). "The Hidden Side of Apple Intelligence." Artificial Ignorance Substack.
Check out some of my favorite Substacks:
Terry Underwood’s Learning to Read, Reading to Learn: The most penetrating investigation of the intersections between compositional theory, literacy studies, and AI on the internet!!!
Suzi’s When Life Gives You AI: An cutting-edge exploration of the intersection among computer science, neuroscience, and philosophy
Alejandro Piad Morffis’s Mostly Harmless Ideas: Unmatched investigations into coding, machine learning, computational theory, and practical AI applications
Michael Woudenberg’s Polymathic Being: Polymathic wisdom brought to you every Sunday morning with your first cup of coffee
Rob Nelson’s AI Log: Incredibly deep and insightful essay about AI’s impact on higher ed, society, and culture.
Michael Spencer’s AI Supremacy: The most comprehensive and current analysis of AI news and trends, featuring numerous intriguing guest posts
Daniel Bashir’s The Gradient Podcast: The top interviews with leading AI experts, researchers, developers, and linguists.
Daniel Nest’s Why Try AI?: The most amazing updates on AI tools and techniques
Riccardo Vocca’s The Intelligent Friend: An intriguing examination of the diverse ways AI is transforming our lives and the world around us.
I'm curious as to how Siri will maintain the ethical age limits on AI use.
Nick, I like this piece. Just a few quick note for now. Last night I was reading Yanis Varoufakis' book, Techno Feudalism. I am struck by the parallels to what I have read so far and points you make in this piece. I think they are complementary, but, as I'm only about a third of the way into the book, will not say more for the moment.
Another thing that struck me is the range different reactions of people who watched the WWDC announcements. While I recognize that the ChatGPT portion is important, while watching it, it actually struck me as not that important. Actually, given what was announced, the AI portions were almost anticlimactic. The thing I am actually most excited about was the earlier announcement that I could access everything on my phone from my Mac. Perhaps because of what I am doing at work this month, I was preoccupied by all of the security and privacy details, which include the ChatGPT connections, but did not particularly pick the part about OpenAI out from the rest.
I am struggling with the idea of agents and agency right now. That's why I haven't commented on your last two posts. I do agree that they are tools. I get what you are saying about Heidegger, though I dislike trying to work through his jargon and personally despise him, I think what you have written is important. At the same time, a lot of my views on agency are changing, largely as a result of what Amitav Ghosh has expounded in his last two non-fiction works (The Nutmeg's Curse a couple of years ago, and more recently, Smoke and Ashes), and reading about what science is learning about agency in plants (most recently, Zoë Schlanger's The Light Eaters). I believe I am starting to reconsider the agency of some of our tools historically, from fire to language to watches, but recognize that what I am talking about played out over much longer periods of time. Sorry if that does not make any sense for now; it is too inchoate in my mind.
Anyway, this is good. There is a lot to chew on in your last few posts. I look forward to seeing where you go with all this.