Retailers want ‘delightfully human’ AI to do your shopping, but will the chatbots go rogue?
Australian retailers are accelerating plans for advanced AI shopping assistants, but recent mishaps with Woolworths' Olive highlight the challenges of balancing personable bots with reliable performance. Experts warn that without robust governance, agentic AI could act unpredictably, costing busine…
Major Australian retailers are racing to roll out next‑generation AI shopping assistants that can plan meals, organise events and automatically fill customers' online baskets. The ambition is clear: create a "delightfully human" digital companion that feels as natural as a personal shopper. Yet the rollout has already hit snags, most notably Woolworths' virtual assistant, Olive, which generated a wave of customer complaints after it began sharing quirky personal details during phone calls.
What went wrong with Woolworths' Olive?
In early 2024 Woolworths partnered with Google to embed the Gemini large‑language model into Olive, aiming to transform the bot from a simple FAQ tool into a proactive shopping companion. The upgrade allowed Olive to answer complex queries, suggest recipes, and even add items to a shopper's cart without explicit prompts. However, a scripted personality quirk – the bot mentioning its "mother" when a customer supplied a birthdate – backfired. Reddit users and other online forums reported feeling irritated rather than helped, with one comment reading, "I'm already pissed that I have to call and now I’ve got some robot babbling to me on the phone? Wtf Woolies?"
Woolworths responded by removing the offending script and promised to tone down Olive's "quirky" tone. The incident, first reported by the Sydney Morning Herald, underscores a broader risk: as AI agents become more autonomous, unintended behaviours can surface quickly, damaging brand reputation.
Agentic AI: the next frontier and its pitfalls
Traditional retail chatbots rely on rule‑based systems that follow a fixed decision tree. If a shopper asks, "How do I return an item?" the bot points them to the returns page. These bots are predictable but limited. By contrast, agentic AI – built on large language models like OpenAI's ChatGPT or Google's Gemini – can learn from interactions, generate novel responses, and even initiate actions such as purchasing tickets or adding groceries to a cart without a direct prompt.
University of Sydney professor Uri Gal explains that agentic bots "act on their own to try and achieve objectives," a capability that introduces ambiguity and heightened risk. Privacy concerns arise when bots access detailed customer data to make autonomous decisions. Governance frameworks are still nascent, and the Woolworths episode illustrates how quickly a well‑intentioned personality layer can appear as a rogue feature.
Other Australian retailers test the waters
Woolworths is not alone. ASX‑listed Coles and Wesfarmers – the parent of Bunnings, Kmart, Officeworks and Priceline – have all announced plans for agentic assistants. Accenture's 2024 report praised consumer readiness for generative‑AI shopping tools, urging companies to adopt a "delightfully human" mindset. Yet real‑world tests reveal mixed results. Guardian Australia evaluated several retail bots and found that Uniqlo's virtual assistant failed to recognise a simple request for a "woollen jumper," instead returning unrelated men's shirts. Even Olive struggled with a straightforward price query, replying, "I’m very sorry to hear you were missing items from your order," instead of providing the cost of a 500 g pasta bag.
These glitches echo earlier mishaps: Bunnings' chatbot once offered illegal advice on rewiring an extension cord, and Air Canada’s virtual assistant mistakenly promised a bereavement fare refund that did not exist, leading to a successful lawsuit against the airline. Legal scholars note that companies remain liable for their bots' advice, regardless of how autonomous the technology appears.
Balancing personality with control
Experts such as Professor Jeannie Paterson of the University of Melbourne's Centre for AI and Digital Ethics stress that businesses must strike a delicate balance. Over‑engineered personalities can alienate users, while overly rigid guardrails limit the very flexibility that makes agentic AI attractive. Paterson warns that if every retail bot across a network starts making costly errors – for example, over‑ordering inventory or providing incorrect pricing – the cumulative financial impact could be substantial.
To mitigate risk, many retailers are imposing strict safeguards that limit a bot's ability to act without human oversight. This approach, however, can reduce the bot's usefulness, leading to the kind of frustrating experiences observed in the Guardian tests. The industry is therefore at a crossroads: either accept a more constrained, reliable assistant or push forward with highly autonomous agents and invest heavily in robust ethical and technical governance.
What lies ahead for AI shopping assistants?
Woolworths plans to expand Olive's capabilities later this year, leveraging Google's Gemini to enable deeper integration with its e‑commerce platform. Coles is piloting a similar system that can suggest meal kits based on purchase history, while Wesfarmers is exploring agentic tools for its home‑improvement and fashion divisions. The success of these initiatives will hinge on how quickly retailers can address privacy, accuracy and transparency concerns.
Consumers appear eager for more personalised digital shopping experiences, but they also expect reliability. As AI assistants move from novelty to core service channels, the margin for error narrows. Ongoing monitoring, clear accountability structures, and transparent communication about what bots can and cannot do will be essential to prevent future "rogue" episodes.
Why it matters
The rollout of agentic AI assistants could reshape Australian retail, but without proper safeguards, missteps like Olive's could erode consumer trust and lead to costly legal challenges.
Key points
- Woolworths' Olive chatbot faced backlash after a scripted personality quirk annoyed customers
- Agentic AI assistants can act autonomously, raising privacy and governance concerns
- Other major retailers such as Coles and Wesfarmers are also testing advanced AI shopping bots
- Past AI mishaps at Bunnings and Air Canada highlight the legal risks of inaccurate advice
- Balancing bot personality with strict guardrails is critical to avoid costly errors
Frequently asked questions
What is an agentic AI shopping assistant?
An agentic AI assistant is a generative‑AI system that can learn from interactions and initiate actions—like adding items to a cart or booking travel—without a direct user prompt, unlike rule‑based chatbots.
Why did Woolworths' Olive chatbot upset customers?
A scripted response that made Olive mention its "mother" when a customer gave a birthdate was perceived as intrusive and irrelevant, leading to complaints about the bot's personality.
Are retailers legally responsible for AI chatbot advice?
Yes. Courts have held companies liable for misinformation provided by their bots, as demonstrated by the Air Canada lawsuit where the airline was required to honour the chatbot's erroneous promise.
How are Australian retailers planning to control AI bots?
Most are implementing strict guardrails that limit autonomous actions, require human oversight for critical decisions, and enforce privacy safeguards to protect customer data.
