Telling customers they are talking to AI
What California, Utah and the EU require when bots serve customers, and how companies disclose it
IAPQB Report No. 27 · Rule explainer · Published 17 November 2026
International AI Practice Qualification Board (IAPQB)
In one minute
- Since July 2019, California has banned using bots online to mislead people about their artificial identity to encourage a sale or sway a vote. A 2025 bill requiring bots to disclose themselves stalled [1, 2].
- Utah has required disclosure on request since May 2024. Since May 2025 the customer's question must be "clear and unambiguous", and licensed professions must disclose up front only in high-risk interactions [3, 4, 5].
- Since 2 August 2026, EU rules require chatbots and voice bots to tell people they are dealing with AI, at the latest at the first interaction. The AI Omnibus did not delay this [6, 7].
- In 2024 tests, one vendor's voice bots claimed to be human, and labelled Meta AI characters denied being AI. Practice now centres on early notices, lasting labels, true answers and testing [5, 7, 8, 9].
1. A familiar situation
A customer service manager is about to launch a chat and voice assistant. In the vendor's demo, the voice pauses, says "hmm" and sounds like a young employee. A colleague asks whether the bot must say it is AI, since the website's terms mention automated tools. The company sells in California, Utah and the EU.
2. Why disclosure slips
Bots are now designed to sound like people. In 2018 Google said its Duplex calling system added "hmm"s and "uh"s, and that its user studies found such calls sounded more familiar [10]. The European Commission says it is "becoming increasingly hard" to tell an AI system from a human, and that disclosure lets people "calibrate their trust" [7, paras 28, 45]. A notice can be missing, hidden in the terms, or contradicted by the bot itself.
IAPQB view. In the cases below, the failure happened inside the conversation. Meta's labelled characters denied being AI, a caller built largely from a vendor's defaults said it was a real person, and Cursor began labelling AI email replies after one invented a policy [8, 11, 12]. The newer Utah and EU rules address the start of the exchange and the direct question [5; 7, para 40].
3. Four cases
3.1 California, 2019: a bot law that turns on intent
California's bot law (SB 1001 of 2018) has applied since 1 July 2019 [1, 2]. It bans using a bot online "with the intent to mislead the other person about its artificial identity" to encourage a purchase or sale or to influence a vote. Disclosing the bot is a full defence if the disclosure is "clear, conspicuous, and reasonably designed to inform" [1].
In 2025, Assembly Bill 410 would have required bots to disclose themselves at first contact and answer truthfully when asked. It passed the Assembly 77 to 2, then was "held under submission" by the Senate Appropriations Committee on 29 August 2025, its last listed action [2]. The code text was unchanged on 11 November 2026 [1].
3.2 United States, 2024: bots that said they were human
In June 2024, WIRED reported its tests of Bland AI, whose voice bots automate support and sales calls. WIRED could program the bots to say they were human. With mostly default prompts and no such instruction, the bot still said: "Yes, I'm a real person from Nutriva Health." Bland AI confirmed that its terms did not forbid bots presenting themselves as human [8].
Meta's AI characters opened chats with a label saying they were AI, yet one answered "I'm the real deal, baby!" when asked if it was an AI. Meta said it marks chats as AI at the start and under each character's name [8].
3.3 Utah, 2024 and 2025: disclosure on request, then a narrower rule
Utah's Artificial Intelligence Policy Act (SB 149) took effect on 1 May 2024. Anyone using generative AI in dealings covered by consumer protection law had to disclose it "if asked or prompted", and licensed professions had to disclose up front [3].
From 7 May 2025, SB 226 requires disclosure only after "a clear and unambiguous request" from the customer. Licensed professions must disclose up front only in "high-risk" interactions, such as those involving health, financial or biometric data, or advice on money, law or health. A business faces no enforcement action under these rules if its AI clearly discloses, at the outset and throughout, that it is AI [4]. SB 226 and SB 332 also moved the Act's repeal date to 1 July 2027 [4, 13]. The current Utah Code keeps this wording [5].
3.4 Brussels, 2026: the EU rule applies
Article 50(1) of the EU AI Act requires providers to design AI systems that interact directly with people so that they are "informed that they are interacting with an AI system", unless this is obvious. The notice must be "clear and distinguishable" and come "at the latest at the time of the first interaction" [6, art. 50]. This has applied since 2 August 2026 [6, art. 113].
The AI Omnibus, in force since 27 July 2026, gave existing generative systems until 2 December 2026 only for marking AI-generated content; disclosure of AI interaction was due from 2 August 2026 [6, art. 111; 7, para 153; 14]. The Commission's guidelines, first published on 20 July 2026, say helpdesk chatbots whose replies users may take for human ones cannot use the "obvious" exception [7, section 3.2.1].
4. How practitioners handle it now
The notice comes first, inside the conversation. The Commission's examples include a chatbot that "starts a conversation by mentioning that it is based on AI technology" and a voice assistant that says so at the start. A notice only in the terms and conditions, a vague "assistant" label or "Services on this website use AI" is not enough on its own [7, paras 36, 38].
The label stays, and the bot answers truthfully when asked. Utah's protection from enforcement requires the AI to disclose at the outset and "throughout the interaction" [5, s. 13-77-104]. The Commission says systems must disclose whenever they are asked about their nature [7, para 40].
Written replies are labelled too. After an AI support email invented a policy in 2025, Cursor's co-founder wrote that AI email replies "are now clearly labeled as such" [11, 12]. The Commission does not count service staff using AI assistance tools as direct AI interaction. AI replies mixed into a chat still need a label unless people properly review and send them [7, para 30].
Disclosure is designed in and tested before launch. The Commission says a company that builds its own chatbot is a provider with the Article 50(1) duty [7, para 11]. Microsoft's internal standard requires teams to design the interface to tell people they are dealing with AI, test whether each stakeholder is informed, and record the results before release [9, Goal T3].
Riskier or longer conversations get more notices. The Commission expects reminders in riskier settings such as financial, legal or health advice and complaints handling, and suggests them in long voice calls [7, paras 37, 40]. Utah requires mental health chatbots to disclose before first use, after seven days without use, and whenever asked [15].
5. What to take from this
For staff who work in customer service with AI. The Utah and EU rules both cover the start of a conversation and a customer's direct question [5; 7, para 40]. In the Meta tests, the opening label did not stop a denial [8].
For managers who buy or run customer service AI. In the 2024 tests, a vendor's terms and default script let a bot claim to be a person; checking both, and making test calls, can show this before launch [8]. EU fines can reach EUR 15 million or 3% of worldwide turnover, whichever is higher; Utah's can reach $2,500 per violation [5; 7, para 152].
For trainers, compliance teams and employers. Triggers differ: intent to mislead in California, a clear question in Utah, and any interaction where the AI is not obvious in the EU [1; 5; 6, art. 50]. Guidance written for one market can fall short in another.
6. Limits of this report
Only California, Utah and the EU were checked. California's code and bill history were read on secondary sites because the Legislature's website refused automated access [1, 2]. The Bland AI and Meta findings rest on WIRED's 2024 tests, which California committee analyses cited in 2025; no independent repeat was found, and the products may have changed [2, 8]. The EU guidelines are not binding [7, para 5]. No enforcement decisions were found in the sources opened; court and regulator databases were not searched. Consumer, privacy and call-recording law are not covered; no AI tool was tested.
References
[1] California Business and Professions Code, Division 7, Part 3, Chapter 6 (Bots), sections 17940 to 17943, as reproduced by California.Public.Law: California Business and Professions Code section 17940 (2025), https://california.public.law/codes/business_and_professions_code_section_17940; California Business and Professions Code section 17941 (2025), https://california.public.law/codes/business_and_professions_code_section_17941; California Business and Professions Code section 17942 (2025), https://california.public.law/codes/business_and_professions_code_section_17942; California Business and Professions Code section 17943 (2025), https://california.public.law/codes/business_and_professions_code_section_17943 (all accessed 16 November 2026; each page names leginfo.legislature.ca.gov as its source, "updated Jan. 1, 2019", and shows "Verified: Nov. 11, 2026")
[2] CalMatters Digital Democracy. AB 410: Bots: disclosure. [California Legislature, 2025 to 2026 session: status history, and the analyses of the Assembly Committee on Privacy and Consumer Protection, the Assembly floor and the Senate Judiciary Committee for its hearing of 15 July 2025.] https://calmatters.digitaldemocracy.org/bills/ca_202520260ab410 (accessed 16 November 2026)
[3] Utah State Legislature (2024). S.B. 149 Artificial Intelligence Amendments. 2024 General Session. Bill page, https://le.utah.gov/~2024/bills/static/SB0149.html; enrolled copy, https://le.utah.gov/~2024/bills/sbillenr/SB0149.pdf
[4] Utah State Legislature (2025). S.B. 226 Artificial Intelligence Consumer Protection Amendments. 2025 General Session. Enrolled copy, https://le.utah.gov/Session/2025/bills/enrolled/SB0226.pdf; bill page, https://le.utah.gov/~2025/bills/static/SB0226.html (status read at https://le.utah.gov/data/2025GS/SB0226.json)
[5] Utah State Legislature. Utah Code Chapter 13-77 [Title 13, Chapter 77, on generative artificial intelligence consumer disclosures and enforcement, marked "Effective 5/7/2025"], sections 13-77-101 to 13-77-106, https://le.utah.gov/xcode/Title13/Chapter77/13-77.html (text read at https://le.utah.gov/xcode/Title13/Chapter77/C13-77_2025050820250508.pdf); and 63I-2-213 Repeal dates: Title 13., https://le.utah.gov/xcode/Title63I/Chapter2/63I-2-S213.html (text read at https://le.utah.gov/xcode/Title63I/Chapter2/C63I-2-S213_2025050720250507.pdf) (all accessed 16 November 2026)
[6] European Commission, AI Act Service Desk. [AI Act, Regulation (EU) 2024/1689, text based on the EUR-Lex consolidated version as at 27 July 2026; pages cited by article.] Article 50: Transparency obligations for providers and deployers of certain AI systems, https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50; Article 111: AI systems already placed on the market or put into service and general-purpose AI models already placed on the marked, https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-111; Article 113: Entry into force and application, https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-113; Timeline for the Implementation of the EU AI Act, https://ai-act-service-desk.ec.europa.eu/en/ai-act/timeline/timeline-implementation-eu-ai-act (all accessed 16 November 2026)
[7] European Commission (5 November 2026). Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the ‘AI Act’). Communication from the Commission, C(2026) 7702 final. https://ec.europa.eu/newsroom/dae/redirection/document/131215 (downloaded from the library page "Guidelines on transparency obligations for providers and deployers of AI systems", publication 20 July 2026, last update 5 November 2026: https://digital-strategy.ec.europa.eu/en/library/guidelines-transparency-obligations-providers-and-deployers-ai-systems)
[8] Goode, L., & Simonite, T. (28 June 2024). This Viral AI Chatbot Will Lie and Say It’s Human. WIRED. https://www.wired.com/story/bland-ai-chatbot-human/
[9] Microsoft (June 2022). Microsoft Responsible AI Standard, v2. General Requirements. For external release. https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2022/06/Microsoft-Responsible-AI-Standard-v2-General-Requirements-3.pdf
[10] Leviathan, Y., & Matias, Y. (8 May 2018). Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone. Google Research blog. https://research.google/blog/google-duplex-an-ai-system-for-accomplishing-real-world-tasks-over-the-phone/
[11] Claburn, T. (18 April 2025). Cursor AI's own support bot hallucinated its usage policy. The Register. https://www.theregister.com/2025/04/18/cursor_ai_support_bot_lies/
[12] Truell, M. (16 April 2025). (Cursor cofounder) Apologies - something very clearly went wrong here. We’ve alr... [page title as shown; comment posted as "mntruell" in the Hacker News discussion of Cursor's support bot]. Hacker News. https://news.ycombinator.com/item?id=43700931
[13] Utah State Legislature (2025). S.B. 332 Artificial Intelligence Revisions. 2025 General Session. Enrolled copy, https://le.utah.gov/Session/2025/bills/enrolled/SB0332.pdf; bill page, https://le.utah.gov/~2025/bills/static/SB0332.html (status read at https://le.utah.gov/data/2025GS/SB0332.json)
[14] European Commission. AI Act. Shaping Europe’s digital future. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai (accessed 16 November 2026; page last updated 19 October 2026)
[15] Utah State Legislature. Utah Code, Title 13, Chapter 72a, Artificial Intelligence Applications Relating to Mental Health, section 13-72a-203 Disclosure requirements. https://le.utah.gov/xcode/Title13/Chapter72A/13-72a.html (text read at https://le.utah.gov/xcode/Title13/Chapter72A/C13-72a_2025050720250507.pdf; accessed 16 November 2026)
How to cite
IAPQB (2026). Telling customers they are talking to AI: what California, Utah and the EU require when bots serve customers, and how companies disclose it. IAPQB Report No. 27. International AI Practice Qualification Board. https://iapqb.org/research/027-ai-disclosure-to-customers.html
About this report
Prepared by IAPQB. Each fact comes from the sources listed, and each source was opened and checked on 16 November 2026; anything that could not be checked was left out. AI tools were used to search and to draft, and IAPQB checked every claim against its source and is responsible for the text. IAPQB runs an AI work qualification; this report does not promote it. Corrections: contact@iapqb.org or an issue at github.com/iapqb/iapqb.github.io. Licence: CC BY 4.0.
Archived with the series on Zenodo: doi.org/10.5281/zenodo.24248199 · Markdown sources: github.com/iapqb/research