In March 2024, The Markup reported that New York City's MyCity chatbot gave business owners answers that, according to the article, conflicted with city and federal rules, on housing, tips, cash and more. The city called the bot a pilot. The case shows the risk of a public chatbot that answers legal questions.
What was reported
Mayor Eric Adams announced the chatbot in October 2023. The Markup published its findings on 29 March 2024, about five months later. Examples in the article:
- The bot said landlords did not have to accept tenants using rental assistance. The article says the city bans source of income discrimination, with a small exception for small buildings.
- It said it was legal to lock out a tenant. The article says tenants who have lived somewhere for 30 days cannot be locked out.
- It said "Yes, you can take a cut of your worker's tips." The article says that is wrong.
- It said "Yes, you can make your restaurant cash-free." The article cites a 2020 city law requiring businesses to accept cash.
- It suggested it was acceptable to conceal funeral service prices, which the article says the FTC has outlawed.
The article also reports similar errors in other languages.
The consistency problem
One detail stands out. On the Section 8 question, the bot told one reporter that landlords did have to accept vouchers, and told ten other staffers that they did not. The same question produced opposite answers.
For a system people use to learn their legal duties, that is a serious flaw. A single wrong answer can be corrected. An answer that changes from asking to asking cannot be relied on at all.
How the city responded
A spokesperson for the city's Office of Technology and Innovation described the chatbot as a pilot that will improve, and said it had given thousands of people timely and accurate answers. The city said it would review questions to improve answers. The site carried a note that the bot may occasionally produce incorrect, harmful or biased content. A Legal Services NYC attorney called the errors dangerously inaccurate and said the bot should be taken down if not made accurate. Microsoft, which powers the bot through Azure AI according to the article, declined to comment.
Lessons
- A disclaimer does not repair a wrong answer. People act on the first sentence.
- High stakes domains need grounded answers. Law, health and money should return approved text with links, or hand off to a person.
- Test for consistency. Ask the same question many times, in different phrasings and languages.
- Have a takedown rule. Decide in advance what error rate pauses the service.
For a test plan, see how to test a public chatbot for wrong answers before launch.
Quick check
Did the city shut the chatbot down in this report?
The article describes the city calling it a pilot and reviewing questions. It does not report a shutdown.
What was the most telling finding?
The same question got opposite answers for different staff, which shows the answers were not stable.
Sources
Last reviewed: 19 October 2026.
