The artificial intelligence (AI) chatbots provided by TurboTax and H&R Block for tax advice were unhelpful or wrong as much as half of the time, a technology columnist at The Washington Post found.

Relying on either of the chatbots for even lightly challenging tax questions could confuse the user, or even trigger an audit, wrote Geoffrey A. Fowler, the paper’s technology columnist.

He gave an example of asking both services about where a daughter should file taxes if she goes to college out of state, getting “irrelevant advice about tax credits and extensions” from TurboTax’s Intuit Assist and a wrong answer from H&R Block’s AI Tax Assist bot: It suggested that she has to file in both places, when the correct answer is that she has to file in the other state only if she has earned income there.

He also reported getting many random, misleading or inaccurate AI answers in response to additional questions.

“The good news is that you can completely ignore the chatbots and still do your taxes,” he wrote, adding that his experience “shows we should be especially wary of generative AI when there are real-life consequences to it being wrong.”

“I feel that my job as a tax professional is very secure,” said Beverly Goodman​​​​, a tax manager at EP Wealth who helped Fowler analyze the AI advice.

Both companies included caveats in their products, describing them as works in progress. “Intuit Assist is still developing and will improve with your help,” said TurboTax. “Al Tax Assist is a digital helper that’s still learning, so please review all responses,” said H&R Block.

TurboTax’s responses to questions are a mix of written-out answers and links to pages written by TurboTax staff and its self-help community of experts, Fowler wrote. Intuit told him that the bot is a hybrid of the self-help system it has had for years and newer generative AI technology designed to help people with less-common questions. Some answers provided by the TurboTax community reminded Fowler of "the unhelpful old Clippy from Microsoft Office."

When he asked why the chatbots were so often wrong,  Intuit spokeswoman Karen Nolan told him, “We had our legacy technology positioned to be the first place Intuit Assist pulled answers from, We have now updated that so Intuit Assist’s self help answers are pulled first from a more advanced, multi faceted search capability.”

After the update, Fowler’s retesting reported that his second round of tests yielded "many more correct, straightforward answers." Yet he found that “the chatbot still responded with answers and links to irrelevant information a quarter of the time.”

H&R Block told Fowler that it uses underlying tech from Microsoft and specifically trained it to get answers from its in-house Tax Institute. Though its answers were more on-topic, he found, the bot was also more likely to make up answers that were just wrong.

H&R Block suggested that his questions lacked the “specificity and clarity” that AI needs to be effective. “Just imagine if you got those kind of excuses from a human tax preparer,” wrote Fowler.

“Is it perfect, no. Will it ever be, probably not. Mainly because we have a very complicated taxation system with ambiguous wording,” H&R Block spokeswoman Teri Daley wrote in an email. “We believe we are being very prudent by putting the language directly above the prompt area to remind individuals to confirm answers. We feel strongly that the other capabilities built into the DIY software, such as the guided interview, and allowing individuals to connect with a Tax Pro free are what ultimately ensure a correct return.”

Fowler credited the companies for telling him that they guarantee the accuracy of one’s final tax return, and will provide assistance if a customer gets audited because he or she listened to their AI’s bad advice. “Let’s hope that comes through a human, and not a chatbot,” he wrote.

"As with any AI tool, when individuals provide detailed, contextual information, AI Tax Assist generates more accurate and helpful responses,” wrote Heather Watts, senior vice president of consumer tax products at H&R Block, in an email to Accounting Today. “Regarding The Washington Post's coverage, our experts at The Tax Institute tested the responses deemed inaccurate, finding that alternate phrasing and use during the guided interview led to more accurate responses," she wrote, adding that H&R Block overall is backed by guarantees including audit support, maximum refunds and 100 percent accuracy.

Intuit has not found a significant number of people experiencing issues with its AI tool, contrary to what the Post reported, Nolan told Accounting Today. Intuit, she said, understands the stakes involved in an accurate tax filing, which explains its guarantee policy.

"At TurboTax, we understand the seriousness of taxes and finances, and we provide every filer with a tax return lifetime guarantee, including 100% accurate calculations, audit support and a maximum refund," she said.