Tax Returns with ChatGPT: Expert Warns of 10 Typical Misconceptions

Nearly half of taxpayers surveyed are open to AI tax advice, but caution is advised

For many, filing a tax return is about as pleasant as dental treatment without anesthesia. However, the effort often pays off. In recent years, the average tax refund in the US was around $3,000. Enough for a vacation, home improvements, or paying down debt.

Today, more and more people are turning to AI chatbots to prepare their taxes. According to the D21 Digital Index 2024/25, AI has become a staple for general research with 25% trusting ChatGPT as a reliable tool for looking up information.

A survey conducted by the Germany-based tax platform BuchhaltungsButler in collaboration with Berlin-based data studio DataPulse Research shows just how far this trust extends: Almost half of those surveyed can imagine replacing their own tax advisor with ChatGPT or a similar AI tool. This applies particularly to younger generations and high earners.

Online forums like Reddit are chock-full of anecdotal experiences using the tool, with some raving about it, and others warning against it. In subreddits like r/tax or r/personalfinance, proponents report that AI support saved them time and money when preparing tax returns.

However, other people have had mixed experiences. Hernan Barahona, an expat freelancer living in Germany who prepared his tax return with AI support, explains that ChatGPT helped him to better understand the basic terms and structures of foreign tax law. Also, as a non-native German speaker, he had many things explained to him in English. However, ChatGPT sometimes lost the thread over long conversations and was prone to oversimplifying nuanced information. Relying solely on the AI almost cost him more than 4,000 dollars.

The bottom line: Enlisting AI support for tax returns might be helpful, but it should also be used with caution.

So, how does one sensibly use AI for tax returns? Maxin Schneider, CEO of BuchhaltungsButler, explains the biggest risks and misconceptions.

One of the most frequent errors when dealing with AI is taking its responses as gospel. But ChatGPT does not provide vetted explanations, like an encyclopedia or a dictionary; instead, it generates text based on probabilities of words occurring together in a phrase. This is exactly why so-called „hallucinations“ occur: The AI can articulate false information so convincingly that it sounds like a fact.

Because tax law depends not only on raw numbers but also on the correct interpretation of legal terms and regulations, misinformation is an inherent risk. A recently published study by EBU, an alliance of public service media, revealed alarming error rates: In a systematic test regarding AI-generated responses, 20 percent of the statements were erroneous, partly outdated, or partly invented. Although the study concerned journalistic content, the mechanism is the same for just about any subject, including taxes: Where interpretation is required, uncertainty looms.

This becomes particularly tricky when the chatbot cites court rulings or other seemingly authoritative sources. Whoever reads those will automatically believe those citations are true. Yet often enough, the citation — or even the source, itself — does not exist at all. A well-known example happened in June 2023, during legal proceedings in the Southern District Court of New York when lawyers submitted a brief that cited several court decisions generated by ChatGPT which turned out not to exist at all. The judge ultimately sanctioned the attorneys and noted that parts of the AI-generated „analysis“ were essentially gibberish.

Many users assume that ChatGPT is providing the most recent information available. But that is not true and can become expensive when doing a tax return.

All ChatGPT models are trained on a set body of information. This information is vast, but it’s not always up to date. The latest information only appears if the model actively searches the web and finds the most current information. Depending on the query, the AI either uses its stored knowledge or fetches current information, if the feature is available. Which sources are included is not always transparent.

However, in tax law, it’s imperative to be up to date on new policies stemming from one-off tax reforms as well as the provisions that change on a set schedule (such as the standard deduction, which is adjusted annually for inflation). Anyone working with outdated guidance risks making errors.

Anyone using ChatGPT has probably seen the many community and custom GPTs that are tailored for a specific purpose. Some even promise help with U.S. taxes. If you type „tax return“ into the GPT Store, you’ll quickly find plenty of them. Many of these models look professional at first glance.

However, that doesn’t change the core problem: ChatGPT, whether the standard model or a custom GPT, is a general-purpose language model. It was not specifically trained on tax law and it has no direct access to professional legal or tax research databases such as Westlaw, LexisNexis, Thomson Reuters Checkpoint, or CCH.

In contrast, professional accounting or tax software systems utilize specialized Tax AIs. These access verified legal texts, rulings, administrative guidelines, and commentaries, and can therefore not only reference exact tax provisions but also correctly explain their application. Furthermore, they recognize erroneous entries or missing mandatory fields in the tax return.

ChatGPT replaces neither a calculator nor a spreadsheet. Although the most recent models calculate significantly more reliably than earlier ones, errors still repeatedly occur with longer or more complex calculation paths. ChatGPT is prone to skipping steps and making mathematical errors that can impact the bottom-line figures. This can become problematic with any mathematical task, but especially with a tax return, where amounts must be precise.

Those using AI for calculations should clearly formulate how the calculation is to be done. An example would be: „Please calculate step by step and check the result“. It also helps to have the complete calculation path displayed so that a human can double-check the formulas and the results.

„You just have to write the right prompt!“ This sentence has become a mantra when it comes to AI. And yes: A clearly formulated prompt helps enormously in getting better answers. But with a tax return, good prompting alone is not enough. In order to give an AI chatbot the right information in the first place, you yourself need a certain basic understanding of your tax situation: What types of income do you have? What counts as a business expense? Which lump sums currently apply?

Tax law is complex and full of special rules. Even in seemingly simple cases there can be exceptions that one must know in order to account for them correctly.

So, anyone who believes they can generate their entire tax return with a single „super prompt“ will quickly hit a wall. It is better to work in stages, ask follow-up questions, and feel your way step-by-step to the suitable solution. And above all: remain critical.

Using AI for tax savings sounds tempting, but it is also more complicated than many people think. The reason for this is that tax optimization is not a single action, but a broad strategy. It is about strategically linking different incomes, deduction possibilities, and life circumstances and coming up with the best scenario that factors in all variables. It therefore requires a certain overall understanding of the filer’s unique financial situation and the ability to recognize complex tax regulations and to use them most effectively.

For example, losses from a sole proprietorship or other pass-through business can often be used to offset other income, such as wages or rental income, but limits and rules may apply. In contrast, investment losses face tighter caps. Certain home improvements made for medical reasons may be deductible, while others are not, and business owners who time investments for tax purposes must navigate complex depreciation rules and incentives. In other words, choices about when and how to spend or donate money, or invest, or sell assets affect how different tax rules interact with each other in a given year and over time. ChatGPT has no full view of your overall situation so it cannot weigh the trade-offs or understand how one decision reshapes the rest of your financial picture, particularly over a longer period of time.

In short: Tax optimization is not the same as a tax trick. It requires a holistic overview and detailed personal knowledge.

A tax return follows clear structures: There are many mandatory fields and cross-references, where the answer to one section may impact the results of another. This is exactly where ChatGPT reaches its limits: The AI knows no mandatory fields, checks no plausibility, and does not recognize contradictions either. It also cannot import information that the IRS has on file (such as withholdings) into the proper fields, the way tax software systems often can.

ChatGPT cannot recognize, for example, whether an entry is relevant or useful for calculating taxes. Tax filers often enter amounts that seem necessary but have no tax impact, such as itemizing deductions even though they take the standard deduction, which is a single fixed-dollar amount.

While commercial U.S. tax software (including those affiliated with IRS Free File) automatically flag missing entries or inconsistencies through built-in validation rules, ChatGPT processes all inputs purely as text. The model cannot reliably determine whether information is complete, accurate, or even permissible.

Added to this is an often-overlooked risk factor: ChatGPT does not recognize so-called „negative signals,“ i.e., entries that lead to higher probabilities of inquiries or an audit from the government. Certain red flags like large expenses with low income, or special foreign declarations, or unusual deductions can trigger an audit.

In more complex cases, this risk becomes even greater: Anyone who has income from various sources, works abroad, or claims special allowances cannot rely on a chatbot to give extra attention to their unique circumstances.

ChatGPT often seems harmless in everyday life, like a smart, chatty friend. Yet behind the tool stands a US-American company that processes and stores the entered data and is allowed to use parts of it for the further development of the AI and the growth of the company. What exactly happens with your data is not always transparent for users.

This is a risk, especially with sensitive tax data. Information regarding income, assets, children, or health costs should not simply be transmitted to a system where it is unclear where and how the data is stored, and whether it is made accessible to third parties.

Nevertheless, there are some ways to limit the data that ChatGPT can trace and store. Those who use the tool without logging in remain fundamentally more anonymous; however, functions such as chat history, file upload, and access to specialized GPTs aren’t available. In the Pro version with login, the use of one’s own inputs for model training can be deactivated in the settings. Furthermore, saved conversations can be deleted regularly. Important to know: These protection mechanisms must be set manually.

ChatGPT can explain a lot, but it cannot submit a tax return. The model has no technical interface to IRS systems or commercial tax software. Anyone working with ChatGPT must later transfer their information manually into their filing system, whether that’s TurboTax, H&R Block, or other tax software, including all return forms and schedules.

Certified U.S. tax software, by contrast, is built to connect securely to IRS e-file systems. Data is transmitted electronically, often with built-in validation checks, plausibility warnings, and automatic archiving. Some providers can even prefill parts of the return by importing data directly from employers or financial institutions, but the taxpayer still has to review everything carefully.

Even if ChatGPT formulates with great certainty, in the end, you alone are liable for the accuracy of the information in the tax return. Neither OpenAI nor anyone else is liable for errors or false recommendations. This is also confirmed by a look at the terms of use which state that the results must be checked independently by the user, especially before they are used further or transmitted to third parties (like the tax office).

While U.S. policymakers are discussing AI regulation and transparency requirements, none of these efforts change the fundamental legal principle: the taxpayer is responsible for their own return, whether it was prepared manually, with tax software, or with assistance from an AI model.

Is ChatGPT totally useless at tax time or a real replacement for the tax advisor? Probably neither one nor the other. For very simple filings, the AI tool can certainly be a good helper to find one’s way through the tax jungle.

Hernán Barahona also comes to this conclusion; he plans to continue using AI in the future. However, always with the knowledge that professional support remains crucial to guarantee accuracy and conformity.

From an expert perspective, this is also true: Anyone working with ChatGPT should assess the role of the AI realistically.