Just for fun last week, I asked Google “How can I be sure my AI answer is correct?” and I think I DID get the correct answer: ”There is no definitive way for an AI to be 100% certain that its own answer is correct.”
Hmmm. If AI itself is telling me to beware of what it’s telling me, why do I put so much stock in its answers? And I suspect that I am not alone.
I notice that there’s less of a tendency to doubt digital information, but maybe if we understand the state of the art as it exists today that won’t be so. AI analyzes its database and picks a “most probable” answer. That database could include data that is no longer current or accurate. Worse yet, there is a pattern of trying to generate an answer that will give the inquirer the answer he wants. AI has been known to convincingly fabricate legal precedents for lawyers and recommend tax strategies that have been discredited.
AI is now part of the software we accountants use for tax research. Even though our software uses only authoritative sources, as a Firm we are cautioned to take any response a step further:
- Is it based on the latest legislation and rulings? Databases are not updated instantaneously. Is there news AI doesn’t know about yet? Where do we need to check to be sure we are not missing a critical update?
- All references have to be checked.
- What else needs to be confirmed before we can rely on our results?
I have sometimes gotten answers that just don’t make sense or are inconsistent that make me start all over, often with a critical eye using the source material AI has highlighted for me during my search.
All this is to advise you not to go to the mat with an unvetted AI strategy. Professionals are reporting that many clients will use AI results to trump their advice. My Ai query returned this advice that I urge you to follow if you want to take that position:
- Cross reference with authoritative sources.
- Verify citations.
- Evaluate plausibility.
- Look for inconsistencies.
- Use fact-checking tools.
- Check for multiple perspectives.
Seems like a lot of extra work to be able to rely on this amazing, brilliant, time-saving tool.
