Is ChatGPT taking over technical accounting?
BACKGROUND
ChatGPT is a chatbot developed by OpenAI Inc. and launched in November 2022. OpenAI Inc. is headquartered in San Fransico, started as a non-for-profit business in 2015, transitioned to a for-profit status in 2019. Currently, ChatGPT is offered in two options, i.e., a free plan and a paid plan for $20/month. Paid plan provides faster response, priority access to new features. Access is provided at openai.com/blog/chatgpt.
The chatbot or a chatterbot simulates interactive human conversation. It can also generate response to specific questions related to various specialized field of knowledge, including medical, legal, accounting and other fields. One of a more controversial use of ChatGPT was by students when doing their homework.
ChatGPT can be integrated into other software application through an API for additional fees.
Currently, ap. 49% of Open AI is owned by Microsoft, following reported $ 10 billion investment in January 2023. Another 49 % is said to be owned by Andreessen Horowitz, Calif based VC firm.
The purpose of this publication is to assess ChatGPT ability to access and retrieve accounting information, answer specific technical accounting questions.
Analysis
We performed the review of ChatGPT capabilities by asking specific accounting questions and analyzing provided answers. Our analysis of the answers was focused on their precision, accuracy and overall relevance. We divided our question in three categories:
- Access to authoritative guidance, other sources of accounting knowledge
- General accounting knowledge and definitions
- Practical questions, fact patterns
Questions asked, ChatGPT provided answers and proposed corrections are provided in Appendix A, Questions, Answers, Corrections.
Overall, we noted that ChatGPT provides accurate answers to practical question involving specific fact patterns, e.g., reporting of warrants attached to preferred stock. ChatGPT was also able to determine which cash redemption features embedded in a warrant would result in liability classification of a warrant and which would not. However, in one question involving determination of a lease commencement date, ChatGPT provided an incorrect answer. In another case, involving determination of functional currency of a foreign subsidiary, the bot did not provide the correct answer either.
We noted ChatGPT lack of ability to provide specific and accurate U.S. GAAP definitions such as definition of a derivative financial instrument. However, the bot was able to more or less accurately describe GAAP notion of a “common control”.
We noted ChatGPT is seemingly unable to access open resources as such authoritative accounting guidance published at asc.fasb.org and accounting guide published by Big Four accounting firms.
We believe that whenever the bot did not provide an accurate answer, the answer provided still gave a good introductory understanding of a relevant subject. For example, even through the bot was not able to accurate describe when a reporting entity should adopt liquidation basis of accounting, it described general terms of its application.
Conclusions
Out of 13 accounting questions, the bot correctly answered 4 or ap. 31%. We believe that accuracy and precision of bot answers are as such that a professional accountant should not rely on the answers without additional research. At the same time, bot answers to some technical accounting questions were surprisingly accurate.
Microsoft has indicated that it will allow business to create their version of ChatGPT. We will see if any specific versions will be created to handle complicated GAAP and other specialized accounting questions.
For the full text please refer to the attached PDF file.
