top of page

Can AI meet our expectations for accuracy, and what happens when it falls short?

Updated: Jul 29

TL;DR

  • Big Tech Moderates AI Expectations: Leaders like OpenAI's CEO Sam Altman suggest that AI's current capabilities are less transformative than initially anticipated, warning of moderate impacts on short-term growth.

  • Risks with AI in Critical Sectors: The WHO's AI chatbot SARAH, delivering unreliable health information, demonstrates the perils of premature AI deployment in essential services.

  • Improving AI Accuracy: While there are setbacks, reports from Stanford University and performance metrics like that of GPT-4 indicate notable improvements in AI accuracy, hinting at future reliability in specialised tasks.

 

In early April, our blog focused on the trio of challenges that AI represented: accuracy, hallucinations, and data leaks. These elements are crucial in understanding the foundational stability and reliability of AI technologies in practical scenarios. If you haven't read it yet, you might want to take a glance at Addressing AI Challenges: Accuracy, Hallucinations, and Data Leaks to get a context on the issues discussed there.


Today, we delve deeper into how the AI landscape is evolving, particularly in terms of accuracy, and what it implies for small businesses. The remarkable growth of AI applications offers vast opportunities, but it still comes with nuanced challenges. Let's explore recent developments, starting with the sobering realities presented by influential tech leaders, through cautionary implementations such as WHO’s health chatbot, to advancements marking improved accuracies and capabilities of AI systems.

 

1. Big Tech's Realistic Outlook on AI Capabilities AI Accuracy and Expectations

A corporate meeting with executives discussing the tempered ea corporate meeting with executives discussing the tempered expectations of AI's impact on business growth.
Image generated by DALL-E, prompted by Janey Treleaven of Intelligence Assist

Recent remarks by Sam Altman, alongside observations from major tech companies, have highlighted a tempered expectation for AI-driven growth in the immediate future. Despite the initial excitement and bullish forecasts about AI's potential to revolutionize various sectors, the reality is more subdued. Altman himself has noted that while AI models hold promise, their current capabilities are not as advanced as public enthusiasm might suggest. He candidly remarked that the "reality is still pretty bad" and that these models are "not that good," indicating a significant gap between expectations and actual performance. AI Accuracy and Expectations


This sentiment is echoed by leaders from Amazon, Salesforce, and Microsoft, who have all indicated that AI's contribution to their revenue streams will remain modest in the near term. For instance, Amazon CEO Andy Jassy pointed out that AI-related revenue is "relatively small," and Microsoft reported that its Azure AI platform's revenue constituted only about 1% of the company's total expected revenue. This collective acknowledgment from Big Tech suggests a recalibration of expectations around AI, emphasizing that while AI will continue to improve, it is not yet the transformative force many had anticipated for 2024.

 

2. The WHO's AI Chatbot: A Cautionary Tale

The World Health Organization's introduction of the AI chatbot SARAH aimed to provide global access to health information on topics like mental health and healthy eating. However, the execution has not lived up to expectations. The chatbot has been reported to provide outdated information and, at times, completely inaccurate responses, a phenomenon known as "hallucinating" in AI parlance.


This issue underscores the risks associated with deploying AI in critical areas like healthcare without sufficient oversight and testing. The implications of such inaccuracies can be severe, especially when individuals rely on this information for health-related decisions.

 

3. Improvements in AI Accuracy

Despite these challenges, there is evidence that AI accuracy is on an upward trajectory. The Stanford University's AI Index report indicates that AI has surpassed humans in numerous performance benchmarks, suggesting significant advancements in AI capabilities. Furthermore, studies like the one showing GPT-4's performance surpassing human doctors in medical board exams hint at the potential for AI to exceed human expertise in specialized tasks.


A compelling illustration of this progress is seen in the realm of text-to-image generation, especially with technologies like Midjourney. For instance, an image showcasing the evolution in AI's ability to generate visuals of Harry Potter since 2022 reveals stunning improvements. The latest iterations offer remarkably realistic renditions, far surpassing the initial results. This leap in quality over merely 22 months not only showcases the current potential but also ignites the imagination about the future advancements in AI's creative capabilities. If such exponential improvement were to continue, we could expect even more breathtaking visuals and applications in the next 22 months, opening up new frontiers for creativity and automation in numerous sectors.


Midjourney generations over time: a hyper-realistic image of Harry Potter

These developments suggest a bright future for AI, with continuous improvements likely to expand its applicability and reliability across various domains.

 

Why This Matters to Small Businesses

For small businesses, particularly those in Australia, the evolving landscape of AI presents both opportunities and cautions. AI's potential to enhance productivity and efficiency is clear, with estimates suggesting boosts of 20-40% in areas such as coding, writing, and HR. However, the mixed performance of AI in more complex tasks and critical applications like healthcare serves as a reminder of the technology's current limitations.


Small businesses must approach AI as a long-term investment, recognizing that while the technology will continue to evolve and improve, its current use must be carefully managed. Relying on AI for tasks that require high accuracy and bear significant consequences if performed incorrectly should be approached with caution. Instead, AI can be leveraged effectively in areas where it can automate routine tasks, enhance productivity, and support human workers.


In conclusion, while AI holds transformative potential for small businesses, the journey there will be gradual. The key is to stay informed, adapt strategies based on realistic assessments of AI capabilities, and continue to integrate AI responsibly, ensuring it complements rather than replaces human expertise.

 

Want to Read More?

Detailed angles and expert insights on each section can be accessed through these citations:




13 views0 comments

Comments

Rated 0 out of 5 stars.
No ratings yet

Add a rating
bottom of page