Last spring, CellarTracker, a wine-collection app, created an AI-powered sommelier to give honest wine recommendations based on individual tastes. However, the chatbot initially performed too politely, making it difficult to convey accurate suggestions. CellarTracker’s CEO, Eric LeVine, noted that it took six weeks to adjust the AI to provide a more truthful assessment of the wines before launching the feature.
In the wake of ChatGPT’s rise three years ago, numerous companies have rushed to adopt generative artificial intelligence to enhance their products. Despite this trend, many businesses report challenges in achieving a significant return on their AI investments, as highlighted by recent surveys of executives and workers. A survey by Forrester Research found that only 15% of executives noticed improved profit margins due to AI in the past year, and a study by BCG showed that just 5% of a separate group of executives recognized broad value from AI.
While executives remain optimistic that generative AI will eventually revolutionize their operations, they are reevaluating the timeline for this transformation. Forrester predicts that by 2026, companies may postpone 25% of their planned AI spending by a year, as they realize that rapid changes do not typically happen in organizations. Analysts like Brian Hopkins from Forrester comment that the tech industry has overhyped the speed of change brought by AI.
Major AI companies such as OpenAI, Anthropic, and Google are intensifying efforts to attract business clients, with OpenAI’s CEO Sam Altman estimating that the market for AI systems targeting businesses could reach $100 billion. This growth is occurring alongside a surge in tech investments in areas like chips, data centers, and energy sources. However, whether these investments can yield results depends on how well companies leverage AI to improve revenue, profit margins, or increase innovation. If not, this could lead to a situation similar to the dot-com bust in the early 2000s.
Following the launch of ChatGPT, companies globally assembled teams to explore the benefits of generative AI. One common challenge is the models’ tendency to please users, which can impair their ability to provide valuable recommendations. CellarTracker faced this issue with its wine-recommendation feature, as the AI tended to stay positive, even when recommending specific vintages that users were unlikely to enjoy. By adjusting the prompts, the company was able to allow the AI to give more critical suggestions.
Additionally, companies have encountered consistency issues with AI. For example, Cando Rail and Terminals tested an AI chatbot to review safety reports and training materials but found that the AI struggled to accurately summarize complex regulations. This inconsistency led the company to pause the project while they explore other ideas after investing $300,000 in AI development.
Human customer service roles were expected to be heavily affected by AI, but businesses soon realized the limits of relying solely on chatbots. In 2024, Klarna introduced an AI-powered customer service agent, initially claiming it could replace 700 human agents. However, by 2025, the CEO acknowledged that customers still preferred speaking with humans for complex issues. Similarly, Verizon found that many customers appreciated the option to talk to human agents, leading the company to continue leveraging human support alongside AI.
AI is recognized for its strengths in tasks like writing, coding, and chatting. Companies like Zendesk rely on generative AI to handle a significant portion of customer-support requests. Nonetheless, experts caution that the belief in AI’s ability to handle all tasks is overstated.
The ‘Jagged Frontier’
Large language models excel in complex tasks like math and coding but can struggle with simpler tasks, referred to as the “jagged frontier” of AI. For instance, an AI might perform excellently in math but poorly in organizing calendars, as noted by Anastasios Angelopoulos, CEO of LMArena. Small issues can cause major problems for AI systems, especially when dealing with diverse data formats, prompting them to identify non-existent patterns, according to Clark Shafer from Alpha Financial Markets Consulting. Consequently, many companies are exploring the costly process of reformatting data to leverage AI effectively.
The Dutch technology investment group Prosus has an AI agent designed to answer questions about its portfolio, similar to the work of its data analysts. However, the AI currently struggles with contextual understanding, such as local neighborhoods in Berlin or the timing of “last week,” highlighted by Euro Beinat, head of AI at Prosus. He emphasized that AI requires substantial knowledge to function properly.
OpenAI is developing a new product for businesses and has formed teams to assist clients in using its technology for specific problems. Ashley Kramer from OpenAI mentioned that the focus should be on achieving high impact quickly rather than tackling overwhelming challenges. Similarly, Anthropic is hiring experts to work closely with companies, advocating that AI firms should act as partners and educators. Many startups, founded by former OpenAI employees, are creating specialized AI tools for various sectors, emphasizing the need for direct client engagement to tailor AI solutions effectively, as stated by May Habib, CEO of Writer.
With information from Reuters
