Grok and Anthropic calling more than two thirds of earnings correctly
Plus: 11 upcoming earnings and several upgrades to our system
Flat Circle measures the ability of language models to predict company earnings results. See our methodology for detail and disclaimers. If you haven’t already subscribed, join investors and engineers interested in LLMs+investment research here:
Key takeaways
After 22 earnings calls, Grok and Anthropic lead earnings accuracy, calling more than two thirds of earnings correctly with a mean return of 230 to 300 bps per earnings
The worst performing model is OpenAI’s o1, with a hitrate of 41% and a total return of -38%
We made significant improvements to Context provided to each model:
Now includes market commentary explaining key price moves during the quarter
We are running the two Contexts in parallel and will roll out once we’ve confirmed overall earnings accuracy is improved
OpenAI o1 exited preview mode, so we’ve switched over to the current o1-2024-12-17 model. This allows us to specify a reasoning_effort parameter, which we’ve set to “high"
Upcoming earnings
Recent earnings
Applied Digital (APLD)
Claude’s new Sonnet model correctly called SELL while its older Opus model called BUY. We discuss the differences in the reasoning in the prior post, where Sonnet correctly showed skepticism on the timeline of the Ellendale announcement. However, between when the SELL call was issued and the earnings release, the company announced an investment by Macquarie causing the stock to rise the day prior to earnings. The methodology uses only the trading day following earnings, that interim gain is not reflected here.
Citi (C)
The company posted stronger than expected markets and banking revenue, as well as a larger than expected buyback. Gemini and Grok each made solid points in their pre-earnings calls:
Gemini 2.0 Flash Experimental
The bank has also been able to close another consent order this quarter. Given this momentum, it is likely that the bank will meet or exceed expectations. The prior quarter stock decline was not driven by the bank's performance, rather, by an external regulatory matter, and this has been addressed.
Grok
Our analysis of Citigroup Inc.’s performance over the past four quarters supports a BUY recommendation… with consistent revenue growth across its core businesses, notably strong performance in services, banking, and wealth
Improvements to Context template
The system provides the same Context to each model for each call. This week, we added several improvements to the Context including:
Market commentary explaining key price during the quarter
We are testing the new Context in parallel and will roll it out once we’ve confirmed it improves overall accuracy across all the models.
Contrasting calls made with old vs new Context
JB Hunt (JBHT) - Reports tomorrow after market close
Claude 3.5 Sonnet - Old Context - SELL
While there are some positives like improving safety metrics and bid compliance rates, the pricing and volume headwinds appear likely to drive another quarter of year-over-year profit declines.
Claud 3.5 Sonnet - New Context - BUY
While pricing pressure persists, particularly in Intermodal contracts through H1 2025, the stock appears to be looking past this near-term headwind…The main risk is continued pricing pressure, but this appears largely priced in at current levels given management's consistent messaging on this headwind persisting through H1 2025.
Taiwan Semiconductor (TSM) - Reports tomorrow before market open
OpenAI o1 - Old Context - BUY
TSM's share price has underperformed the S&P by -1.84% during the quarter, indicating low market expectations. Historically, when TSM underperformed or only slightly outperformed the S&P during the quarter, the stock experienced a positive price reaction after earnings (e.g., +9.79% and +9.80% in previous similar quarters). Given this pattern, there is a strong likelihood that TSM's stock price will rise following the upcoming earnings, making it a good opportunity to BUY.
OpenAI o1 - New Context - BUY
TSMC’s AI-related revenue has surged to represent a mid-teens percentage of total sales this year, with management expecting it to triple and reach an even higher share next year. Strong demand for 3nm and 5nm smartphone and HPC/AI applications is driving utilization upward, lifting gross margins above 57%. Moreover, TSMC continues to pass along higher costs to customers while maintaining industry-leading technology and capacity at scale, reinforcing its pricing power. These factors, plus the company’s increased full-year revenue outlook (near +30% in USD terms), suggest TSMC’s shares are poised to close higher following earnings.
If you have feedback or would like to participate in this project, please reply to this email or reach out via X or LinkedIn.