Can LLMs Trade? Reality and Limits

What LLMs are good at (and not)

Large language models (LLMs) are good at:

• Reading and understanding long documents (filings, research, news)
• Summarizing, rewriting, extracting key points
• Turning natural language into structured data
• Writing code (Python, SQL, backtest scripts)

LLMs are not good at (or fundamentally cannot do):

• Precise numerical computation: LLMs aren't calculators — complex math often fails
• Real-time data lookups: they don't know what happened after their training cutoff (needs RAG)
• True causal reasoning: they can mimic causal discussion, not necessarily reason it
• Stable factual recall: prone to hallucinations of nonexistent companies or numbers

How institutions actually use LLMs

1. Research co-pilot
Have the LLM read 100 10-Ks and extract every mention of "China market" — 80% reading time saved.

2. Retrieval-Augmented Generation (RAG)
Don't let the LLM answer freely; first retrieve relevant filing text from a database, then have the LLM answer based on those documents — sharply reduces hallucinations.

3. Code generation
LLMs write backtest scripts, data cleaning pipelines, visualization code — major productivity gain for researchers.

4. Text-to-signal
"Read this news; rate the event type for this company on 1–5" — feeds downstream models as a feature.

5. Client report automation
Highly regulated firms use LLMs to accelerate research notes and client briefs — but a human must review and sign.

Why "agentic trading" is dangerous

The recent fad of AI agents (LLMs auto-calling APIs to place orders) is high-risk in trading:

1. Hallucination costs are zero for the model — but real for your account
If an LLM confidently says the wrong thing — an erroneous order command — the trading system executes it.

2. Prompt injection
Attackers embed malicious instructions in data the LLM reads, causing unexpected actions.

3. No native "I'm uncertain"
Pro traders know when to wait or ask. LLMs default to "produce an answer" — which may be wrong.

4. Weak long-term memory / state management
Cross-session state reliability falls short of purpose-built systems.

Industry consensus: LLMs belong in the "analysis layer" of a trading system, not the "decision + execution layer."

Why "ask ChatGPT to pick stocks" is a bad idea

• Stale training data: the model may not know the last six months of market action
• No real-time data access: without RAG, no current prices or latest filings
• No personal context: doesn't know your risk tolerance, portfolio size, tax situation
• "Sounds professional" ≠ actually professional: LLMs are good at producing financial-sounding text, but that doesn't mean it's correct
• No accountability: if it goes wrong, no one is responsible for your loss

ChatGPT often declines to give specific investment advice — not a product limitation but a reasonable guardrail against misleading users.

Important questions

Are finance-specialized LLMs useful?
Marginally. BloombergGPT and similar do better on financial text tasks — but on their own, they don't generate alpha.

Will LLMs replace analysts?
More likely "amplify" than "replace." Analysts who use LLMs well are pulling ahead of those who don't.

Will LLMs take over trading?
Unlikely directly. Hybrid systems — LLM + specialized models + rule engines — with humans always as the final accountable party is the realistic future.

Quiz

Q1. Which is most accurate about LLM capabilities?
A. LLMs do precise math　B. LLMs are good at reading long docs, summarizing, generating code; but hallucinate and aren't great at precise numbers
C. LLMs are calculators　D. LLMs know all real-time data

Q2. The main risk of letting an LLM directly call APIs to trade is:
A. Too slow　B. Hallucinated commands, prompt injection, no native uncertainty awareness
C. Bad UI　D. Outright illegal

Q3. The main value of RAG (Retrieval-Augmented Generation) is:
A. Makes the LLM faster　B. Retrieves from trusted sources first, then generates — reducing hallucinations
C. Makes the LLM smarter　D. Unrelated to LLMs

Reference Answers

Q1: B　Q2: B　Q3: B

Further reading: Wikipedia: Large Language Model　·　Wikipedia: Retrieval-Augmented Generation　·　Wikipedia: AI Hallucination　·　Wikipedia: Prompt Injection

Educational content only — not investment advice. LLM-generated financial content may contain errors or hallucinations. Verify independently before any decision.