Trained an LLM to predict "What will Trump do?"
Details
- External ID
- 47090597
- Source
- HN
- Company
- —
- Product
- Trained an LLM to predict "What will Trump do?"
- Website domain
- huggingface.co
- Launched
- Feb. 20, 2026
- Cohort
- —
- Upvotes
- 10
- Upvotes percentile
- 0.5316711590296496
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:25 p.m.
- Updated at
- Sept. 7, 2026, 9:25 p.m.
Description
Hey HN! I RL-tuned an open-source LLM (gpt-oss-120b — 120B MoE, but only 5.1B active params) to predict "What will Trump do?" in any situation, trained on nothing but public news collected automatically from search queries. The trained model beats GPT-5, and both dataset and trained model are open sourced.Data generation: Generated 2,108 binary forecasting questions from just a search query and a date range using the Lightning Rod SDK (https://github.com/lightning-rod-labs/lightningrod-python-sd...). Questions are generated from historic news articles — like "Will Trump impose 25% tariffs on Mexico by March 1?" — and resolved by checking what actually happened after the deadline. No human annotation — the whole pipeline is automated.Training: GRPO with Brier score as the reward signal. LoRA rank 32, 50 training steps.Results: Slight accuracy edge over GPT-5 (Brier 0.194 vs 0.200), but big gains in calibration — the RL-tuned model produces much better probabilities (ECE 0.079 vs 0.091).Dataset: https://huggingface.co/datasets/LightningRodLabs/WWTD-2025This is a fully automated way to spin up domain expert LLMs from public web data with just a few search queries, no labeling/annotation required.I’d love any feedback, or suggestions for what domain expert to train next!
Enrichment
- Theme
- ML inference and model optimization
- Vertical
- Horizontal
- Function
- Agent / copilot
- Audience
- B2C
- AI stance
- AI-native
- Project type
- Hobby / open-source project
- Normalized one-liner
- llm trained to predict political actions
- Manually corrected
- False
Could you build this?
No Fine-tuning a 120B parameter MoE foundation model using reinforcement learning (GRPO/LoRA) requires massive GPU compute clusters and specialized ML engineering skills.
What it would actually take: The system requires an automated data pipeline scraping and synthesizing news events into forecasting question-resolution pairs, followed by distributed RL training (e.g., GRPO/PPO with PEFT LoRA) on a 120-billion-parameter MoE model like gpt-oss. The hard parts include managing distributed multi-GPU cluster infrastructure (vLLM/Deepspeed/Megatron), avoiding reward hacking in the LLM policy, and calibrating probabilistic forecasts. This demands specialized research-level machine learning engineering and substantial compute budgets.
Discussion
2 comments analyzed.
Competitors mentioned: GPT-5
Competitors
Other products that read as similar to this one — 46 launches clear the similarity bar, closest 8 shown.
Attention rank: #27 of 47 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 108 days after the earliest competitor.
- Open-source LLM and dataset for sports forecasting (Pro Golf) · hn · 2026-02-24 · 7 upvotes · similarity 0.52
- Open-source AMDGCN kernels for optimizing LLM inference · hn · 2026-08-25 · 5 upvotes · similarity 0.37
- Foresight by Lightning Rod · ph · 2026-06-30 · 246 upvotes · similarity 0.37
- AI Timeline · hn · 2026-02-23 · 174 upvotes · similarity 0.36
- FutureSearch, AI forecasting you can verify · hn · 2026-08-03 · 11 upvotes · similarity 0.36
- We built an AI tool for working with massive LLM chat log datasets · hn · 2025-11-19 · 16 upvotes · similarity 0.36
- I built a tiny LLM to demystify how language models work · hn · 2026-04-06 · 915 upvotes · similarity 0.36
- predictive-analytics-r · github · 2026-09-27 · 8 upvotes · similarity 0.35
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a agent / copilot tool for Agriculture yet.