AI High Signal

i disagree with the first point, there is nothing that says that scaling up current transformer recipe is not enough for "intelligence". you take kimi K3 or your favorite open arch + some stability fixes, you scale it to 50T parameters, you do classical training end to end, you

We need a model that learns *in spite* of its inductive bias, not thanks to it. We have to stop being cross-entropy fetishists.

True! This is the era of the RL objective. The biggest economy on this earth (US stock market) runs on subscriptions. Subscriptions require vendor lock-in. Currently, the frontier is pushing “Variance” learning from customer data, like how Google won the search market. But I

The Bitter Lesson 2.0

Making novel variations of evals is very useful, public evals tend to get memorized!

I see some are drawing the wrong inference from this study: they are focusing on this meta benchmark as yet another benchmark for model picking, i.e. open vs closed models, model families etc. That’s too narrow of an interpretation! You need to do your own private evals on

holy shit, an art form of scientific grift next METR/Epoch/AA employee right here you all need to work on your epistemic hygiene

GPT-5.6 Sol and Luna were built by roughly the same people using roughly the same infra, I don't think we can blame the 20 point difference in Luna just on memorization. There's something else being tracked here in addition to memorization. Maybe MaxxBench questions are slightly

I'm both surprised and also not at all

Pre-announcing the pre-announcement

Been working for this moment for over a decade. What I've witnessed behind the scenes, together with what we have been building, tells me we are about to enter a truly incredible era of AI-powered scientific discovery. It's an incredible time to be doing research. I will be

2026 is the turning point for humanity. Google DeepMind’s Pushmeet Kohli says AI is accelerating scientific discovery and that upcoming releases will offer a glimpse of the progress. He says the team also aims to help the wider scientific community use these tools responsibly.

Reflection's Beam was trained with my AdamC algorithm! Cool... Thinking Machine's blog post also mentioned using it, so it's seeing a lot of use. It really does fix late-stage gradient norm issues.

Congrats to everyone there!

No one (actually) working on open source is surprised by this release. For months, @reflection_ai engineers have been down in the trenches contributing to projects like transformers, vLLM, SGLang, TRL, and OpenEnv. Go check the commits. I'm super excited to see an open lab

interesting fact about Beam: 23B*23,8T/37B*14,8T≈0,9996! It's an iso-flop replication of DeepSeek V3, 2 years later. Not of V3's efficiency, though. in tender loving Chinese quant hands, one gimped H100 is > one GB300 in NVL72.

Congrats to the Reflection team on Beam 🎉 Be sure to join us next week at our NYC meet-up with the Reflection team and NVIDIA

(voice note) spent a couple of days thinking about this. I'm finding it hard to reconcile what Apple did with its engineers where they're restricting them to 200, 300 dollars a day for using LLMs. Like, at this point, the engineer is actually kind of decoupled from the work they

Apple software is dead

A $100 budget buys 40 Intelligence Index tasks (a mix of task archetypes) with Opus 5.5 at medium reasoning effort, or 10 tasks at max effort. The same $100 is 166 tasks with DeepSeek V4.1 Flash 1017 (max) or 2,500 with GPT-6 Luna (medium).

Apple engineers aren’t using their Macs to code with local models? 🤔🧐

Amazing, this is so fun, yet useful!

Impressive. Dwarkesh says that over the course of the year models "have gotten much better", but it's mostly GPTs. OpenAI gained 583 Elo between 5.4 and Astra. Over *a longer* time, Anthropic's best improved by 163. OpenAI is training on something similar to this scenario.

Honestly quite impressed with the methodology on this benchmark. @dwarkesh_sp want a job designing evals at @OpenAI?

I really love the full bandwidth transformer research - One of the beautiful works of art of this year! “For example, the Bytedance experiments suggest a variant of the Full Bandwidth Transformer might be comparable to a model of twice the size, implying significant savings in

The End of AI Frontiers . There's a bit of a train wreck here that's painful to express. Nevertheless, it's a significant event for me and quite a few other folks involved so I wanted to memorialize a bit.

The End of AI Frontiers . There's a bit of a train wreck here that's painful to express. Nevertheless, it's a significant event for me and quite a few other folks involved so I wanted to memorialize a bit.

The End of AI Frontiers . There's a bit of a train wreck here that's painful to express. Nevertheless, it's a significant event for me and quite a few other folks involved so I wanted to memorialize a bit.

nous shoulda just called it a wasian finetune and sidestepped all the criticism

Sounds like Notion because it is!

Anthropic proposes an answer to Wenfeng's question about continual learning: …have a smarter model write more .md files. It's impressive how far this goes, but I still believe that it's a crutch with a ceiling set by the pretrain's ability, just like RLMs are a context crutch.

Fable playing Pokemon just like me FR Great post on memory from Dr. "Strictly Bangers" @RLanceMartin

Highly recommended read. Learn how one of the most advanced agent memory systems works. Your memory system is one of the biggest levers you have in the harness. I also don't think a one-size-fits-all memory system exists today. Memory is something you constantly have to

thom just took away karpathy’s mote. 🤍

wtf, this is excellent and actually somewhat scary. Thinky did this on a tiny compute and token budget, without any fancy architecture or RL tricks. This is the same level of achievement as 1017. Maybe better. This is FINALLY what can be expected of top-tier ex-Frontier staff.

Thanks @arcprize! One of my favorite eval for monitoring RL health🫡

Inkling is ENSOULED!

Kimi K3.1 Pro from @Kimi_Moonshot is live on Tinker today with 256K context. K3.1 Pro is the strongest open-weights model on several benchmarks of agentic coding and knowledge work, and is the largest model on Tinker today.

kimi is dropping open weights for a 3T param model and Mckinsey is still in month 7 of the copilot pilot

Congrats on the ProgramBench (@jyangballin @KLieret) results!! This is our 'almost resolved' metric. The competition is really heating up.

what a release, congrats to the kimi team!

on 10/24/2026, a present for the last programmer's day of humans

Pretty solid pelican

Yesterday, the first details regarding Anthropic's internal models were published. "Anthropic's model 4" is significantly better than its predecessors, and not just by a small margin. A clear jump in performance can be seen in this benchmark.

okay this was the closest I have been to a heart attack

Anthropic currently has Model 4. 🫠

Is this the largest ablation of harnesses and models ever performed?

neat

Hermes supremacy tweet #9138

mini-swe-agent is 130 lines of code and it keeps on winning in harness comparisons.

What an American open-weight frontier lab looks like. Inkling v1 -> v2 was a feat in itself. I can’t wait to see where v3 lands, and will be shoving 2 Small on the 6000’s once the weights drop

Inkling 2 small is an extremely impressive release, especially for its size. It is a Mixture-of-Experts model with 594B total parameters, 17B active, and supports image inputs and variable effort. In coding benchmarks, it easily keeps pace with significantly larger models. What

ok this is just ridiculous

Proud to release our new model, Inkling 2 Small. Check out our blog post -- it includes a lot of detail about our training process. To me, the most interesting part of this release was what we learned about alignment, which you can read about in the sections "Safety and

Inkling 2 small is very strong on vision perception, and also good at planing/coding/acting.

I haven’t seen Nouamane in person. for more than a year. Now I know why

Nouamane cooked again 🔥

After the Ultra Scale Playbook, Nouamane is back with this crazy guide to train MoEs! It's probably the most comprehensive resource if you want to get into large scale MoE training and an absolute gem packed with beautiful visualization!

📖 The Ultimate MoE Training Handbook 📖 We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.

📖 The Ultimate MoE Training Handbook 📖 We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.

📖 The Ultimate MoE Training Handbook 📖 We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.

Distilling most of our expertise in kernel and inference engine optimization into an agent loop that powers our production. A 2-year project that we're finally able to share

Together Compiler discovers the most efficient inference runtimes with continuous compilation for any combination of workload, model and silicon. Excited to share the details about the research and engineering that went into it. Together Compiler (or TKCC) is a production

Great work that pushes our understanding of how agents and experts can collaborate on optimizing system performance Beyond the strong results, the team also found some surprising cases where our human inductive biases end up limiting what agents can achieve

The Together Compiler (TKCC) from @simran_s_arora and her team is honestly one of the coolest things ever, and we have validated one of the InferenceX/AgentX results! The ability to continuously improve performance automatically while ensuring accuracy is great enough to quickly