Toutes les curations

Making novel variations of evals is very useful, public evals tend to get memorized!

I see some are drawing the wrong inference from this study: they are focusing on this meta benchmark as yet another benchmark for model picking, i.e. open vs closed models, model families etc. That’s too narrow of an interpretation! You need to do your own private evals on

holy shit, an art form of scientific grift next METR/Epoch/AA employee right here you all need to work on your epistemic hygiene

GPT-5.6 Sol and Luna were built by roughly the same people using roughly the same infra, I don't think we can blame the 20 point difference in Luna just on memorization. There's something else being tracked here in addition to memorization. Maybe MaxxBench questions are slightly

Pre-announcing the pre-announcement

1000 scientific breakthroughs drop from Google incoming?

Been working for this moment for over a decade. What I've witnessed behind the scenes, together with what we have been building, tells me we are about to enter a truly incredible era of AI-powered scientific discovery. It's an incredible time to be doing research. I will be

2026 is the turning point for humanity. Google DeepMind’s Pushmeet Kohli says AI is accelerating scientific discovery and that upcoming releases will offer a glimpse of the progress. He says the team also aims to help the wider scientific community use these tools responsibly.

What did he mean by this 👀

I’ve been actively on the hunt for non LLM based AI models and I’ve invested into one already Looking for totally new designs

i disagree with the first point, there is nothing that says that scaling up current transformer recipe is not enough for "intelligence". you take kimi K3 or your favorite open arch + some stability fixes, you scale it to 50T parameters, you do classical training end to end, you

We need a model that learns *in spite* of its inductive bias, not thanks to it. We have to stop being cross-entropy fetishists.

True! This is the era of the RL objective. The biggest economy on this earth (US stock market) runs on subscriptions. Subscriptions require vendor lock-in. Currently, the frontier is pushing “Variance” learning from customer data, like how Google won the search market. But I

The Bitter Lesson 2.0

The era of super interesting AI brain surgery is upon us We saw this with Evil Qwen where someone changed 2 parameters and altered the entire personality We are seeing it with Nous taking the power of open source models and removing the Chinese marching orders At the end of

nous shoulda just called it a wasian finetune and sidestepped all the criticism

I really love the full bandwidth transformer research - One of the beautiful works of art of this year! “For example, the Bytedance experiments suggest a variant of the Full Bandwidth Transformer might be comparable to a model of twice the size, implying significant savings in

AI can also DREAM now? Anthropic inspired from how our brains organize memories during sleep to build a memory system (similar to wikis) for Claude. agents can now “dream” about past experiences: revisit its transcripts, correct mistakes, and evolve its memory wiki when not in

Sounds like Notion because it is!

POV: you're Claude playing Pokémon. You've fallen off the same ledge three times. You write "I'm going to stop rather than keep looping." The test setup auto-replies: continue. So you keep walking, all the way back to the last town, and never make it. Most relatable thing an

Claude wrote "Mt Moon DONE" in its own Pokémon notes. It wasn't done. One wrong note, and it fell off the same ledge three times. Anthropic's "dreaming" pass checked the note against the old transcript and fixed it. That run reached Cerulean City. I'll take memory that checks

Anthropic proposes an answer to Wenfeng's question about continual learning: …have a smarter model write more .md files. It's impressive how far this goes, but I still believe that it's a crutch with a ceiling set by the pretrain's ability, just like RLMs are a context crutch.

Every day I appreciate DeepSeek's "there was a service outage, then we noticed it's better and faster, then we checked and saw a new fingerprint, and then checked and there were weights on Huggingface; a few hours later the X account posted benchmarks" PR strategy more

CHINA IS ABOUT TO DROP SOMETHING BIG AGAIN. Qwen just posted a poster with one Chinese line and left the last word blank. If you can't read Chinese, you missed the whole message. The line is from Zhuangzi, kids in China learn it in school: "In the northern sea there is a

Lets go: looks like Qwen 4 is coming today. Brave yourself, OpenAI. Another Open source flagship is dropping. you better compete with GPT-6.2 ;)

“There are fish in the north. Its name is…”

Qwen4 🥲

wtf, this is excellent and actually somewhat scary. Thinky did this on a tiny compute and token budget, without any fancy architecture or RL tricks. This is the same level of achievement as 1017. Maybe better. This is FINALLY what can be expected of top-tier ex-Frontier staff.

Thanks @arcprize! One of my favorite eval for monitoring RL health🫡