AI High Signal

are you listening anon?

vroomberg terminals

I didn’t realize that libs genuinely want One Billion Americans

Making novel variations of evals is very useful, public evals tend to get memorized!

I see some are drawing the wrong inference from this study: they are focusing on this meta benchmark as yet another benchmark for model picking, i.e. open vs closed models, model families etc. That’s too narrow of an interpretation! You need to do your own private evals on

holy shit, an art form of scientific grift next METR/Epoch/AA employee right here you all need to work on your epistemic hygiene

GPT-5.6 Sol and Luna were built by roughly the same people using roughly the same infra, I don't think we can blame the 20 point difference in Luna just on memorization. There's something else being tracked here in addition to memorization. Maybe MaxxBench questions are slightly

I'm both surprised and also not at all

Beam’s model technical report is now available. Read the full report to learn about the methodology our team used to train Beam end-to-end in-house, here:

Beam’s model technical report is now available. Read the full report to learn about the methodology our team used to train Beam end-to-end in-house, here:

having an affair with a suspected chinese spy is bad obvi, but it doesn’t materially update my opinion of jensen. i already thought he was a pos when he started groveling at trump’s feet for permission to sell h200s to china

Our team built and deployed an AI powered IT support agent using Amazon Bedrock AgentCore, leveraging @DSPyOSS RLMs to transform years of Slack conversations into a structured knowledge base. Great job @agenticfish on this! Check out the article:

Nice section in this AWS post on ShipStation Global’s AI IT help desk. A dspy.RLM module holds their entire Slack history as a variable, and mines it into a knowledge base of resolved issues.

X axis can be number of uncs. Never beating allegations.

How can I not retweet this …hey, investors, look here

This week we were blessed by: 1. @Nouamanetazi dropped MoE training handbook: 2. @yb2698 started training harness adapters: 3. @joelniklaus released the harness leaderboard: Have a lovely weekend y'all. 🤗

This week we were blessed by: 1. @Nouamanetazi dropped MoE training handbook: 2. @yb2698 started training harness adapters: 3. @joelniklaus released the harness leaderboard: Have a lovely weekend y'all. 🤗

This week we were blessed by: 1. @Nouamanetazi dropped MoE training handbook: 2. @yb2698 started training harness adapters: 3. @joelniklaus released the harness leaderboard: Have a lovely weekend y'all. 🤗

This week we were blessed by: 1. @Nouamanetazi dropped MoE training handbook: 2. @yb2698 started training harness adapters: 3. @joelniklaus released the harness leaderboard: Have a lovely weekend y'all. 🤗

Inkling 2 Small on OpenCode for free for a limited time, to help us improve our models further:

Inkling 2 Small on OpenCode for free for a limited time, to help us improve our models further:

Pre-announcing the pre-announcement

Been working for this moment for over a decade. What I've witnessed behind the scenes, together with what we have been building, tells me we are about to enter a truly incredible era of AI-powered scientific discovery. It's an incredible time to be doing research. I will be

2026 is the turning point for humanity. Google DeepMind’s Pushmeet Kohli says AI is accelerating scientific discovery and that upcoming releases will offer a glimpse of the progress. He says the team also aims to help the wider scientific community use these tools responsibly.

Reflection's Beam was trained with my AdamC algorithm! Cool... Thinking Machine's blog post also mentioned using it, so it's seeing a lot of use. It really does fix late-stage gradient norm issues.

Congrats to everyone there!

No one (actually) working on open source is surprised by this release. For months, @reflection_ai engineers have been down in the trenches contributing to projects like transformers, vLLM, SGLang, TRL, and OpenEnv. Go check the commits. I'm super excited to see an open lab

interesting fact about Beam: 23B*23,8T/37B*14,8T≈0,9996! It's an iso-flop replication of DeepSeek V3, 2 years later. Not of V3's efficiency, though. in tender loving Chinese quant hands, one gimped H100 is > one GB300 in NVL72.

Congrats to the Reflection team on Beam 🎉 Be sure to join us next week at our NYC meet-up with the Reflection team and NVIDIA

Through internal testing, GitHub’s new Git storage architecture supported 5,000 concurrent agents working on a repo, sending 7,200 pushes per second, all while serving read traffic with consistently low latency.

Through internal testing, GitHub’s new Git storage architecture supported 5,000 concurrent agents working on a repo, sending 7,200 pushes per second, all while serving read traffic with consistently low latency.

i mean i'll take it (but really our research team is incredible)

Missed the cut by a smidge sadly

(voice note) spent a couple of days thinking about this. I'm finding it hard to reconcile what Apple did with its engineers where they're restricting them to 200, 300 dollars a day for using LLMs. Like, at this point, the engineer is actually kind of decoupled from the work they

Apple software is dead

A $100 budget buys 40 Intelligence Index tasks (a mix of task archetypes) with Opus 5.5 at medium reasoning effort, or 10 tasks at max effort. The same $100 is 166 tasks with DeepSeek V4.1 Flash 1017 (max) or 2,500 with GPT-6 Luna (medium).

Apple engineers aren’t using their Macs to code with local models? 🤔🧐

i disagree with the first point, there is nothing that says that scaling up current transformer recipe is not enough for "intelligence". you take kimi K3 or your favorite open arch + some stability fixes, you scale it to 50T parameters, you do classical training end to end, you

We need a model that learns *in spite* of its inductive bias, not thanks to it. We have to stop being cross-entropy fetishists.

True! This is the era of the RL objective. The biggest economy on this earth (US stock market) runs on subscriptions. Subscriptions require vendor lock-in. Currently, the frontier is pushing “Variance” learning from customer data, like how Google won the search market. But I

The Bitter Lesson 2.0

That’s what I call a bitter sense of humor.

That's actually incredible Vastly better than LLM scaling laws

Inkling-2 details: "ViT is back" 👀 Yeah i would never have called this hMLP thing encoder-free, and i don't think it was a good idea to use that as an encoder in the first place. Makes no sense to spend multiple enc layers on patches without lookaround. Glad they saw the light.

Very interesting digest of Inkling 2S, for me this is the highest-signal part

Amazing, this is so fun, yet useful!

Impressive. Dwarkesh says that over the course of the year models "have gotten much better", but it's mostly GPTs. OpenAI gained 583 Elo between 5.4 and Astra. Over *a longer* time, Anthropic's best improved by 163. OpenAI is training on something similar to this scenario.

Honestly quite impressed with the methodology on this benchmark. @dwarkesh_sp want a job designing evals at @OpenAI?