Training a 1 trillion parameter model

Update: 2025-09-04

Description

Kimi K2 and Moonshot AI's history, avoiding loss spikes during training, the muon optimizer, and data parallelism

Comments

In Channel

A deep dive on OpenAI Atlas

2025-10-3101:13:09

The browser wars are just getting started

2025-10-2957:57

Are we in an AI bubble?

2025-10-2401:15:19

LLMs can get brain rot too

2025-10-2301:00:34

AMD is back in the AI chipset race

2025-10-1758:27

The inaugural listener mailbag

2025-10-1455:46

California legislators come for LLMs

2025-10-0901:07:53

Move over TikTok - a new feed's in town

2025-10-0701:02:24

Gen z struggles to find coding jobs fr no cap

2025-10-0251:49

The power of ten million deadlifters

2025-09-3001:06:27

How countries are actually using AI

2025-09-2551:18

Your new raybans just got smarter

2025-09-2301:00:54

The claude code conspiracy

2025-09-1953:50

Will McTighe on selling through social media

2025-09-1601:10:57

Eating some mooncake

2025-09-1233:55

Training a 1 trillion parameter model

2025-09-0443:08

Nano banana is our favorite fruit

2025-09-0250:38

A breakdown of Genie 3's world model

2025-08-2857:10

Using chat in history class

2025-08-2139:52

AI is your new favorite songwriter

2025-08-1949:51

00:00

Training a 1 trillion parameter model

Pierce Freeman & Richard Diehl Martinez

#box-pro-ellipsis-176222469638992{-webkit-line-clamp:2;}Training a 1 trillion parameter model

Training a 1 trillion parameter model

Pierce Freeman & Richard Diehl Martinez

Training a 1 trillion parameter model