How Distillation Makes AI Models Smaller and Cheaper | Quanta Magazine
www.quantamagazine.org
Fundamental technique lets researchers use a big, expensive “teacher” model to train a “student” model for less.

The Chinese AI company DeepSeek released a chatbot earlier this year called R1, which drew a huge amount of attention. Most of it focused on the fact that a relatively small and unknown company said it had built a chatbot that rivaled the performance of those from the world’s most famous AI companies, but using a fraction of the computer power and cost. As a result, the stocks of many Western tech…

Source

From Quanta Magazine via this RSS feed