Microsoft-CTO Kevin Scott glaubt, dass die „Skalierungsgesetze“ des LLM trotz Kritik Bestand haben werden | „Was auch immer andere Leute denken, wir sind nicht auf der Seite der abnehmenden Grenzerträge bei der Skalierung“, sagte Scott. „Und ich versuche, den Leuten klarzumachen, dass es sich hier um eine Exponentialfunktion handelt.“

„I try to help people understand there is an exponential here, and the unfortunate thing is you only get to sample it every couple of years because it just takes a while to build supercomputers and then train models on top of them.“

LLM scaling laws refer to patterns explored by OpenAI researchers in 2020 showing that the performance of language models tends to improve predictably as the models get larger (more parameters), are trained on more data, and have access to more computational power (compute).

The laws suggest that simply scaling up model size and training data can lead to significant improvements in AI capabilities without necessarily requiring fundamental algorithmic breakthroughs.“

Hey look another rich old man crawling out of the shadows to spout opinions that affect his bottom line