Once you’ve trained your large language model on the entire written output of humanity, where do you go? Here’s Ilya Sutskever, ex-OpenAI, admitting to Reuters that they’ve plateaued: [Reuters] The…

I’m usually the one saying “AI is already as good as it’s gonna get, for a long while.”

This article, in contrast, is quotes from folks making the next AI generation - saying the same.

  • brucethemoose@lemmy.worldEnglish
    1·
    3 hours ago

    Llama 3.1 is not even a “true” distillation either, but its kinda complicated, like you said.

    Yeah Qwen undoubtedly has synthetic data lol. It’s even in the base model, which isn’t really their “fault” as its presumably part of the web scrape.