• @[email protected]
    link
    fedilink
    English
    82 months ago

    they built a model specifically to work well on the benchmarks.

    To be fair, I’m pretty sure that’s what everyone is doing. If you’re not measuring against something, there’s no way to tell if you’re doing anything at all.

    • @[email protected]
      link
      fedilink
      English
      12 months ago

      My point was a mixture of Experts model could suffer from generalization. Although in reading more I’m not sure if it’s the newer R model that had the MoE element.