• Toes♀
    link
    fedilink
    English
    220 days ago

    Would 256GB/s be too slow for large llms?

      • @[email protected]
        link
        fedilink
        English
        219 days ago

        Many LLM operations rely on fast memory and gpus seem to have that. Even though their memory is soldered and vbios is practically a black box that is tightly controlled. Nothing on a GPU is modular or repairable without soldering skills(and tools).