• mlfh@lm.mlfh.org
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 day ago

    I’m working on this exact problem at home now, and finding 24GB of ddr4, a 6th-gen i7, and an old 8GB rx580 is more than enough to run pretty capable models up to ~35B parameters just fine, and equivalent MoE models shockingly fast. Not exactly cheap, with the price of everything so crazy right now, but certainly cheaper altogether than I’d imagined.