DeepSeek's free 685B-parameter AI model runs at 20 tokens/second on Apple's Mac Studio, outperforming Claude Sonnet while using just 200 watts, challenging OpenAI's cloud-dependent business model.
I run deepseek-r1:14b locally, though it needs to go into RAM and runs slower its still a reasonably good speed. Keeps up with reading it. Should try a larger one at some point, but its quite a bit to download when you get to the larger ones. Usually run ~7b size as that can fit in VRAM and runs way faster.
Deepseek is an absolutely massive model, it’s not the one people will be running. Rather, look at qwen/qwq, gemma and a number of other smaller ones
No, people who want something approaching chatgpt but local want to run at least deepseek V3 32B.
Qwen at least fares much worse for my usage as do deepseek V3 under 32B.
The hell is v3 32b. Are you talking about a distill
I run deepseek-r1:14b locally, though it needs to go into RAM and runs slower its still a reasonably good speed. Keeps up with reading it. Should try a larger one at some point, but its quite a bit to download when you get to the larger ones. Usually run ~7b size as that can fit in VRAM and runs way faster.