OpenAI’s first open source language model since GPT-2
You must log in or register to comment.
*if you have a laptop with 16gb of vram. Otherwise you’ll be watching ollama hit your CPU for 5 minutes with no output
Isn’t that true for most models until someone destiles and quantises them so they can run on common hardware?
This is the internet, we’re only allowed to be snarky here.
Yes, but 20 billion parameters is too much for most GPUs, regardless of quantization. You would need at least 14GB, and even that’s unlikely without offloading major parts to the CPU and system RAM (which kills the token rate).
I mean yeah, but that doesn’t make the title any more true.
No thanks.
Agreed.
paywall free version?
