ryujin470@fedia.io to Technology@beehaw.org · 2 months agoOpenAI releases a free GPT model that can run on your laptopwww.theverge.comexternal-linkmessage-square8fedilinkarrow-up138arrow-down17file-text
arrow-up131arrow-down1external-linkOpenAI releases a free GPT model that can run on your laptopwww.theverge.comryujin470@fedia.io to Technology@beehaw.org · 2 months agomessage-square8fedilinkfile-text
minus-squareCyberSeeker@discuss.tchncs.delinkfedilinkEnglisharrow-up1·2 months agoYes, but 20 billion parameters is too much for most GPUs, regardless of quantization. You would need at least 14GB, and even that’s unlikely without offloading major parts to the CPU and system RAM (which kills the token rate).
Yes, but 20 billion parameters is too much for most GPUs, regardless of quantization. You would need at least 14GB, and even that’s unlikely without offloading major parts to the CPU and system RAM (which kills the token rate).