Expertly explained. Thank you! It’s pretty rad what you can get out of a quantized model on home hardware, but I still can’t understand why people are trying to use it for anything resembling productivity.

It sounds like the typical tech industry:

“Look how amazing this is!” (Full power)

“Uh…uh oh, that’s unsustainable. Let’s quietly drop it.” (Way reduced power)

“People are saying it’s not as good, we can offer them LLM+ plus for better accuracy!” (3/4 power with subscription)