![](/static/253f0d9b/assets/icons/icon-96x96.png)
![](https://lemmy.ml/pictrs/image/q98XK4sKtw.png)
I have no idea how to fix the problem, but I’ve read somewhere that burn (a relatively new machine learning framework in Rust) is capable of loading models like stable diffusion. As Burn is built with webGPU and all the shader transpiler-stuff that comes with it doesn’t that mean that it can also run easily on (even older) AMD cards? I think what’s lacking is equal performance as nvidia drivers are heavily optimized already.
Maybe someone knows more here?
It cannot “analyze” it. It’s fundamentally not how LLM’s work. The LLM has a finite set of “tokens”: words and word-pieces like “dog”, “house”, but also like “berry” and “straw” or “rasp”. When it reads the input it splits the words into the recognized tokens. It’s like a lookup table. The input becomes “token15, token20043, token1923, token984, token1234, …” and so on. The LLM “thinks” of these tokens as coordinates in a very high dimensional space. But it cannot go back and examine the actual contents (letters) in each token. It has to get the information about the number or “r” from somewhere else. So it has likely ingested some texts where the number of "r"s in strawberry is discussed. But it can never actually “test” it.
A completely new architecture or paradigm is needed to make these LLM’s capable of reading letter by letter and keep some kind of count-memory.