They do math, just in a very weird (and obviously not super reliable) way. There is a recent paper by anthropic that explains it, I can track it down if you’d be interested.
Broadly speaking, the weights in a model will form sorts of “circuits” which can perform certain tasks. On something hard like factoring numbers the performance is probably abysmal but I’d guess the model is still trying to approximate the task somehow.
They do math, just in a very weird (and obviously not super reliable) way. There is a recent paper by anthropic that explains it, I can track it down if you’d be interested.
Broadly speaking, the weights in a model will form sorts of “circuits” which can perform certain tasks. On something hard like factoring numbers the performance is probably abysmal but I’d guess the model is still trying to approximate the task somehow.