Avram Piltch is the editor in chief of Tom’s Hardware, and he’s written a thoroughly researched article breaking down the promises and failures of LLM AIs.
Avram Piltch is the editor in chief of Tom’s Hardware, and he’s written a thoroughly researched article breaking down the promises and failures of LLM AIs.
Isn’t interpolation and extrapolation the same thing effectively, given a complex enough system?
Depending on the geometry of the state space, very literally yes. Think about a sphere, there’s a straight line passing from Denver to Guadalajara, roughly hitting Delhi on the way. Is Delhi in between them (interpolation), or behind one from the other (extrapolation)? Kind of both, unless you move the goalposts to add distance limits on interpolation, which could themselves be broken by another geometry
No, repeated extrapolation results in eventually making everything that ever could be made, constant interpolation would result in creating the same “average” work over and over.
The difference is infinite vs zero variety.
Fun fact, an open interval is topologically isomorphic the the entire number line. In practice they’re often different but you started talking about limits (“eventually”), where that will definitely come up.