/sci/ - Science & Math

>>15140816
kek. lel even.
GPT is just a machine to generate plausible, vacuous, shallow texts. Rather than representing intelligence, GPT rather highlights the contrast between true and faux intelligence.
Sure, it might seem like the dynamic
>"feed it a lot of text" => "ability to simulate logical reasoning increases"
implies that GPT will eventually achieve logical reasoning ability. But that isn't really the case. If you imagine a graph "fed material" x "logical reasoning ability", then right now we are about at the asymptote. You can feed GPT a trillion more texts, and it won't get much smarter.

>just feed it more material
The corpus that GPT3 uses is already ultra-representative. It's simply a large fraction of all text that humans have generated over the last two decades. The syntactic and even semantic abstract value of a Wikipedia article from 2010 that describes a city is about identical to a Wikipedia article from 2023 that describes an anatomical feature. The same applies to news articles, blog posts, etc.

The reason GPT won't get smarter is because this model's current apparent logical ability heavily sources from the inherent logic of 1) syntax 2) semantics 3) knowledge. Notice that the last point sounds vaguely anthromorphic. With "knowledge" I just mean the abstraction layer above semantics. There is in an inherent logic to sentences of the form e.g.
>water extinguishes fire
That GPT can track. But this "well of inherent logic" runs dry quickly. It only applies to simple statements.
Also note how genuinely interesting and intelligent (in humans -- if I needed to clarify) statements don't rely on knowledge much, but on depth of reasoning.

Pic related, it's GPT7 trying to gain an edge from training ever more data (no secret insights hidden in standard human generated internet texts tho)