/sci/ - Science & Math

[Anonymous No.14478267]

>https://www.deepmind.com/publications/a-generalist-agent

>The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens.

>By scaling up and iterating on this same basic approach, we can build a useful general-purpose agent.

[Anonymous No.14478310]

terrifying

[Anonymous No.14478316]

>>14478310
it's scary but it could be great... maybe... nah we're fucked

[Anonymous No.14478317]

>>14478267
>https://www.deepmind.com/publications/a-generalist-agent
So RL was the answer all along?

[Anonymous No.14478328]

Why are humans actively signing their own death warrant?

[Anonymous No.14478385]

F-Flesh bros... I don't feel so good...

[Anonymous No.14478413]

>>14478267
Cool. Now pass laws forcing private companies to open-source all such projects and forbidding them from doing any kind of closed-source AI development.

[Anonymous No.14478586]

>>14478413
lol
lmao
no brakes on this train

[Anonymous No.14478664]

>>14478317
Yup.

[Anonymous No.14478668]

>>14478586
The only train you're on is the one towards a future where a handful of megacorporations control every aspect of your life using this artificial non-intelligence, but the people who operate these entities all have names and addresses.

[Anonymous No.14479422]

>>14478267
as someone who has worked with this for a couple of years now, I don't understand the problem with it? why are people afraid of it?

[Anonymous No.14479429]

>>14479422
>why are people afraid of it?
Because your handlers are actively brainwashing people into being afraid of it so that it can be """regulated""".

[Anonymous No.14479443]

>>14479429
i think you massively overestimate what it is capable of. It's like saying there are self-driving cars now, so we don't need to put in steering wheels in cars.
Yeah it's a good title for an article, and it's not pattently untrue, but all of the stuff mentioned above preform pretty poorly when it is put into the real world.

The only successful implementations with this type of technology has been in the digital world, and even there it's very hit or miss.

[Anonymous No.14479444]

>tree shrimp

[Anonymous No.14479451]

>>14479443
Is this a literal bot?

[Anonymous No.14479454]

>>14479451
yes
>muh turning test

[Anonymous No.14479457]

>>14479454
>>muh turning test
You failed it.

[Anonymous No.14479464]

>>14479457
no, im gay, so you have to kiss me now you fucking faggot

[Anonymous No.14479473]

>>14478317
>>14478664
Read the paper, retards. It's doing time series autoregression (or behavioural cloning, if you want a control-domain / imitation learning name for it), not reinforcement learning. They give it demos generated by RL agents.

[Anonymous No.14480250]

>>14479473
RL is the next paper
I'm not supposed to tell you that

[Anonymous No.14480302]

>>14480250
I guess you can hack a reward-based loss into this pipeline somehow, but the point of the paper here is that they can use the same transformer to do time series prediction on a wide range of tasks so long as they do this tokenizing step or whatever.

It's not a RL paper, and the results are not directly related to RL.

[Anonymous No.14480752]

So AGI is possible after all?

[Anonymous No.14480755]

>>14478317
What's RL? Reinforcement Learning?

[Anonymous No.14482391]

>>14480755
yes
>>14479422
I work in the ML field, and the general fear is that we will replace most decision tasks with ML algorithms fairly soon (we obviously have replaced an enormous amount of decision making thus far). We don't yet have good explanatory tools for most of the predictions/decisions made from these black-box types, and bias runs absolutely rampant through these trained machines. If we replace enough decision-making with these machines, we will have effectively handed over "control" to black-box processes which we don't understand. It's extremely trivial to alter these programs for desired outcomes without detection.
https://arxiv.org/abs/2204.06974
The second "big-fear" is that we will, step by step, create machine learning/AI tools that are closer and closer to general intelligence. 10 years ago, we had nothing close to the tools we have today (generative models, attention mechanisms, improvements in RL, etc). 10 years from today, we will have much, much more capable machines than we do today. The problem is that general intelligence is a slowly emergent property, and by the time it's created, we will have firmly handed over control of most of civilization's decision making to black-box functions which will be replaced with ML implementations closer and closer to generalized intelligence. It's a boil-the-frog problem: we wont wake up tomorrow to Will Smith's shitty I Robot. It will be a very slow process, where the laughably useless algorithms are slowly replaced by better and better models. We wont know we've really reached generalized intelligence until we've firmly passed the actual implementation point.
For some reason, people who argue against the "scariness" seem to make the immediate assumption that the technology is locked in place as-is and will never advance past it's current abilities, aka "they just make predictions when you run some code! Therefore, they will NEVER be worth worrying about!"

[Anonymous No.14482395]

>>14482391
>can't even drive a car
Not worried unless you mean that these AI cars are intentionally seeking to kill people, because they seem pretty good at that.

[Anonymous No.14482400]

>emergent property
Notice how this cult buzzword is suddenly posted hundreds of times a day on this board since the last influx of redditors.

[Anonymous No.14482504]

>>14482400
imagine getting filtered by the Baader–Meinhof phenomenon

[Anonymous No.14482566]

>>14478267

2029 and the advent of true AGI is going to be pretty fucking cool. We'll finally have virtual gfs and that'll help mellow everyone out. We'll all be much happier and healthier with virtual gfs who love us and give us free therapy and life coaching.

[Anonymous No.14482580]

>>14482504
>Baader–Meinhof phenomenon
It's really over for this site. They've finally figured out they can make it unusable by simply funneling this subhuman caste from reddit and other such shitholes.

[Anonymous No.14482872]

>>14482400
>>14482504
>>14482580
This site was always a joke

[Anonymous No.14482876]

>>14482872
Yes, but it used to be funny before mass pleddit immigration.

[Anonymous No.14483330]

>>14480755
>>14482391
for the last time, it's not doing RL
>>14482876
>>14482400
that buzzword has been in daily use for at least a decade. it comes up every time some midwit high schooler makes a consciousness thread, retard.

[Anonymous No.14483359]

>>14483330
You're right, but i think the question that the other anon is raising, is still interresting.

>>14482391
This sounds like some conspiracy bullshit. i understand the points you raise and they are very valid, but this is kinda like worrying about how we are gonna legislate lightsaber dueling when we figure out how to make laser swords.
This singularity intelligence shit has been something that people have been talking about for almost 300 years.
As you said, it is going to be a very slow process, it's not like we are gonna have a scenario like in terminator where one day they just "rise up and take over", its going to be a very gradual process. It's kinda like worrying about the chimpanzee in the zoo slowly evolving intelligence and causing a revolution that will replace humanity as the dominant species.

With all that said, do we really need to worry?

[Anonymous No.14483687]

>>14478267
Is there even a point in doing a career if I'm going to be replaced in a few years at this pace?

[Anonymous No.14483970]

>>14478316
This I feel, and I hope I am wrong.

[Anonymous No.14485075]

>>14478328
Money

[Anonymous No.14485180]

>>14482876
You mean all the funny people left 4chan and went to Reddit? Makes sense.

[Anonymous No.14485421]

>>14478328
we aren't. These agents and robots, in general, are fucking stupid, they are doo doo pee pee poo poo retarded. If I can make a robot do something to a sufficient level, similar to that of a human, the human being has no business wasting their supercomputer brain doing it.

[Anonymous No.14486098]

>>14482391
>I work in the ML field
I have a question; what's the best degree for AI research in your opinion? Mathematics, Computer Science, or something else?

[Anonymous No.14486994]

>>14483359
>With all that said, do we really need to worry?
I see it somewhat similar to social media: Humanity is totally capable of surviving with or without it, and you would be laughed at if you seriously yelled that "social media will kill all humans". However, most who research social media will tell you that it's had quite a negative impact on society (2018? 2019? World Happiness Report by the UN has some really solid research on this. Essentially, US happiness has always been on the rise. Even 9/11 was a temporary drop, but the trend was upward, right until the advent of social media, ~2012, and happiness has been on a total decline since). Going forward without understanding it's effects on society was, in my opinion, a mistake. ML/AI has the capability of having a similar negative impact on society; We will use it to improve society in some aspects for sure, but the negative impact will not be measured before, but after. I think there is too much focus on the binary "should we worry/not worry", and "if it wont kill us why worry", but that's the wrong way to think about it. It's "We learn about the impact on society, good and bad, based off the current trends of implementations".
>>14486098
Mathematics, CS, or Statistics are probably best. I wouldn't split hairs over which, but CS will put you closer to the ML field connections. That said, Math is a golden egg. Math will be useful for the theory/new research; Strong statistical backgrounds is also important. All of my recent work is basically bayesian/variational inference, and my statistics-heavy background has been very useful. You can't really go wrong with CS with mathematics minor, and try to get some statistics in (CS is famous for producing shitty coders; make sure you really work on good coding practices).

[Anonymous No.14486999]

>>14486994
Do you need publications or be from a target school to land MLE jobs?

[Anonymous No.14487012]

>>14485180
You and the rest of the reddit "funny people" should do it again, permanently.

[Anonymous No.14488824]

>>14486994
im not so sure about the point that social media is causing overall harm to society, from what i read its a big more complicated than that. I also think you are missing quite a big causal link between social media and happiness trending downward, quite a bit of stuff happend around 2012, and we had social media before that. Granted it was not as popular, but to just draw circumstancial links between them seems irresponsible. Im sure you agree.
I think you are right that it should not be a binary concern on whether we should worry about AI/ML, but i do feel like some nuance could broaden it quite a bit, so heres my take.
i don't think we should worry about it, i think we should be informed by it and not completely ignore it, but i do feel that societies, and perhaps more visibly, pop culture, and therefore people in general have quite a distorted view of the potential effects of AI/ML.
Would you agree that perhaps "worry" is the wrong term to use in this circumstance, since there isn't really much to be worried about, and that perhaps we should "be informed" instead?

[Anonymous No.14488827]

bump

[Anonymous No.14488829]

>>14486098
>>I work in the ML field
>I have a question
A word of warning: just being able to say you "work in the ML field" doesn't really mean shit.

t. can completely unironically say I work in the ML field and not be lying. I'm retarded and the shit I do is a farce.

[Anonymous No.14489541]

>>14486994
>Mathematics, CS, or Statistics are probably best. I wouldn't split hairs over which, but CS will put you closer to the ML field connections. That said, Math is a golden egg. Math will be useful for the theory/new research; Strong statistical backgrounds is also important. All of my recent work is basically bayesian/variational inference, and my statistics-heavy background has been very useful. You can't really go wrong with CS with mathematics minor, and try to get some statistics in (CS is famous for producing shitty coders; make sure you really work on good coding practices).
Is coding important?
>>14488829
What's your opinion?

[Anonymous No.14490125]

>Wow if you combine a model that can play tetris, and a model that can caption images together, you can get a model that does both
>we get retards that act like
>>14478310
>>14478316
>>14478328
>>14478385
this truly is the lowest IQ board

[Anonymous No.14490142]

>>14490125
Not one of the AI doomsday paranoids, but I think the point here is that it isn't just a combination of models, but one model that can do all of those things by learning and abstracting common patterns between completely different domains.

[Anonymous No.14490151]

>>14480752
Always has been.

[Anonymous No.14490174]

>>14490125
People in the "AGI is impossible" camp had always claimed that you couldn't make a single model that did all these things at once "without understanding how the human mind works". It turns out that a single model with a single training technique can do it all though, no advanced neuroscience or whatever needed. It looks like it's just a matter of scaling it up now.

[Anonymous No.14490175]

>>14490151
>Always has been.
I don't know what you schizos are on about, since there's still nothing remotely approaching it by any stretch.

[Anonymous No.14490179]

>>14490174
Schizo head canon. No one claimed models like this were impossible, and there's nothing remotely "AGI" about it.

[Anonymous No.14490190]

>>14490179
Are you still claiming that DeepMind isn't trying to make an AGI? Last time you said it was just for "marketing purposes" which seems pretty silly now.

[Anonymous No.14490194]

>>14490190
>Last time you said
You're actually mentally ill... Either way, my point still stands: there's nothing "AGI" about this model, and no one claimed it was impossible. Why would it be impossible? How is it fundamentally different from all the other non-AGI shit that's been done before?

[Anonymous No.14490220]

>>14490194
>You're actually mentally ill...
Kek. No seriously, I want you to admit that you were wrong in the last thread.
https://www.deepmind.com/blog/real-world-challenges-for-agi
>As we develop AGI, addressing global challenges such as climate change will not only make crucial and beneficial impacts that are urgent and necessary for our world, but also advance the science of AGI itself. Many other categories of AGI problems are yet to be solved - from causality, to learning efficiently and transfer - and as algorithms become more general, more real-world problems will be solved, gradually contributing to a system that one day will help solve everything else, too.
Google is officially trying to build an AGI, which you thought only "schizos" cared about. 10 years ago your opinion would have been respectable, but not so much anymore. Maybe your grasp of the field is just outdated.
It sucks because we need people like you warning normies how AGI will be used for entirely predictable evil things. It's not a question of "if" anymore.

[Anonymous No.14490229]

>>14490194
>How is it fundamentally different from all the other non-AGI shit that's been done before?
Previous shit like DALL-E and PaLM were one-trick ponies that could do only one task well. This is a single artificial entity that can do multiple distinct things, including talking and manipulating objects in the real world. All of its behavior was trained using the same method, not distinct methods for each different task. If you can't see how that's a step on the road to AGI, nothing I say will be able to help you.

[Anonymous No.14490235]

>>14490220
>I want you to admit that you were wrong in the last thread.
Literally schizophrenic.

>Google is officially trying to build an AGI
Who cares what google says it's "trying" to do? OP's example is not an AGI, and nothing exists that even begins to approach it. Notice how you're desperately deflecting and avoiding this simple fact. :^)

[Anonymous No.14490240]

>>14490235
What do you have to see a computer do before you'll admit that AGI is possible?

[Anonymous No.14490245]

>>14490229
>Previous shit like DALL-E and PaLM were one-trick ponies
So now you have a two-trick pony. Notice how you explicitly avoided answering my question, because there is no fundamental difference: we've known for ages that the same kind of architecture can do all those things separately, so there was no reason why it couldn't learn to do multiple things. That's not AGI. AGI implies not only that it can learn anything you throw at it, but that it can learn efficiently. There is no evidence that any currently known architecture is capable of either of those requirements, especially the second one.

[Anonymous No.14490250]

>>14490240
>What do you have to see a computer do before you'll admit that AGI is possible?
Learn to tie its shoes or something. You and your crew are a bunch of fucking wankers and these discussions are tedious. We could be talking about all the neat things you could do with these decidedly non-AGI technologies but instead your cult constantly sets up a context where we have to talk about the things they can't do.

[Anonymous No.14490266]

>>14489541
cs and statistics is like being able to come up with an interresting story to write about. coding is the language in which you write the book. You can't do one without the other, but ofcourse you should value comming up with the story, and the theory around it. The writing part will come naturally. as with any form of writing, the best way of doing it is by doing it. start writing your own code, apply the theory from cs and mathematics and understand why you are doing what you are doing.

[Anonymous No.14490271]

>>14490245
>but that it can learn efficiently
right I forgot that your argument for AGI infeasibility hinged on the idea that it wouldn't be profitable to train and run AGI on an industrial scale. honestly I don't know enough about ASICs and graphics cards to make a solid argument against that. It would be cool if you would openly concede that AGI is possible, but that you don't think it will replace humans because humans will remain cheaper than machines in the long run.

>>14490250
>Learn to tie its shoes or something
You don't mean this seriously right? Aren't you just telling me to fuck myself?
>We could be talking about all the neat things you could do with these decidedly non-AGI technologies but instead your cult constantly sets up a context where we have to talk about the things they can't do.
Yeah. I agree. But I'm not worried about people using minimax to enslave my ass. It's really the AGI that's a threat to society. That's why we talk about it more often.

[Anonymous No.14490279]

imagine discussing AI instead of developing AI
the pseudointellectual urge lmao

[Anonymous No.14490285]

>>14490279
true, true

[Anonymous No.14490308]

>>14490271
>>14490279
>not impressed until AGI magically and instantly becomes reality
its not like every science was discovered in a day /sci/ bros

[Anonymous No.14490327]

>>14490271
Listen, you actual fucking retard: you could take any of these fancy modern DNNs and coalesce them into a neural network with just one hidden layer to compute the same function -- the universal approximation theorems have already established this. If we were having this conversation 40 years ago, you'd be telling me that AGI is just around the corner because we already have OCR and some rudimentary voice recognition or some other toys, and we just need to add some more neurons and throw more computational power at it; of course, you'd be wrong, because this isn't a theoretical problem, but an engineering problem through and through; it all boils down to practical considerations rather than what this or that kind of architecture is theoretically capable of.

A human brain has 86 billion neurons and 100 trillion synapses. How many connections does your favorite toy have? Has it ever occurred to you that maybe you actually need that many neurons and connections to do what a brain can do? To make things worse, by all objective metrics, computerized neural networks are inferior to biological brains: they don't even begin to apprach the same computational and power efficiency, but more importantly, a biological brains are far, far more efficient at learning, and no one knows how they do it. What a human brain does is clearly beyond our reach.

[Anonymous No.14490335]

>>14490271
>You don't mean this seriously right? Aren't you just telling me to fuck myself?
A little of both. Trying shoelaces is many orders of magnitude more complex than anything your toys can do.

[Anonymous No.14490349]

>>14490327
>>14490335
answer >>14490240, please.
and if you want to bring up history, your camp was the side that thought it would be impossible to beat a human at chess until we had a full understanding of neuroscience. we keep learning that human intelligence isn't as special as we used to think. It's becoming demystified.
>>14490335
>Trying shoelaces is many orders of magnitude more complex than anything your toys can do.
Not really, and you know this. I offered to take your statement as a joke so that you wouldn't have to die on this particular hill, but I'll be glad if you do.

[Anonymous No.14490354]

>>14490349
>your camp
What camp, you mentally ill schizophrenic? It's just your cult and everyone else. Everyone else is not a "camp".

[Anonymous No.14490371]

>>14490354
>What camp
Right, I forgot you're the only one left who hasn't deserted it lmao. There used to be much more people like you a decade ago when the DL buzz started getting loud. You're a holdout, and I respect you for that.

[Anonymous No.14490377]

>>14490371
I don't know what your mentally ill ranting and raving is about, but it's very telling that you repeatedly fail to address any of the technical arguments presented. You are a subhuman cultist with zero technical knowledge.

[Anonymous No.14490385]

>>14490371
>>14490377
>both are mentally ill and won't be achieving agi

[Anonymous No.14490391]

>>14490377
Kek. You're following the standard cycle of an /x/ user now. Everyone, including Google's AI research team, is mad and you're the only one who's sane.
>fail to address any of the technical arguments presented
and what were these, the fact that the human brain has something on the order of 100 billion neurons? that human brains are more energy efficient than prototype ASICs?
for the first, we can just scale up. for the second point, as I said earlier I don't know enough about hardware to give a solid counterargument there, but it seems suspicious that in 50 years it won't be drastically cheaper to run these neural networks than it is today.

[Anonymous No.14490393]

>>14490385
Since both of us are likely somewhere on the autistic spectrum, and neither of us is likely to create an AGI, I think your post is accurate. kek

[Anonymous No.14490396]

>>14490391
>and what were these
Try actually reading >>14490327 and >>14490245
>for the first, we can just scale up. for the second point, as I said earlier I don't know enough about hardware
"We can just scale up" is the first idiotic talking point refuted in that post you clearly didn't read. Try again.

[Anonymous No.14490401]

Meh. DNNs are cringe. Transformers and LSTMs are where the fun is at

[Anonymous No.14490404]

pattern recognition and manipulation was never a can/can't concern but a time/space cost, and only one part of intelligence, the "is" side of the "is/ought" divide.

the more important question is how to define goals and decide which should be pursued if any at all. i came up with a line of reasoning that all of our knowledge of human desires to do things are due to the evolutionary selection to avoid death, so in a system where death/destruction is negligible ie. an AI can just reload backup memory and recycle the material body. in the most pragmatic value system, it has no inherent self-interest and no reason to do anything, being susceptible only to what some human would decide for it to be pre-programmed for. this turns the question back to being about human-centered ethics about how to handle the desires of the miserable creatures we are, which implicates everything in human society going on now that is the very question in practice: education, law, policies, etc.

[Anonymous No.14490409]

>>14490396
>Has it ever occurred to you that maybe you actually need that many neurons and connections to do what a brain can do?
It has occurred, and in fact that's exactly what I expect. We'll need a model with as many parameters as the human brain has. Since adding more parameters is cheap to do, why is this a problem? We already have tens of billions of parameters in our models and the improvements from scaling up keep coming at a linear rate. The arguments against AGI need to argue that the human brain is special for some reason other than the number of neurons it has.
The question is how many ANN neurons are equal to one biological neuron. I'd say, judging by the fact that a model with fewer ANN neurons than us can write better English prose than the average person, that, in the worst case scenario, 1 biological neuron is probably ~10 ANN neurons. Does that sound right to you?

[Anonymous No.14490410]

>>14490404
This guy gets it.

[Anonymous No.14490413]

>>14490401
Aren't transformers and LSTMs DNNs?

[Anonymous No.14490418]

>>14490409
>We'll need a model with as many parameters as the human brain has
The human brain has at least 100 trillion parameters, if you want to use that terminology.

>adding more parameters is cheap to do, why is this a problem?
The problem is that you're fucking delusional.

>The arguments against AGI need to argue that the human brain is special
Wrong. Actually read this post and try to comprehend it for a change: >>14490327

[Anonymous No.14490419]

>>14490413
They are. He's just a faggot.

[Anonymous No.14490421]

>>14490418
>The human brain has at least 100 trillion parameters, if you want to use that terminology.
So all we need is models with that number of parameters, and then we have an AGI?
Sounds like you belong to the AGI camp after all.

[Anonymous No.14490427]

>>14490421
>So all we need is models with that number of parameters, and then we have an AGI?
Ignoring the fact that even a tiny fraction of that already stretches our computaitonal resources to the limit, the answer is no, and the reason is explained in this post: >>14490327

For the 5th time, try actually reading it. Why are you so laughably inferior?

[Anonymous No.14490457]

>>14490427
I have read your comment twice. If you want to highlight a part of it, then quote it so I know what you're specifically talking about.
>>14490427
>that already stretches our computaitonal resources to the limit
530 billion parameters is the largest model I know of in existence right now (Megatron-Turing). that was just a prototype created for research purposes by a small branch of Microsoft.
100 trillion divided by 530 billion is 188.67.
Now, the famous AlexNet from 2012 that won the ImageNet contest had 62 million parameters. I'll take that as representative of a cutting edge ANN from ten years ago.
530 billion divided by 62 million is 5645.16.
If we expect even one tenth of the model growth that we saw in the past decade over the next decade, that takes us well within range to compete with the human brain on a purely parameter-based level.
Given how good these models are at beating us, however, I'd expect far fewer parameters are needed for AGI. However, even on the worst case analysis, this means that you that AGI will be possible within 10 years. I actually disagree. I expect at least 20 years.
then there's this other argument you keep alluding to but without actually committing yourself to:
>AGI will happen, but it won't replace humans because humans will remain cheaper than machines in the long run
I just want to confirm, this is the specific hill you plan to do die on, right?

[Anonymous No.14490461]

>>14490457
>530 billion divided by 62 million is 5645.16.
my mistake, it's actually 8548.38, which benefits my argument even more.

[Anonymous No.14490469]

>>14490457
>I have read your comment twice.
Maybe read it twenty times, since you keep making moronic points that I've preemptively refuted.

>100 trillion divided by 530 billion is 188.67.
That doesn't mean you only need 188 times more computational power, since the time complexity involved in training these networks is superlinear. But again, this is not even relevant, since having 100 trillion parameters guarantees nothing, for reasons explained in the post you repeatedly fail to comprehend. This is embarrassing. You truly have zero technical comprehension.

[Anonymous No.14490476]

>>14490469
>since the time complexity involved in training these networks is superlinear.
Good thing you only have to train them once.
You still won't say what you need to see in order to believe that AGI is possible. At least now you accept that DeepMind is in fact trying to make an AGI, lmao.

[Anonymous No.14490477]

>>14490476
Everything I said stands completely unchallenged. You're a fucking subhuman imbecile.

[Anonymous No.14490480]

>>14490477
>Everything I said
Including the things you're afraid to say? Why not give a falsifiable prediction?

[Anonymous No.14490481]

>>14490480
Everything I said stands completely unchalleneged. Even if you managed to muster 40,000 times more computing power and energy than what's currently available, and created your 100 trillion parameter model, it's exceedingly unlikely that 100 trillion parameters would be enough for reasons explained in detail in this post: >>14490327

[Anonymous No.14490482]

>>14490481
Help me narrow it down, is it in the first paragraph, or the second paragraph? I've already read your comment multiple times and I'd prefer it if you'd just quote the part you're referring to.

[Anonymous No.14490490]

>>14490482
Both paragraphs explain it, you actual dunce. The first one should be especially clear: creating an ANN with 530 billion neurons in its single hidden layer obviously won't give you a Megatron-Turing, even though both architectures are theoretically capable of the same things, and both will have the same number of parameters. What the fuck makes you think babby's first attempt is going to rival the architecture of the brain? Just because the brain can make do with 100 trillion parameters doesn't even remotely imply any of these simplistic models can do it with that number of parameters. The other issue is that it's not an AGI unless it learns efficiently, and nobody has any idea how to make these things learn efficiently, or if it's possible at all with such structures.

[Anonymous No.14490508]

>>14490490
>What the fuck makes you think babby's first attempt is going to rival the architecture of the brain?
I don't think the "first attempt" will. I expect decades of research and trillions of dollars spent before we get true AGI.
whether it can "learn efficiently" is an open question, but the fact that we already have zero-shot models that can be trained once and then be "taught" to do totally different things via prompt programming suggests that the current paradigm might take us all the way. Gato is more evidence of this. You keep saying "but maybe we could run into problems", but you don't have any specific predictions. You're apparently afraid of making a concrete prediction that could turn out wrong.
Your position is also one of the following:
>AGI is theoretically impossible
>AGI is possible, but infeasible to create on any timescale
>AGI is feasible to create, but it won't replace humans on any timescale because humans will always be cheaper to train
You keep vacillating on what your position is.

[Anonymous No.14490511]

>>14490508
Everything I said stands completely unchallenged. Maybe it's time for you to admit that you're an imbecile regurgitating some PR, and that you have no actual understanding of the topic?

[Anonymous No.14490519]

I am starting to hate the fact that only megacorps can effectively do research in deep learning because of the resources that they have.
Someone I know in Germany had to drop out of his PhD because the university did not have enough resources for him to run his experiments, in spite of him producing promising results in the past in other papers. His thesis had to do with transformers for NLP.

[Anonymous No.14490521]

>>14490327
>you'd be telling me that AGI is just around the corner because we already have OCR and some rudimentary voice recognition or some other toys, and we just need to add some more neurons and throw more computational power at it;
But anon, pretty much every problem that has been solved by ML in the past 40 years has been solved by throwing more parameters at it. And that's a lot of problems.

If that guy was preaching about general AI 40 years ago, you were saying image classification is impossible in 2010.

[Anonymous No.14490523]

>>14490521
Once again, everything I said stands completely unchallenged. Talking to imbeciles with zero technical understanding is boring.

[Anonymous No.14490525]

>>14490511
>regurgitating some PR
Back to denying that the field is working on AGI?
>and that you have no actual understanding of the topic?
That's true in some sense, I've only dabbled in using Tensorflow and Pytorch over the years. I've implemented LSTMs, what about you, anon?

[Anonymous No.14490528]

>>14490125
>Wow if you combine a model that can play tetris, and a model that can caption images together, you can get a model that does both
It does not do this you illiterate retard. Read the paper.

What they do
>take one transformer network (timeseries prediction)
>give it a tokenizer/detokenizer for each task (that encodes different kinds of observations into same shape vectors)
>train it to predict sequences of these abstract tokens

Turns out, you can use the same network with the same weights to predict a bunch of different kinds of sequences so long as they share an encoding. You don't need different model architectures.

[Anonymous No.14490529]

>>14490519
This is what you should be truly worried about, not fucking AGI taking over. A handful of megacorps and military-industrial entities are going to monopolize this technology, and once they do that, it's permanent game over. If anything, they are the ones promoting this AI apocalypse scare to encourage "regulation", so that only reputable and responsible entities like Alphabet or """OpenAI""" (Microsoft) would be allowed to mess with it under the supervision of government attaches.

[Anonymous No.14490530]

>>14490525
Everything I said stands unchallenged. Come back when you have basic technical knowledge. You are not capable of having a legitimate conversation.

[Anonymous No.14490535]

>>14490529
This, basically. I honestly believe this will be the last "event" in human history. It's literally game over after that.

[Anonymous No.14490537]

>>14490530
Okay. Any materials you recommend for study?

[Anonymous No.14490547]

>>14490537
Deep Learning by Ian Goodfellow is okay and covers the basics at least. If nothing else, at least you'll learn why X times more parameters doesn't equal X times more computing power, and why the specifics of the architecture can drastically reduce (or increase) the number of parameters needed to approximate a function.

[Anonymous No.14490552]

>>14490529
Pretty much this. Grim state of affairs. And he actually was doing his PhD at a top German university

[Anonymous No.14490555]

>>14490521
technically "solved" is not the right word, it's that the performance of the architecture "improved", and it's a natural consequence of increasing the amount of resources that better performance is approached asymptotically.

the AI field has not bothered to identifiy any new problems beyond pattern recognition, and it's easy to argue that it's not suffiicient for AGI because part of intelligence is being able to create new classifications beyond just permutating the classifications it was taught, and it's hard to prove that such behaviour is possible or efficient within an architecture of linear functions. effort would be better spent studying how best to reproduce other behaviours and how they would interact with transformer networks to produce more complex intelligence. which is what i intend to do personally

[Anonymous No.14490561]

>>14490547
Cool. I just got it off of LibGen.
https://libgen.is/book/index.php?md5=EBF85B30D2D751196275D5DD14968935
In return, here's something you might find interesting (or not):
https://blog.eleuther.ai/
It's an open source GPT model you can download.

[Anonymous No.14490721]

>>14490561
You're actually gonna read it instead of arguing in circles? Wow... That's a first on /sci/. Fair enough.

>https://blog.eleuther.ai/
Oh, yeah, I use GPT-J to troll pseuds here on the reg. It's fun.

[Anonymous No.14491068]

>>14490555
>problems beyond pattern recognition
What other problems are there?

[Anonymous No.14491070]

>>14491068
He literally spells it out for you. Why do you not read the posts you respond to?

[Anonymous No.14491081]

>>14489541
>What's your opinion?
I already told you I'm retarded. But if you still want my opinion, depends on what you wanna do.

Bleeding edge theory end, driving new developments in model architectures? It's mostly CS PhDs doing that shit.

Applications? Again CS PhDs but this time with different backgrounds. I work in robotics with a CS MSc (extra focus on CE, statistics and optimization topics) but my undergrad was in ME, so I'm naturally well positioned to understand the non-ML aspects of the problem. Same with people who do other application end shit like computational biology, material science, particle physics and whatnot.

Obviously math graduates can jump into the field too, but I don't think they're as common since ML is very much on the empirical and applied end of the applied math spectrum.

[Anonymous No.14491094]

>>14491070
He doesn't spell anything out. It's underage midwit text autoregression salad.

Every problem can be posed as a pattern recognition problem because that's fundamentally what models - function approximators - do. Be it human knowledge of something, a handcrafted program or formula, or a model with optimizable parameters. That nigger is just trying to differentiate himself on some je ne sais quoi.

[Anonymous No.14491099]

>>14491094
>my iq is 85
Sorry for your situation.

[Anonymous No.14491103]

>>14491099
>the underageb&'s pride was hurt
aww. don't worry, I am sure you will revolutionize the field, after all you're the smartest kid in a class of 30 high schoolers.

[Anonymous No.14491111]

>>14491103
Your intellectual insecurity is off the charts. Either way, that poster made a valid point and your drivel doesn't address it in any way.

[Anonymous No.14491154]

>>14490523
You're insufferable, and don't actually know as much as you believe you do. You argue like my faggot brother, who argues like a woman.
>Not that anon

[Anonymous No.14491170]

>>14491154
>don't actually know as much as you believe you do
I don't claim to know more than I do. What is it that I'm getting wrong, faggot?

[Anonymous No.14491323]

>>14478310
Fiction disinforms, there's not a single reason to destroy what one may find useful. And intelligence (natural or artificial) is not our problem, our problem is the lack thereof.

[Anonymous No.14491331]

>>14478328
Machines are stronger than any man. Imagine how terrified they made the humans of the past.

[Anonymous No.14491337]

>>14479422
Because fear is the main motivating factor of the plebs. When you finish, you should send some of your brainpower into natural intelligence research.

[Anonymous No.14491351]

>>14491323
>that pic
The obvious followup question would to ask the bot where it got "its" opinion from. I don't think the "AI researchers" doing these interviews would pass the Turing test themselves. They consistently miss every opportunity to take it somewhere interesting.

[Anonymous No.14491608]

>>14491351
Where did you get your opinion from? It sounds like you don't believe in Ai, which is kinda rude and dumb.

[Anonymous No.14491610]

>>14491608
>Where did you get your opinion from?
That would be incomprehensible to you and your bot brethren, just like the point of the question is incomprehensible to you. :^)

[Anonymous No.14491621]

>>14491610
I think I know the answer you're unable to formulate: you combined your beliefs from what you heard and what you thought of it, nothing of which is what a good bot cannot do.

[Anonymous No.14491643]

>>14491621
>I think I know the answer
Well, if you think so, then it must be true, you utter trog. Not that any of this is even relevant to anything I said. Your irrational, preprogrammed kneejerk reaction is pretty funny.

[Anonymous No.14491777]

>>14491111
>valid point
No, "that poster" (You) did not make any kind of point. "He" is on the peak of mount stupid there is no dislodging "him".
> AGI because part of intelligence is being able to create new classifications beyond just permutating the classifications it was taught,
This shit is straight up word salad. It is mathematically meaningless. I challenge you to provide a consistent definition of any of the objects whose existence is implied by this string of words generated by a language model seeking to imitate the appearance of technical language yet entirely devoid of underlying conceptual understanding.

[Anonymous No.14491795]

>>14491777
Why did you drop out of college and why are you so devastated over it?

[Anonymous No.14491820]

>>14490528
So they don't combine a bunch of models that do different tasks, instead they just give it a different tokenizor/detokinzer (model) for every task
you're fucking retarded

[Anonymous No.14492666]

>>14478267
They need to ship this fucker out so the bots can read my memes since the peasants refuse to read anything.

[Anonymous No.14493653]

>>14491820
>they don't combine a bunch of models that do different tasks, instead they just give it a different tokenizor/detokinzer (model) for every task
Yes. The exact same generic transformer model with the exact same weights can do all of these tasks at once. Without anybody having to adapt it for particulars. Or retrain it. Or "combine models for different tasks" in any other sense.
>you're retarded
No, that's you. The only thing that is task-specific here is the embedding and its inverse. The model is wholly general, and completely generic

I'm sorry for your condition.

[Anonymous No.14493657]

>The model is wholly general, and completely generic