[ 3 / biz / cgl / ck / diy / fa / ic / jp / lit / sci / vr / vt ] [ index / top / reports ] [ become a patron ] [ status ]
2023-11: Warosu is now out of extended maintenance.

/sci/ - Science & Math


View post   

File: 115 KB, 635x588, MLG.jpg [View same] [iqdb] [saucenao] [google]
10809198 No.10809198 [Reply] [Original]

Studying, trying and discussing everything related to machine learning, neuronal networks and AGI.

Old thread >>10769848

Ongoing projects:
Different DeepNude implementations with czechcasting dataset ( https://mega.nz/#!IKwxXAgI!YpI648jsxvPdyfKwlAyypVEm_jnUgWXde8MJmjNgSBI )
Pepe generation with StyleGAN

Related links:
>dreamscopeapp.com - recreates an image in the style of another image (made pic related using this)
>deepart.io - same as above but it takes longer
>thispersondoesnotexist.com - generates portraits of people
>thiswaifudoesnotexist.net - generates portraits of anime girls
>thiscatdoesnotexist.com - generates (mostly terrible) pictures of cats
>ganbreeder.app - generates images of certain "topics"
>waifu2x.booru.pics - doubles the size of images and helps with noise reduction
>talktotransformer.com - generates several paragraphs of text from a prompt
>https://make.girls.moe/ - another anime character generator

>> No.10809368

why are you linking waifu AI?

It's not SOTA.

http://forums.qhimm.com/index.php?topic=18798.0

Try this ESRGAN-for-retards GUI
or directly https://github.com/xinntao/ESRGAN

There are manga dataset trained versions that will do better than waifu

>> No.10809393
File: 514 KB, 1800x1800, 2019-03-01-stylegan-twdne-64bestsamples.jpg [View same] [iqdb] [saucenao] [google]
10809393

>>10809198
>thiswaifudoesnotexist.net - generates portraits of anime girls
Why can't I hold all these waifus?!

>> No.10809482

>>10809393
Why not? I have [math]G_{64}\underbrace{\uparrow\uparrow...\uparrow}_{G_{64}} G_{64}[/math] waifus. Doing pretty fine.

>> No.10809663 [DELETED] 

>>10809482
I have tree(3) waifus

>> No.10809711

>>10809198
>>thispersondoesnotexist.com - generates portraits of people

What sort of training/architecture is required for generating images with this quality? I've been trying for days on the celeba dataset with a keras GAN model, but couldn't get them to look nowhere near as good as this

>> No.10809714
File: 172 KB, 1040x711, 3.png [View same] [iqdb] [saucenao] [google]
10809714

I spent the better half of the day stripping down the feed forward neural network (the pure python implementation for digit recognition, from that popular online book) to a more concise script. Mostly for the sake of making a video about it, maybe tomorrow.

https://gist.github.com/Nikolaj-K/ecb7cde9aadbf4efa08cf1b0456a17ee

https://www.youtube.com/c/NikolajKuntner

>> No.10809731

>>10809714
I had already found your YT channel by pure coincidence. Glad to know you are from /sci/.

I'm a sub

>> No.10809736

>>10809711
It's the pretrained StyleGAN model.

>> No.10809814

Damn, I didn't know we even had a general for this, I haven't been on /sci/ in over a year
Fuck I need to start bosting

>> No.10809844

>>10809814
This is only a second thread, so you hadn't missed much.

>> No.10809851

>>10809714
Theory is a good thing for sure, but at this point it mostly the waste of time. Smart people already made all libraries needed to make really cool things without thinking too much how exactly it works on the background.

>> No.10809887
File: 60 KB, 512x512, fake_img_0064_134000.jpg [View same] [iqdb] [saucenao] [google]
10809887

Cherry picked some pepes from last generations. What do you think boys, is it good enough?

>> No.10809900
File: 22 KB, 480x360, hqdefault.jpg [View same] [iqdb] [saucenao] [google]
10809900

>>10809851
I think theory is necessary for innovation and understanding - but even beyond any argument of that sort, the math of it is more fun to me than the use of it (my PhD is in statistical physic theory)

>> No.10809927

>>10809393
Can't tell if I am a flawed fleshy pattern recognition machine or all red eye brown hair waifus are just trained from a single holo image set.

>> No.10809967

How important is data structures/algs to ML? I haven't had a course in them but I am pretty good at learning on the fly.

>> No.10810146

>>10809887
Wow, that's way better than any pics I saw previously of PepeGan

It's not good enough yet though keep improving

>> No.10810369

>>10809887
>>10810146
Ok, training the machine to draw pepe is one thing..but training the machine to come up with rare pepes that are doing certain tasks or in different poses are another.

>> No.10810588

>>10809967
not too important

>> No.10810637

>>10810369
Task like this isn't well understood yet. There is conditional GANs, but they require manual labeling. Rare pepes may be achieved by mixing different datasets and then looking for unique pepes in latent space.

>> No.10810642

>>10809967
Depending of what you gonna do with it. If you want to invent new engines loss function, then you probably need some understanding of crucial ML algorithms. If yoh wanna just play with you barely have to understand programming at all.

>> No.10810900
File: 55 KB, 1280x720, dougie.jpg [View same] [iqdb] [saucenao] [google]
10810900

>be Data analyst
>softer education background with business masters (some financial analysis, stats etc)
>end up being breddy good at Python, knows Pandas etc inside out
>company makes Data Science team, get put on it, title change to Data Scientist
>we're slowly looking into ML projects
>beginning to realise programming is the easy bit here

I need to git gud, I'm a fraud DS atm.

I have the programming skills but not the stats knowledge to know what I'm really doing when fucking around with Sklearn.

I have a vague idea of how decision trees and random forests works, and know the difference between supervised/unsupervised, classification/regression etc but that's about all.

What do /mlg/? I want to live the data meme for real

>> No.10810925

>>10810900
Commit to read statistics texts for at least 30 minutes a day (after 30 you can stop and do something else, without bad conscious, or - if you're still interested - read on). This way you read 15 hours a month or more.
Search for book recommendations in book recommendations threads on StackExchange and use libgen to download it.

>> No.10810938

>>10810925
I'm up for that

Give me a book to start with and I'll go from there

>> No.10810987
File: 156 KB, 768x1024, 1_O9HyohP5spct_XgxvG6PDg.jpg [View same] [iqdb] [saucenao] [google]
10810987

>>10810938
Read this

>> No.10810994

>Evolution of Face Generation | Evolution of GANs
https://www.youtube.com/watch?v=C1YUYWP-6rE

>> No.10810999

>>10810900
Also watch this https://www.youtube.com/watch?v=1lxHH1UBTBU

>> No.10811021

>>10809927
kek you're right

>> No.10811122
File: 1.44 MB, 256x256, 0.webm [View same] [iqdb] [saucenao] [google]
10811122

I think I'm done with it for now. Results are pretty shitty, but I think you could do much better if use checkpoint from anime face generation models instead of training it completely from scratch. And better graphics card would be helpful for sure. Don't even try it on the average laptop like I did.

Here is the dataset I collected along with the script for data augmentation: https://mega.nz/#!QeAHFSgB!3DNv5MAjM-_g2gDX7IyQtU8lDpbWOmZ88vtiXya7BTA
This is the implementation of StyleGAN I used: https://github.com/taki0112/StyleGAN-Tensorflow but I would rather recommend to use original one.

>> No.10811185

>>10809731
>I'm a sub
Very forward anon, he does look like a big ol bear though.

>> No.10811188

>>10809198
Thank you for not calling this:
>Artificial Intelligence General
That's good of you.

>> No.10811192

>>10811122
That's great anon

Good to see actual projects appearing on /sci/

>> No.10811195

>>10809887
Some of them are about the level of shitposters on paint here

Which means it's pretty damn good

>> No.10811217

I want to actually get started with ML. What's a simple project that's more original than reading digits?

>> No.10811246
File: 26 KB, 1000x600, Tetration_by_escape.png [View same] [iqdb] [saucenao] [google]
10811246

>>10811217
Try to predict function from it's graph. Like you feed the network set of points and spits the underlying function like so [(0, 0), (0.4, 0.389), (1.3, 0.964),...] -> sin(x). It does not require huge amount of memory or processing and all data points might be generated on the fly, so it should be super easy even for beginners.

>> No.10811261

>>10810925
>Commit to read statistics texts for at least 30 minutes a day (after 30 you can stop and do something else, without bad conscious, or - if you're still interested - read on). This way you read 15 hours a month or more.
Side off topic post but this is a very good advice. I too wasted most of my life inefficiently trying to force myself to study stuff for 6 hours a day without interruptions because dude stoicism lmao, but every single time I burned out just a week later and had to revert to two weeks of mediocrity and impulse satisfaction before I could even go back to studying reliably for more than an hour. The reason for this is that our brains haven't evolved for long-term reward evaluation and will always seek short-term rewards for whatever you're about to lose your precious energy on, and if you fail to provide that reward and instead whip yourself like an autistic stoic, yes, you will be able to lose your energy and stare at the text but your brain would shut down and most likely even associate this task with negative stimuli as it is not rewarding, up until one day you forget about it and start seeking short-term rewards again.

The only way to avoid this is either associating studying with short term rewards (good fucking luck with that), or rewarding yourself every 30 or so minutes before the brain starts realizing that you're not getting anything off of this and shuts down again.

>> No.10811365

If this general is going to survive we need a pasta

I suggest some good ML blogs would be a good thing to have in there. Not the million 'how to do Big Data Machine Learnin' by pajeets but actual good ones doing interesting things with their data

This kind of thing:
https://invenia.github.io/blog/2018/07/18/EIA-coal/

Though maybe more interesting than that

>> No.10811401

>>10811217

i'd try to train a 9x9 mini alphago, or whatever your favorite board game is as long as the rules aren't too complicated. maybe start off with a simple cnn model, bootstrap it with a supervised pretraining on real 9x9 games, and then see if you can improve it with reinforcement learning via adversarial self-play.

you could even start out with tic-tac-toe as a sort of sanity check

>>10811246

neural networks aren't good regression models. they're really more suited for discrete classification tasks

>> No.10811413

>>10811401
>neural networks aren't good regression models. they're really more suited for discrete classification tasks
Everything much simpler. This task is impossible.

>> No.10811418

>>10809711
If I recall correctly, the authors used multiple super-expensive Nvidia Tesla cards and the training still took several days. If you're running a normal computer with a normal GPU (even if it was 2080 Ti) you're out of luck.

>> No.10811423

>>10811365
There are various awesome-deep-learning github repos and some are quite extensive and good

>> No.10811455

>>10811246
Good one.

>> No.10812172

>>10811122
It's an infinite continuum of Pepes!!

>> No.10812702

>>10811418
I know, i don't expect to get the same result. But there's a tutorial for PyTorch, where a regular GAN is used, it's available here: https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html

I'm trying to adapt the same model for Keras. It should be able to run on a regular GPU and generate reasonably good looking images, but still mine look like total crap and are taking too many epochs of training... I'm starting to believe that it's some limitation of Keras, since I'm using the exact same architecture.

>> No.10812758

>>10812702
>Keras
Bad idea. Keras was built with feed-forward networks in mind. You can make GAN with it, but it will work 10 times slower than tensorflow. And keras not make much sense standalone now, because it's a official part of tensorflow. So my advice to you, if you wanna learn GANs, take any relatively simple models and try to manually convert it from tf1 to tf2. That would be not only a good way to learn tf2, but will actually do useful job for the community.

>> No.10812824

>>10812758
thx for the advice. But I don't think it's solely a running time problem because, as I said, my model takes too many epochs (something like 10) to generate the shitty pictures, while the PyTorch implementation takes about 3 or 4, using the same minibatch size. I suspect it's because Keras doesn't allow us to calculate the gradients for the discriminator with both fake and real images, and then update the weights simultaneously, as its done in the PyTorch tutorial. Don't know if TF will allow this, but I might try it. I understand that Keras isn't the best option, but I'd still like to know why it's not working anyway...

Also, is there any reason for using TF2 instead of PyTorch for pure research? Most of my colleagues at uni only use Keras and PyTorch. No one touches TF for some reason. I've used TF1 a while ago, but it was for GRUs, which are terrible to implement with Keras.

>> No.10812835

>>10809887
Neat. Can you share the project and dataset?
That would be awesome.
You're using some gan?

>> No.10812984

>>10812835
Here it is. >>10811122

>> No.10812999

>>10812984
Oh, thanks mate!

>> No.10813163

>>10809844
Well I had a data scientist job interview today, for a big bank. They seemed really impressed but I have my technical interview (i hope) later with some of their quants.

>>10809900
Don't listen to that guy. If you don't know the theory behind your ML algorithms you are utter garbage.
Btw nice field, were you doing statistical mechanics? A lot of good data scientists seem to come from stat mech and solid state physics.

>> No.10813169

>>10809967
It depends!
For running your ML algorithm, like the other guy said you barely need any programming.

For DEPLOYMENT and big data you need to know a lot of programming and IT stuff in general. Even some basic web development is really useful.

>> No.10813228

>>10811217
Clean some data and learn Pandas.
Then try to predict something with linear regression.

>> No.10813852
File: 9 KB, 200x200, Idris_ava.png [View same] [iqdb] [saucenao] [google]
10813852

Is there a way where we can raise the participation in this thread - maybe get some more collaborative work going? Those 10 posts a day that the thread gets are surprisingly disappointing.

>>10813163
Despite having had jobs in augmented reality, quantum computing and the blockchain space, I never got deep into machine learning and data science. I was doing statistical physics for chemical kinetics, actually.

>> No.10814159

I've noticed almost everybody here has a background in a meme field. Guess serious scientists should avoid the trash can that is Data Science.

>> No.10814164

>>10811122
It's actually super fucking cool

>> No.10814177

>>10813852
Last thread was one of very few that reached bump limit and not die like of the threads on /sci/. So it's not so bad. We still have a guy who was working on different DeepNude implementation. By the way original version is up on the air again https://github.com/lwlodo/deep_nude

You can also try to do better pepe generator, or just expand the dataset it if you have some rare pepes.

>> No.10814230

>>10811246
Sounds like basic linear regression if you want to detect linear combinations of basic functions.

>>10811401
Yup adversarial stuff sounds interesting, but I have no idea where to start. Tic-tac-toe is so simple even minimax is overkill, but big games like chess and go require big hardware.

>> No.10814245

>>10814230
>Sounds like basic linear regression if you want to detect linear combinations of basic functions.
Linear regression will give you polynomial of high orders what is boring. Here is you have to find as elegant solution as possible.

>> No.10814311

>>10809198
is this Norvig book in your image?

>> No.10814415 [DELETED] 

>>10814177
I was thinking of an engaging collaborative effort - doesn't need to be a product

>> No.10814432
File: 131 KB, 862x537, vag.png [View same] [iqdb] [saucenao] [google]
10814432

>>10814177
literally
>bobs or vagene - the program

It's also pig disgusting Python style, but so be it.

Anyway..
I was thinking of an engaging collaborative effort - doesn't need to be a product

>> No.10814449

>>10814432
>I was thinking of an engaging collaborative effort - doesn't need to be a product
What was your idea? I willing to listen.

>> No.10814470

>>10814230
>Yup adversarial stuff sounds interesting, but I have no idea where to start. Tic-tac-toe is so simple even minimax is overkill

write function that takes the board state and returns the winner, e.g. 1 if player won, 0 if it's a draw, -1 if they lost.

design a cnn, or a fully-connected net to take tic-tac-toe board and output a distribution over possible moves

randomly initialize network

play network against itself, randomly sampling moves from output distribution.

update weights to make network more likely to pick the moves the winner picked at each turn

>> No.10814698

I’m doing a project to predict gentrification in neighborhoods of Nashville, based on publicly-available building permit data from Davidson County. I’m gonna pair the public data with info from the ATTOM API which is for real estate.

Southeastern US cities are extremely interesting to study because they’re undergoing a big change.
Generally in planning and deal estate, we expect mixed-use buildings to cause gentrification and I’m going to see if that’s true.

>> No.10814730

>>10814698
Why is this machine learning and not plain statistics?

>> No.10814733

>>10814449
Err, I didn't actively think about about.

Might be that the interests in this group diverge but one could imagine, for example, a reading group.

>> No.10814794

>>10814730
I'm using data to train a machine learning algorithm to predict things so we can apply this algorithm to other datasets. You might use statistical methods in a machine learning algorithm (linear regression) or more modern stuff like boosted decision trees.
Also lots of NLP and data-wrangling.

>> No.10815333

>>10814733
I don't really understand what you are talking about. What exactly are you gonna read? New papers?

>> No.10815372

Does anybody have experience with StyleGAN's latent space? I generated a face with the pretrained FFHQ model and want to find nearby faces. I think the lower latents are supposed to be coarse and the higher ones finer details, but when I randomize some latents in the 175-200 range, I still get completely different faces.

Has anybody made something like this for FFHQ?
https://colab.research.google.com/drive/1LiWxqJJMR5dg4BxwUgighaWp2U_enaFd#offline=true&sandboxMode=true

>> No.10815392

>>10815372
Take two different latent representations. Interpolate between them. Profit!

>> No.10815405

Damn, be glad you’re not a fresh grad looking for an entry-level data science job.
I have a bunch of projects, tons of MOOCs, MS Mathematics, and I still can’t get hired. My city is shit but if I apply out of state they don’t want a candidate from far away.

>> No.10815418

>>10815405
Imagine living in 3th-world country, bro. Better yet in the shithole with 80 average IQ.

>> No.10815492

>>10814794
Fair warning: I love NLP work but finding that work is a bit of a ballache.

>> No.10815546

>>10815405
So don't tell them where you live you fucking retard

>> No.10815572

>>10815418
>3th-world country
>Treeth world country
LMAO at your refugee tier English.
I'll turn 360 and walk away.

>> No.10815665

>>10815492
Ironically I have a good connection for getting hired as an NLP guy at a big company, I just want to make sure I know NLP really well before I apply/get referred.
Just need to find time to finish up this sentiment analysis project I'm doing along with my other stuff

>> No.10815742

>>10815492
Ex-linguist that's been wanting to move into tat area for a while now

What makes it suck?

Currently working as a Data Scientist (really more like a Data Analyst though), trying to move in that direction, so I'm considering doing the degree in Edinburgh for Speech & Language Processing

>> No.10815744

>>10815405
Could always try Data Analyst first and move in

Could take a while though

>> No.10815747
File: 58 KB, 615x350, pentti.jpg [View same] [iqdb] [saucenao] [google]
10815747

Buddy of mine recommended this for ML learning:

https://www.fast.ai/

Anyone heard of them?

NLP seems to be a big thing with them

>> No.10815995

>>10815744
Kek tried that a bunch.
Every time they thought I was overqualified because of the MS (which I think is a really stupid reason). I have some more connections I'm gonna try though. I honestly haven't used any of my connections yet.

>> No.10815998

could I train a model to do eye tracking using images from a webcam? I was thinking of using 5 images, one for each corner and then one for the current image. this way I could calibrate it, then just refeed it the 4 images to give it more data to work with.

What kinda accuracy do you think I could get? How much data would I need to gather and how long would it take to train? I'm a programmer with a small bit of ML knowledge and read a book on keras. I don't want to sink too much time into this if it's hopeless. If I could get within a few centimeters of screen position that would be enough for what I'm doing.

>> No.10816174

>>10815995
>overqualified
There is no such thing. This excuse is ridiculous. Just say you don't wanna have incels in your team. Is it so hard?

>> No.10816549
File: 974 KB, 512x288, 68747470733a2f2f64656570.gif [View same] [iqdb] [saucenao] [google]
10816549

>>10809198
https://gen.studio/

>> No.10816565

>>10815995
I know a guy who worked for a starup for 3 montha and now works for another company full time being paid 80k and he hasn't even finished his degree yet.

Maybe you just need to apply to more places?

>> No.10816715

Very interesting talk about weird way of learning that ANN often use, and how to get rid of it.
https://www.youtube.com/watch?v=mUt7w4UoYqM

>> No.10816812

>>10815405
Which city?

>> No.10816923

>>10815747
This is for someone who already knows essential machine learning and wants to pick a specialization.
I wouldn’t say this is where you start to learn ML.
If you want to learn machine learning very well and on the cheap, dataquest.io is the way to go.

>> No.10817019

>>10816715

tl;dr?

i wonder whether this robustness, or lack thereof has anything to do with the linear nature of resnet and other modern architectures. would a l2-regularized net with tanh activations be as sensitive? i haven't done any work on adversarial perturbations

>> No.10817034

>>10817019
cont.

most numerical optimization methods rely on having some way to locally approximate the loss function. gradient descent, newtons method, and so forth. but there are many areas in the parameter space of an ReLU activated resnet where these approximations are very poor, of course due to the non-analytic, non-smooth ReLU. i have a hunch that this is why ReLU activated resnets are sensitive to small perturbations after training.

>> No.10817062

>>10817034
cont.

what's interesting is that these nets can still achieve very good performance, due to their non-linearity, the ability of each neuron to 'classify", but this is basically just because they're so over-parameterized that random initialization usually doesn't leave them far from a solution. gradient descent is blind to the negative inputs.

>> No.10817123

>>10817019
>>10817034
>>10817062
cont.

didn't watch the whole presentation but it looks like they suggest adding noise to the input. this would increase the range of activation patterns encountered during training and somewhat mitigate the problem but i still think the issue has to do at least partly with the ReLU.

call me an idiot but i still think it's likely that ReLUs and excessively deep (>50 layers) CNNs will fall out of favor, at least in applications where inference speed is a priority.

>> No.10817124

>>10815742
That might be a good degree (I know Edinburgh's a great school) but don't get the impression you need another degree.

NLP is still a young field where there's lots of room for innovation. If I were you I'd pull up a Yelp review dataset or something and get hacking with NLTK then move on to Word2Vec and other packages to test the waters.

>> No.10817129

>>10817019
>tl;dr?
Most cool part to me that you can actually distinguish robust and non-robust features and force the network to learn only robust ones. I wonder if it can be applied to GAN's discriminator and therefore achieve better results with same training time and processing consumption.

> wonder whether this robustness, or lack thereof has anything to do with the linear nature of resnet and other modern architectures. would a l2-regularized net with tanh activations be as sensitive?
They say that it cannot be the only answer. Somehow these non-robust features encoded into our datasets and have positive predictive value. i.e. noise that can fool a model contain some useful information that people just cannot see, but model overvalued this features and this is the problem.

>> No.10817163
File: 219 KB, 899x455, of_sorts.gif [View same] [iqdb] [saucenao] [google]
10817163

As planned (>>10806532) I made that introductory neural network video, elaborating on a version of the pure Python implementation in the Nielson book

https://youtu.be/z2aq21lMw40

>> No.10817192

>>10817163
I admire your enthusiasm, but please, try to compact you videos into 10-15 minutes range. Not so much people have time to watch more than half-hour long videos, so you essentially limit your potential audience significantly. Ofc, this only the advice.

>> No.10817328

Anyone have any experience in pipelining?
I’m trying to transform some datasets in Luigi

>> No.10817412 [DELETED] 

>>10817129
>They say that it cannot be the only answer. Somehow these non-robust features encoded into our datasets and have positive predictive value. i.e. noise that can fool a model contain some useful information that people just cannot see, but model overvalued this features and this is the problem.

perhaps, but keep in mind relus probably don't help the network learn wide margins. again, it can't "see" the negative values.

>> No.10817564

>>10809887
Damn dude, keep up the good work there.

>> No.10817576
File: 1.06 MB, 1308x564, zzz.png [View same] [iqdb] [saucenao] [google]
10817576

>>10816715
I just took a look at the paper (https://arxiv.org/pdf/1805.12152.pdf ) and damn this shit is amazing. Adversarial examples that actually make sense! Do you understand what it means anon? You can take any video from youtube and make a fucking nightmare out of it. Look at page 23.

>> No.10817595
File: 940 KB, 640x695, 333.png [View same] [iqdb] [saucenao] [google]
10817595

>>10817576
Also you can generate images like that, that super easy to misinterpreted.

>> No.10817630

What exactly is Machine Learning? Is it related to AI?

>> No.10817668

>>10817630
This might help you.
https://www.google.com/search?client=urmum&q=machine+learning&sourceid=dudetrustme&ie=UTF-8&oe=UTF-8

>> No.10817717
File: 292 KB, 1402x717, pepe.png [View same] [iqdb] [saucenao] [google]
10817717

>>10817595
You can even make pepe with it!
http://gradientscience.org/robust_apps/

>> No.10817769
File: 259 KB, 320x320, pepe2.png [View same] [iqdb] [saucenao] [google]
10817769

>>10817717

>> No.10817805

>>10817769
Fuckin saved, you made a nightmare fren

>> No.10818162

>>10817717
Bleeding out of my eyes but still happy.

>> No.10818337

>>10814177
I've only recently started to into programming, and for the life of me I cannot get torch and torchvision to work on PyCharm, despite installing them through CMD and verifying that the files are in the site-packages directory. I get a module not found error for torchvision. Any advice on finishing this installation?

>> No.10818540

>>10817769
Try for moonman

>> No.10818579

Anyone know the approximate difference in processing time between a dynamic and static computational graph?

>> No.10818665

>>10818579
It depends on what you're processing. Is this a homework question?

If you have a computation graph processing framework and it can produce either static or dynamic computation graphs, then the determination of which one will execute faster is based on the nature of what you are processing, specifically whether the parameter space of the description of the computation to process is sparse or dense. If it's sparse, dynamic computation graphs are faster. If it's dense, static computation graphs are faster. I can explain to you why if you're interested and don't already understand.

>> No.10818861

>>10818337
Complete at least couple of python guides before trying ML

>> No.10819100

>>10818665
No this is for research. Currently I’m using a dynamic graph (executing tensorflow code in eager mode) due to the constraints of the graph (monotonic and lower triangular structure) which make it hard to build a static computational graph. But this will eventually be used to process a ton pictures at very high resolution so I’m wondering if this will fuck me later down the road. How would you define a dense parameter space?

>> No.10819200

Tensorflow has those embedded TPU.
What about pytorch? Jetson nano?

>> No.10819257

Serious question: why do so many people go straight to deep learning instead of learning the ropes of machine learning?

Coming from a pure math background this is bizarre to me; you skipped thousands of steps. How are you supposed to even evaluate your models or understand the algorithms without a solid foundation in statistics?

>> No.10819487

>>10819257
I'm not sure if deep learning is a harder model than others. I.e. I'm tempted to argue that among machine learning frameworks, deep learning is one generally conceptually simple one.

>> No.10819497

How can computers learn when all they do is shuffle symbols around?

>> No.10819498

>>10819257
Deep learning is popular because of how easy it is. Many people do deep learning without understanding how it even works. That's how you know it's a powerful abstraction.

>> No.10819502

>>10819497
How can humans learn when all they do is move electrons and ions around?

>> No.10819510

>>10819257
Because it's the trendy new thing and people want to put it on their resumes

>> No.10819540

>>10817129
>They say that it cannot be the only answer. Somehow these non-robust features encoded into our datasets and have positive predictive value. i.e. noise that can fool a model contain some useful information that people just cannot see, but model overvalued this features and this is the problem.

i have to wonder. take their example with two gaussians, and train an relu-activated neuron to separate them. without relu, the gradients would oppose each other, and the separating hyperplane would fall into the middle between the two, probably with some margin on either side, ensuring robustness. but with an relu, wouldn't the separating hyperplane be forced right up against one class, because it's "pushed" that way by the other and there are no opposing gradients until some of the inputs become positive?

correct me if i'm wrong, but i think that relu probably does not encourage wide separating margins. i'll go out on a limb and guess that this might contribute to its sensitivity to seemingly imperceptible perturbations.

>> No.10819622

>>10819200
TPUs are compute devices in Google's data center. There's support for them in Tensorflow, but you have to adapt your code to actually use them and rent TPU compute. Most people use it with GPU. Torch used to be faster than Tensorflow on GPU.

>> No.10819657

>>10819540
cont.

in a way, the ReLU wouldn't so much learn a separator but rather, it would ignore inputs that very likely belong to one of the classes. perhaps DCNNs with relus do something akin to the process of elimination.

>> No.10819664

>>10819487
I guess you’re right. I always “look up” to it because I’m so focused on stats but I guess if you can engineer it it works.

>> No.10819693

>>10809198
Is anyone here into NLP? That's by far the most useful AI tool. Highest potential, for sure. Imagine the day you could type in a few tags, and an AI generates a book, or something similar.

I'm making a summarizer in pytorch as my bachelor degree "master thesis" (what would this even become..? Bachelor thesis..?). It's great fun!

>> No.10820113

>>10816549
How dis ?

>> No.10820342

>>10819622
I meant those tpu edge stuff. But it works only with tensorflow lite.
In contrast those Jetson nano are cuda enabled

>> No.10820389

>>10819622
>>10820342
You can't train the model on the TPU. So they basically useless.

>> No.10820622

>>10819693
I'm gonna suck off someone at openai for the gpt2 source

>> No.10820679

I have a 2.6 GPA in Math from an average school. Am I smart enough for machine learning? I'm just a lowly c++ programmer right now.

>> No.10820680

>>10820622
https://github.com/openai/gpt-2
Where can I get my suck? Also, are you qt?

>> No.10820725

>>10820622
>It started out with some harmless handwritten digit image recognition in his spare time
>now he's sucking off post docs in the alley behind Chili's for gpt2 source code
AI, not even once

>> No.10821347

Last thread I started labeling the czech casting dataset by pose, but I only got about 1/4 through before I get too lazy. I'll upload what I got in two archives. The first is 9268 pictures with manually checked labels. Second is 32626 pictures that were labeled automatically. The classifier works well, but there will be some errors. I hope somebody can finish checking the labels. Some stats from the checked label part:

face 420
dressed_facing_front 1178
dressed_facing_back 51
dressed_facing_back_looking_over_shoulder 79
dressed_facing_left 469
dressed_facing_right 561
dressed_facing_front_upper_body 584
dressed_facing_left_upper_body 15
dressed_facing_right_upper_body 81
topless_facing_front 436
topless_facing_back 144
topless_facing_back_looking_over_shoulder 147
topless_facing_left 225
topless_facing_right 303
topless_facing_front_upper_body 214
topless_facing_left_upper_body 4
topless_facing_right_upper_body 35
naked_facing_front 504
naked_facing_back 380
naked_facing_back_looking_over_shoulder 409
naked_facing_left 236
naked_facing_right 307
naked_facing_front_upper_body 339
naked_facing_left_upper_body 10
naked_facing_right_upper_body 70
naked_sitting 426
naked_sitting_spread_hiding_pussy 59
naked_sitting_spread_showing_pussy 442
left_tit 399
right_tit 394
pussy 329
unclassified 18

The filenames contain the girls name and number. There are 2042 girls in the whole dataset. The checked label archive is here:
https://mega.nz/#!IbpVTYaB!NnfNi8M2TyLOOvQR28izDSenrvQPCU-OVpqihcNO1EI
I post the second archive in a second post when it's uploaded.

>> No.10821431

>>10821347
Here is the second part with labels that need to be checked:
https://mega.nz/#!AL4knSyD!uE2JFHluoLnAGytlwT3VA3JwZUb_O5kS1PtadZZJRz4

face 1473
dressed_facing_front 4321
dressed_facing_back 55
dressed_facing_back_looking_over_shoulder 74
dressed_facing_left 2087
dressed_facing_right 2039
dressed_facing_front_upper_body 1869
dressed_facing_left_upper_body 7
dressed_facing_right_upper_body 82
topless_facing_front 1631
topless_facing_back 395
topless_facing_back_looking_over_shoulder 362
topless_facing_left 831
topless_facing_right 1048
topless_facing_front_upper_body 672
topless_facing_left_upper_body 3
topless_facing_right_upper_body 26
naked_facing_front 1610
naked_facing_back 1554
naked_facing_back_looking_over_shoulder 1792
naked_facing_left 945
naked_facing_right 973
naked_facing_front_upper_body 1213
naked_facing_left_upper_body 5
naked_facing_right_upper_body 54
naked_sitting 1608
naked_sitting_spread_hiding_pussy 297
naked_sitting_spread_showing_pussy 1242
left_tit 1602
right_tit 1627
pussy 1129

>> No.10821590
File: 128 KB, 1080x1350, 1467069577329.jpg [View same] [iqdb] [saucenao] [google]
10821590

>>10821347
>>10821431
>Last thread I started labeling the czech casting dataset by pose
>9268 pictures with manually checked labels

Most Autistic Person 2019

>> No.10821617

>>10821590
Not autistic enough, because I didn't finish the job. Also it's basically just looking at porn.

>> No.10821650
File: 2.46 MB, 2508x2558, yann-lecun-facebook-03-10-17_4932-edit-copy.jpg [View same] [iqdb] [saucenao] [google]
10821650

>>10821590
And now imagine how mnist or cfar10 was made somewhere in middle ages. And they was labeled by very few people, in case of mnist just one.

>> No.10821744

>>10821650
I don't know, doing monkey work labelling at the but paid a premium to do so is pretty based when you don't feel like working hard
And we label things for free with captcha

>> No.10821910

>>10821347
Looking at those body part folders is kind of weird.
https://files.catbox.moe/60frrz.jpg

>> No.10821932

>>10813163
what's your background?

>> No.10822336

>>10820679
You have a math degree and experience programming in a low-level language. I'd say you could get a machine learning engineer job if you started knocking out MOOCs.

>> No.10823524

Quite a good review of neuralink by actuall neuroscientists/machine learning people. Not much new if you seen the presentation, except of potential problems that was purposefully omitted.

https://youtu.be/B2-YiXuXdp8

>> No.10823851

>>10822336
The machine learning jobs I found require PhD and an outstanding on your GitHub.
And most of them are no-name small startup companies.
Maybe different on larger companies, but the hiring process there is a pain in the ass and looks like a black box lottery system.

>> No.10823852

>>10823524
what do you mean I'm banned from digital agriculture

>> No.10823952

>>10823851
I got an ML job with just a CS degree and no projects. Just talk about regressions and tensorflow and it's all good.

>> No.10823963

>>10823952
What company? What's your role? How much is the pay?

>> No.10823976

>>10823952
What city?
I'm pretty confident in my ML skills (I like pytorch better tho), but my home projects are not impressive because I lack good hardware. I am in a PhD course in robotics and not AI and my job applications get ignored basically.

>> No.10823991

>>10823976
Don't bother with ML. You don't have any relevant experience. The PhD course is not that quantitative anyway. Competition is too fierce in this industry.

>> No.10823992

>>10814159
>data science team at my work is comprised of people in non STEM degrees
>literally someone from hr in the team
>no one knows sql and can barely even use excel
>only can manage python if it's following an online tutorial to the t
>projects go months over the time line because they can just throw around buzzwords to out of touch managers
>ends up copy and pasting some python library tutorial
>praised for their innovation

This area is filled to the brim with retards and its unbelievable

>> No.10824029

>>10823991
I was getting rejected at the job I was really looking for and figured out "fuck this, I'm gonna try my luck at those ML stuff too". Now that I am relatively indifferent regarding which job I get, I might as well surf with the hype.

>> No.10824096

So how I learn what a good architecture is? When I'm playing in PyTorch it feels like I'm just guessing what layers and filters to throw in at random.

>> No.10824097

Hey guys, is a course in Reinforcement Learning worth taking beyond that standard ML and DL courses?

>> No.10824152

Were there any progress on the pepe or deepnude project itt appart from the release of the pepe program?

>> No.10824519

>>10823851
>>and an outstanding on your GitHub
Huh? Is there some rating?

>> No.10824637

>>10824519
He means a Github that is instantly recognised as belonging to a very intelligent person.

>> No.10824647

>>10824519
I meant outstanding project.
Like your dotfiles don't count.
I mean that's what I've been asked for. And since my GitHub is basically shitposting and basic but trivial scripts, it didn't count. And I was denied the programming interview, and straight refused because I lacked computer science skills as they said.

>> No.10824691

are there any datesets from HR departments, like applications + pictures? with y = 1 if good employee?

>> No.10824698

I’m working on implementing neat from scratch in java, and also have the only open source fully ndimensional es hyperneat algorithm that I’m training to trade shitcoin if your interesting in helping or looking at these projects reply to this post and I’ll link them

>> No.10824703

>>10824691
There should be because I seen some generated employee profiles, but too lazy to search for it right now.

>> No.10824728

>>10824691
Here's a dataset for you:

Bisexual black woman: y=1
Heterosexual white man: y=-1

>> No.10824798

>>10817328

Still wondering if anyone has Luigi experience

>> No.10824802

>>10824097
Udacity has a bunch of stuff like that.
You don't have to pay for it if you don't want, the content's publicly available.

>> No.10824915
File: 58 KB, 489x700, 50a87a5c0d09c.jpg [View same] [iqdb] [saucenao] [google]
10824915

>>10824798
My whole life is Luigi experience.

>> No.10824940

I'm on the last chapter on "introduction to statistical learning", I skipped all the exercises in R.

Is this a bad idea?
Do anybody know where I can find a book which implements basic ML concepts (Linear regression, Logistic regression, Cross validation, Polynomial regression, Decision trees, Support vector machines and methods for unsupervised learning such as K-nearest neighbor) with Python.

>> No.10824983

>>10824940
here are the exercises and chapters implemented in python


https://github.com/a-martyn/ISL-python
https://github.com/JWarmenhoven/ISLR-python
https://github.com/emredjan/ISL-python
there is more

>> No.10825026

>>10824983
This is perfect, thanks!

>> No.10825329
File: 65 KB, 375x499, deeplearning.jpg [View same] [iqdb] [saucenao] [google]
10825329

What are the most relevant chapters in Goodfellow? I plan to go through all of it eventually.

>> No.10825700

How to get into python for machine learning projects?

>> No.10825865

>>10825700
Just start doing guides. That's it.

>> No.10825887

>>10810369
>>10810637
You could create set pivot points like how animators do with 3d models, and then just generate angles for the pivots

>> No.10825889

>>10811122
I'm gonna dab on other boards with this webm

>> No.10825990

>>10825329
after the math chapters, why not just read the chapters in order?

>> No.10826137

Am I schitzo or is this pure fucking kino?
Prompt
>The lords of starfell would send forth adventurers to collect the remains of fallen stars, in the hopes to stave off the inevitable Eternal Night.
Generated text
>The lords of starfell would send forth adventurers to collect the remains of fallen stars, in the hopes to stave off the inevitable Eternal Night. The night was short-lived, however—the gods soon realized that the dead starlings had no use for the endless cycles of sleep and rise, and were unwilling to keep them in a mindless prison. They banished the beasts and their inhabitants to the nightlands, where, while the stars did not wane, the stars themselves began to wane, and the gods were forced to summon the star-watchers and prepare to send forth the night, only to find that while the stars once existed—they were not needed. The stars are not still, and their presence has long since faded from this realm. The night is here in the sky for everyone with a soul."
—The Nightwatch
>"The sky over the moon is the starlight. The night has no power or purpose here. You stand on a cliffside watching your home fall under the spell of the stars...but there is more at play than simply falling in love with an empty sky."
—Gods, Stars

I must be insane, because I'm feeling some deeper meaning in this block of text that couldn't possibly actually be there.

>> No.10826162

>>10826137
Damn, now it's doing nothing just ripping off entire blocks of text from established settings like GoT or D&D

>> No.10826192

>>10823851
Ask someone working on applied ML at Google anything lmao.

>> No.10826247
File: 57 KB, 270x385, 0A853B7C-507B-4048-865F-7760C6842369.jpg [View same] [iqdb] [saucenao] [google]
10826247

Did you learn optimal transport today /MLG/?

https://arxiv.org/abs/1803.00567
https://optimaltransport.github.io/book/

>> No.10826490

Can someone link the Machine Learning reading list?

I know PRML but what are some other books?

>> No.10826520

>>10826192
do you have to wear a buttplug to work

>> No.10827106
File: 25 KB, 1024x683, god.jpg [View same] [iqdb] [saucenao] [google]
10827106

>>10826192
How much of your time is waiting for new hardware and trying to turns screws till it's actually as fast as the designers claim?

More interestingly, do you make use of ML for something that's not on every bodies mind just yet?

>> No.10827456
File: 262 KB, 439x583, 1451e7f9-3c34-436e-972c-49b2f03201d8.png [View same] [iqdb] [saucenao] [google]
10827456

https://twitter.com/OpenAI/status/1153289143964725249
based

>> No.10827474

What are the research areas of machine learning that uses advanced math (more than basic LA or stats)?

The only ones I know are topological data analysis and hyperbolic embeddings.

>> No.10827491

>>10827474
I heard a lot of ideas how to build autproofer with AI. But presumably all of them failed.

>> No.10827493

>>10827474
The optimisation of the a neural network can involve some interesting math. That's if you're looking beyond the typically used stochastic gradient descent.

>> No.10827507
File: 61 KB, 675x1200, Ds0vDceVsAA_zZW.jpg [View same] [iqdb] [saucenao] [google]
10827507

I've been learning from MIT's deep learning basics course, any recommendations for where to go after?

>> No.10827540

>>10827474
Convex analysis, Riemann geometry, functional analysis in statistics learning, optimal transport.

http://geometricdeeplearning.com/
https://github.com/yao-lab/yao-lab.github.io/blob/master/book_datasci.pdf

Hamiltonian Monte carlo methods
https://arxiv.org/abs/1701.02434
Using jets operators for higher order automatic differentiation.
https://arxiv.org/abs/1812.11592

>> No.10827745

>>10827507
This lexeis guy (or whatever his name) with the short hairs?
Man what a poser. Same tier as this YouTubeIndian guy with the white hair spot and the soiboi faces on the thumbnails that keeps popping up in the recommend videos.