[ 3 / biz / cgl / ck / diy / fa / ic / jp / lit / sci / vr / vt ] [ index / top / reports ] [ become a patron ] [ status ]
2023-11: Warosu is now out of extended maintenance.

/sci/ - Science & Math

View post   
File: 550 KB, 1536x6188, GoogLeNet.png [View same] [iqdb] [saucenao] [google]
7544453 No.7544453 [Reply] [Original]

Any Deep Net fag here? How does one know what structure of a deep net would bring the best result for a certain problem? Experience? Grid searching? I'm asking this because I'm not a richfag in a not so rich lab, each time I train a dataset with 5000 images, it would take 3-4 consecutive days on our server computer.

Pic related, GoogLeNet model for image classification. It contains about 30 layers.

Also, deepnet/machine learning/AI general I guess.

>> No.7544537

Well, I guess not many people who do deep learning go to /sci/, they are busy making money.

>> No.7544557

>How does one know what structure of a deep net would bring the best result for a certain problem?
You don't. Theoretically you could do some model selection/averaging in a Bayesian way but that's not possible with neural nets for performance reasons. Typically you will find a paper that deals with a similar dataset and rip the architecture off, maybe do some tweaking. The rule of thumb is to have enough parameters so you can overfit the model and drive the loss to some very small number and then apply heavy regularisation.

>> No.7544590

>>7544557
Well shit, I am already do that. But it seems like the whole process is not very efficient, at least for poorfags.

>> No.7544629
File: 27 KB, 325x358, 1399555947903.jpg [View same] [iqdb] [saucenao] [google]
7544629

>>7544453
What a long ass graph. Don't tell me those are what behind the 4chan image captchas.

>> No.7544692

>>7544629
Well yes, they are.

>> No.7544903

>Pointless threads about hard AI, Skynet, terminator, world domination by robots, etc. are always hot
>Real thread about AI, no one gives a shit
Welp.

>> No.7544916

How would being a richfag assist you?

Basically you're asking how to work with what you have

>> No.7544930

>>7544916
>How would being a richfag assist you?
It helps me train my model faster. So in cases where I picked the wrong params, I can retrain another set of params without losing much time.

>> No.7545069

How powerful is your server?
Have you tried renting some EC2 spot instances?
While you need to pay for them, it might be worth it if your server sucks and if you only need some time on a good machine to tune your model (we recently used them at my company to quickly train some models using GA and it was worth it).

Spot Prices: http://ec2price.com/
Instance Types: https://aws.amazon.com/ec2/instance-types/

Some examples:

m3.2xlarge - 8 Cores / No GPU / 30 GB RAM / 2 x 80 GB SSD -> ~ $0.10/hour
c3.8xlarge - 32 Cores / No GPU / 60 GB RAM / 2 x 320 GB SSD -> ~ $0.30/hour
g2.8xlarge - 32 Cores / 4 GPUs / 60 GB RAM / 2 x 120 GB SSD -> ~ $0.40/hour

P.S.: There might be some cheaper cloud providers out there, I have no idea.

>> No.7545070

>>7544916
>How would being a rich fag assist you

See >>7545069

>> No.7546099

>>7545069
The sever has about 16 cores with 60 GB RAM I think, no GPU.

>> No.7546210

>>7545069
>$0.10/hour
Shit, that was fucking neat, never knew about these. I think we can afford those.

>> No.7546602

>>7546210
If you are not familiar with EC2, note that I said SPOT INSTANCES, not regular instances, which are sold in an auction fashion.

Regular instances are more expensive.

>> No.7547207

>>7546602
Yeah, I can see that from the graph.

>> No.7547457

>>7544903
It is to be expected since /sci/ consists of mostly teenager tier /b/tards who think they know shit after watching a few youtube videos.

>> No.7548866

>>7544453
What is FC in that network?