/sci/ - Science & Math

>sentiment analysis

Currently farming a dataset for positive/negative sentiment posts from twitter. Using the keywords :) and :(

Here's the script: https://pastebin.com/uhr0LJrK

Ignore the passive aggressive classifier there, was just trying something out. The reason I'm alternating streams is because twitter doesn't let you filter two things at the same time because they want $ so I just worked around it.

Still gotta figure out how to collect neutral sentiment posts. Any ideas? Anyone want to help me collect the data set? I'm going to host it for us to play with it

>>9704080
I've got good programming experience, so I know what I'm doing. When I said I need to learn, I meant I need to learn the underlying math of statistical learning which I think is very important. I would relate it to the analysis of algorithms: knowing big O notation gives you a huge edge over other programmers.

Here is my current workflow:
>Need to accomplish x task
>Look up how to accomplish it from some material
>Understand it, and do it
>Afterwards, look up the underlying math for the techniques I'm using and its caveats (this is what I mean by learn afterwards)

For example: I'm working on classifying text. I look up on methods to do it. I learn that first I need to vectorize the text, and then train an SVM. I do this, and run up some metrics against it. After completing this, I bust out a piece of paper and jump on wiki or my textbook and write up how an SVM works, what f1 is, different methods of word vectorizing etc. I look up the documentation of what I just used, and take note and try out different things that are available. This would be the learning part.

I don't see how this technique of learning is inefficient.

>>9704080
I have 5 years of programming experience, and have worked full time for 2 years. I know what I'm doing. When I said I need to learn, I meant I need to learn the underlying statistics of statistical learning which I think is very important. I would relate it to the analysis of algorithms: knowing that material gives you a huge edge over other programmers.

Here is my current workflow:
>Need to accomplish x task
>Look up how to accomplish it from some material
>Understand it, and do it
>Afterwards, look up the underlying math for the techniques I'm using and its caveats
>>9704080
I have 5 years of programming experience, and have worked full time for 2 years. I know what I'm doing. When I said I need to learn, I meant I need to learn the underlying statistics of statistical learning which I think is very important. I would relate it to the analysis of algorithms: knowing that material gives you a huge edge over other programmers.

Here is my current workflow:
>Need to accomplish x task
>Look up how to accomplish it from some material
>Understand it, and do it
>Afterwards, look up the underlying math for the techniques I'm using and its caveats (this is what I mean by learn afterwards)

For example: I'm working on classifying text. I look up on methods to do it. I learn that first I need to vectorize the text, and then train an SVM. I do this, find that I my f1 score is 0.92.

After completing this, I bust out a piece of paper and jump on wiki or my textbook and write up how an SVM works, what f1 is, etc. This would be the learning part. I don't see how this is inefficient.

>>9657063
Software engineering 3; group project. My group dropped the course after the first iteration. Essentially, we had a month to implement a board game online in a browser and make it work across multiple clients on different computers. The game itself has a lot of different rules and different states. I've worked as a software developer, and I'm confident in my abilities. But man, coding 8h straight drains you. At work, I was at most working 3-4h focused at most, and the rest of the time I was helping and in meetings.

>>9657066
I think this is the best course of action. Most of the class has dropped it at this point so I don't think this will be a bad idea. I think my salt will have some significance as I have actually finished the project and will likely pass the course with an alright grade.

Unfortunately, I've had to skip all my other classes to have time to code this thing. It sucks because I really enjoyed my other classes; analysis of algorithms, wireless networks, and computer security. Looking back at my commits, I'm realizing I put in 45h a week for the past two.

I'm just glad I'm done it. I'm gassed.