Posts Tagged: Scala

Dec 12

My Education in Machine Learning via Coursera, A Review So Far

As of today I’ve completed my fifth course at Coursera, all but one being directly related to Machine Learning. The fact that you can now take classes given by many of most well known researchers in their field who work at some of the most lauded institutions for no cost at all is a testament to the ever growing impact that the internet has on our lives.  It’s quite a gift that these classes started to become available at right about the same time as when Machine Learning demand started to sky rocket (and at right about the same time that I entered the field professionally).

Note that all effort estimations include the time spent watching lectures, reading related materials, taking quizes and completing programming assignments.  Classes are listed in the order they were taken.

Machine Learning (Fall 2011)
Estimated Effort: 10-20 Hours a Week
Taught by Andrew Ng of Stanford University, this class gives a whirlwind tour of the traditional machine learning landscape. In taking this class you’ll build basic models for Regression, Neural Networks, Support Vector Machines, Clustering, Recommendation Systems, and Anomaly Detection. While this class doesn’t cover any one of these topics in depth, this is a great class to take if you want to get your bearings and learn a few useful tricks along the way. I highly recommend this class for anyone interested in Machine Learning who is looking for a good place to start.

Probabilistic Graphical Models (Spring 2012)
Estimated Effort: 20-30 Hours a Week
Taught by Daphne Koller of Stanford University, who de facto wrote The Book on Probabilistic Graphical Models (weighing in at 1280 small print pages). This class was a huge time investment, but well worth the effort. Probabilistic Graphical Models are the relational databases of the Machine Learning world in that they provide a structured way to represent, understand and infer statistical models. While Daphne couldn’t cover the entire book in a single class, she made an amazing effort of it. After you complete this course you will most definitely be able to leverage many different kinds of Probabilistic Graphical Models in the real world.

Functional Programming Principles in Scala (Fall 2012)
Estimated Effort: 3-5 Hours a Week
Taught by Martin Odersky who is the primary author of the Scala programming language. I entered this class with an existing strong knowledge of functional programming and so I’d expect this class to be a bigger time investment for someone who isn’t quite as comfortable with the topic. While not directly related to Machine Learning, knowledge of Scala allows leverage of distributed platforms such as Hadoop and Spark which can be quite useful in large scale Entity Resolution and Machine Learning efforts. Also related, one of the most advanced frameworks for Probabilistic Graphical Models is written in and designed to be used from Scala.  In taking this class you’ll most certainly become proficient in the Scala language, but not quite familiar with the full breadth of its libraries. As far as functional programming goes, you can expect to learn quite a bit about the basics such as recursion and immutable data structures, but nothing so advanced as co-recursion, continuation passing or meta programming. Most interesting to myself was the focus on beta reduction rules for the various language constructs.  These together loosely form a primer for implementing functional languages.

Social Network Analysis (Fall 2012)
Estimated Effort: 5-10 Hours a Week
Taught by Lada Adamic of the University of Michigan. Social Network Analysis stands out from the others as I’ve never been exposed anything quite like it before. In this class you learn to measure various properties of networks and several different methods for generating them. The purpose in this is to better understand the structure, growth and spread of information in real human social networks. The focus of the class was largely on intuition and I was a bit unhappy with the sparsity of the mathematics, but this certainly makes it a very accessible introduction to the topic. After completing this class I guarantee you’ll see new insight into your corporate structure and will see your twitter network in a whole new way. If I were to pick one class from this list to recommend to a friend no matter background or interest, this would be it.

Neural Networks for Machine Learning (Fall 2012)
Estimated Effort: 10-30 Hours a Week
Taught by Geoffrey Hinton of the University of Toronto, who is a pioneer and one of the most well respected people in his field. Note that going in you’ll be expected to have a strong working knowledge of Calculus, which is not a prerequisite for any of the other classes listed here. I had hoped that this class would have been as worthwhile as the Probabilistic Graphical Models course given its instructor, but sadly it was not. Regretfully, I can only say that this class was poorly put together. It has a meager four programming assignments, the first two of which follow a simple multiple choice coding formula, and the following two of which are unexpectedly much more difficult in requiring you to both derive your own equations and implement the result of that derivation. It was extremely hard to predict how much time I would need to spend on any given assignment.  Having already learned to use Perceptrons and simple Backpropagation in Andrew Ng’s class, the only new hands-on skill I gained was implementing Restricted Boltzmann Machines. To be fair, I did acquire quite a bit of knowledge about the Neural Networks landscape, and Restricted Boltzmann Machines are a core component of Deep Belief Networks.  However, looking back at the sheer quantity of skills and knowledge I gained in the other classes listed here, I can’t help but feel this class could have been much better.

Jan 12

Musicians, Mechanics, and Mathematicians

Thank you all for your comments on my previous post, I appreciate the time you all took in sharing your perspectives very much.  Many of you have brought up great analogies to demonstrate how you feel and in reading these responses I realized I must not have been very clear.

There are some musical geniuses who have composed great works without having been taught even the basics of music theory. However, this doesn’t mean they’re not doing math. The human brain excels at building approximate mathematical models and a rare few minds are capable of building exceedingly complex ones. Still, formal knowledge of the patterns of music allow a musician to both play the same song in new and interesting ways and see the underlying connections between different pieces. As a composer it informs how rhythms and melody can be juxtaposed or fitted together to create a desired effect in a way much more profound than trial and error. It expands the musician’s mind and makes them better at what they do. 

Another great example is that of the wrench wielding mechanic. There are a great many mechanics who went to trade school and learned the high level basics of engines and a long list of procedures. They might not understand the details of combustion or material science but they can replace brake pads or swap in a new timing belt without too much difficulty. After many years of experience some may have even built a mental model so superb that they can take you for a spin around the block and tell you exactly what’s wrong.

And still, as the mechanic reaches for their bolts and wrench they might not think of the underlying mathematics of what they are about to perform. Yet, you can be sure the people who made those tools worried over it greatly. If they didn’t the wrench would not fit the head or worse, the bolt might shear under the stress applied. While they surely tested many bolts before shipping their product, they certainly didn’t before creating a formal model of how the tools were shaped or how they would perform. Even if the tools might happen to work without testing, they probably wouldn’t work very well and to sell tools made in this way would be grossly negligent.

Yet, I can’t be the only one who has suffered many near catastrophes at the hands of inept mechanics over the years. From the time a post-brake change air bubble in my brake line made my car roll out into traffic or the punctured gas tank that almost left me stranded at the side of the road. One might wonder if they even bothered testing their work.

Some might think programmers shouldn’t be beholden the same strictness as our creations aren’t usually capable of quite so much damage. Instead, the worst things most are liable to do are destroying important information, providing incorrect data to critical systems, leaking private information, sharing unsalted (or worse, unencrypted) passwords or causing people to become unable to access their bank or medical records. No big deal really, just shoveling data.

I’d love to see every programmer taking the time to learn deeply about the mathematical modeling of data and programs, but I know that’s not reasonable. However, it takes just a little bit of learning to leverage tools with very complex underlying mathematics made by others. You don’t need to be an expert in category theory to use Haskell any more than you need to be an expert in set theory to use SQL. F# and Scala are even more accessible as they have access to all of the libraries and patterns you would be familiar with as a programmer who works in .NET or Java.

So, I’m not asking that you go out and spend years studying your way to the equivalent of a PhD. Instead what I ask is that you just take a little time to understand what’s possible and then use tools made by people who do have that kind of deep understanding.

I know I wouldn’t want to drive a car with parts made by someone who didn’t use models, would you?

Huge thanks to @danfinch and @TheColonial for proof reading.