What are the mathematical prerequisites for adaptive machine learning algorithms?

Problem Detail: 

I am a PhD student in Computer Science who switched his PhD a little bit towards ML algorithms combined with something else... I am an expert in that something else, say image processing, but not an expert in Machine Learning. What should I read/learn to get the ML mathematics right? Especially for adaptive learning algorithms.

Asked By : paxRoman

Answered By : ScienceGuyRob

The precise mathematical skills that you'll need for your studies will vary depending on exactly which area of ML you'll be working in. In general though Bayes theorem, probability theory and statistics will be your constant companion. Getting a good grasp of these is extremely helpful when it comes to fully understanding how learning algorithms work, and how they can be modified.

Partial differential equations, linear algebra, and mathematical induction are also very important. However beyond these it is difficult to say exactly which areas you should brush up on. In my opinion it depends on what problem you are trying to solve.

For instance is it a regression problem? Is it a classification problem? Are you clustering data? You mention adaptive algorithms, but many ML algorithms could be considered adaptive. As you become familiar with the ML field you'll find that people use different terms to describe exactly the same thing :) a little frustrating to be honest! But the second link you provided is from a non - computer science journal, so just be aware the definition of adaptive in this paper may mean something different in ML. For instance I don't work in adaptive algorithms, but my work has to adapt to high throughput data streams so is most certainly adaptive. But it falls under the category of Semi-supervised learning.

I would recommend taking a look at the following books for more information:

Machine Learning by Tom M. Mitchell, 1997 - This is a great and relatively easy going introduction to machine learning. Despite being over ten years old, this book covers the key topic areas, and introduces you to important concepts. Indirectly this will introduce you to the mathematics of Machine Learning. You'll be able to tell from this book what type of mathematics skills your particular problem may require.

Artificial Intelligence: A Modern Approach by Stuart Russell & Peter Norvig, 3rd Edition, 2009 - All encompassing text (1000+ pages) covers machine learning in great detail. Authoritative text.

Pattern Recognition and Machine Learning (Information Science and Statistics), Christopher M. Bishop, 2006 - Excellent authoritative text that goes in to ML in great depth. This includes introductory chapters that get you up to speed with the mathematics required to understand learning algorithms (including probability). Can be hard going if you have no mathematics experience.

To support any reading you do I suggest you take a look at the Andrew Ng machine learning course on Coursera. I watched some of these before starting my own study, so too did many of my friends.

Best Answer from StackOverflow

Question Source : http://cs.stackexchange.com/questions/3510