Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Overview

From The Discipline of Machine Learning by Tom Mitchell:

The field of Machine Learning seeks to answer the question "How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?" This question covers a broad range of learning tasks, such as how to design autonomous mobile robots that learn to navigate from their own experience, how to data mine historical medical records to learn which future patients will respond best to which treatments, and how to build search engines that automatically customize to their user's interests. To be more precise, we say that a machine learns with respect to a particular task T, performance metric P, and type of experience E, if the system reliably improves its performance P at task T, following experience E. Depending on how we specify T, P, and E, the learning task might also be called by names such as data mining, autonomous discovery, database updating, programming by example, etc.

High level machine learning problems include:

  • supervised learning (tag);
  • unsupervised learning (tag);
  • semi-supervised learning (tag);
  • outlier or anomaly detection (tag); and
  • reinforcement learning (tag).

References

The following threads have details of references on the subject:

The following journals are dedicated to research in Machine Learning:

20090 questions
39
votes
3 answers

Application of machine learning methods in StackExchange websites

I have a Machine Learning course this semester and the professor asked us to find a real-world problem and solve it by one of machine learning methods introduced in the class, as: Decision Trees Artificial Neural Networks Support Vector…
Isaac
  • 1,003
36
votes
1 answer

explain meaning and purpose of L2 normalization

Let me say at the outset that I am very new to machine learning, and not great at math. I understand what TF-IDF does, but in the book I am reading it also notes the following (it's discussing how scikit-learn does things): Both classes…
Stephen
  • 1,076
33
votes
3 answers

How to judge if a supervised machine learning model is overfitting or not?

Can anyone tell me how to judge if a supervised machine learning model is overfitting or not? If I don't have an external validation dataset, I want to know if I can use ROC of 10 fold cross validation to explain overfitting. If I have an external…
user22062
  • 1,419
  • 3
  • 17
  • 22
33
votes
3 answers

Number of features vs. number of observations

Are there any papers/books/ideas about the relationship between the number of features and the number of observations one needs to have to train a "robust" classifier? For example, assume I have 1000 features and 10 observations from two classes as…
Leo
  • 2,634
30
votes
2 answers

What is the definition of Top-n accuracy?

I'm reading a Scientific paper on image classification. In the experimental results they speak of top-1 and top-5 accuracy but i've never heard of the term, nor can find it using google. Can someone give me a definition or point me somewhere? :)
Enoon
  • 402
28
votes
1 answer

What is behind Google Prediction API?

Google Prediction API is a cloud service where user can submit some training data to train some mysterious classifier and later ask it to classify incoming data, for instance to implement spam filters or predict user preferences. But what is behind…
user88
27
votes
2 answers

Do all machine learning algorithms separate data linearly?

I am an enthusiast of programming and machine learning. Only a few months back I started learning about machine learning programming. Like many who don't have a quantitative science background I also started learning about ML by tinkering with the…
Eka
  • 2,251
25
votes
2 answers

Why is pruning not needed for random forest trees?

Breiman says that the trees are grown with out pruning. Why? I mean to say that there must be a solid reason why the trees in random forest are not pruned. On the other hand it is considered very important to prune a single decision tree to avoid…
Z Khan
  • 251
25
votes
2 answers

Classifier vs model vs estimator

What is the difference between a classifier, model and estimator? From what I can tell: an estimator is a predictor found from regression algorithm a classifier is a predictor found from a classification algorithm a model can be both an estimator…
user46925
20
votes
2 answers

Does KNN have a loss function?

I didn't find a definition of loss function on wiki in the context of machine learning. this one is less formal though, it is clear enough. At its core, a loss function is incredibly simple: it’s a method of evaluating how well your algorithm models…
JJJohn
  • 1,875
  • 2
  • 12
  • 28
19
votes
3 answers

Machine learning curse of dimensionality explained?

I'm having trouble understanding the curse of dimensionality. Specifically, I came across it while doing the scikit-learn tutorial in python. Can someone please explain the below in a simpler manner? Sorry I have been trying to understand for the…
Chowza
  • 495
19
votes
2 answers

Average precision vs precision

Using sklearn.metrics in Python, I calculated average precision (with average_precision_score) and precision (with classification_report) while testing model metrics. However, I got different answers (0.89 vs 0.93, respectively). I read the…
bcarmel
  • 193
16
votes
2 answers

Definition of Regressor

Hi i have been learning Machine Learning for over six months, and I started to see the term Regressor and can 't really google it out for an exact definetion. I fully understand logistic regression and linear regression..but just can't figure out…
r poon
  • 301
  • 1
  • 2
  • 6
15
votes
2 answers

Supervised learning with "rare" events, when rarity is due to the large number of counter-factual events

Suppose you get to observe "matches" between buyers and sellers in a market. You also get to observe characteristics of both buyers and sellers which you would like to use to predict future matches & make recommendations to both sides of the market.…
John Horton
  • 253
  • 2
  • 7
15
votes
1 answer

Understanding no free lunch theorem in Duda et al's Pattern Classification

I have some questions about the notations used in Section 9.2 Lack of Inherent Superiority of Any Classifier in Duda, Hart and Stork's Pattern Classification. First let me quote some relevant text from the book: For simplicity consider a…
Tim
  • 19,445
1
2 3
29 30