What is Machine Learning?

By definition from Tom Mitchell:

A computer program is learning form experience E w/ respect to some class of tasks T, and performance measure P, if its performance at tasks in T, as measured by P, improves w/ experience E.

Breaking it down into simple terms…

  1. You improve over tasks T.
  • Ex: Predicting the price of the house in the market.
  1. W/ repect to some performance P
  • Ex: How much the house is sold for.
  1. Based on some experience E
  • Ex: Data of houses sold in some specific year and the characteristics of the house (how many floors, age of the house, etc.).

This example (predicting the price of the house) is an example of supervised learning.

Machine learning requires explicit learning.

  • It requires data in order for it to learn
  • It is ruled-based.

Why do we need to learn?

  • Because we are inexperienced and are trying to be experts at something.

What are the diffrent types of machine learning?

  1. Supervised
  • Classification - Outputs are categorical and finite (ex: Yes/No)
  1. Unsupervised (Descriptive) Learning
  2. Reinforcement Learning

What are unknown variables?

They are variables that can’t be predicted. They are also treated as random variables.

  • Ex: Predicting tomorrow’s temperature.

IMPORTANT: Random variables have probability distributions (probabilities of one of the outcomes (like rolling a fair die) ).

  • Random variables have uncertain outcomes (getting a 1 from rolling a fair die)

What is the most common form of ML?

Supervised learning is the most common form of ML.

Supervised Learning

Supervised learning makes the assumption that there is a function that perfectly maps each input to its output. Its task is to approximate that function.

  • Supervised learning models work w/ input-output pairs.
    1. x (bold lowercase): is the input vector
    2. y (bold lowercase): is scalar (unless output is multi-dimensional).

NOTE: multi-dimensional means it contains two outputs.

The functions below are supervised learning models Actual function that we assume exists: is represented using the function f(${x_i}$) = $ {y_i} $ where:

  1. $ {x_i} $ is the ith data instance’s input
  2. $ {y_i} $ data instance’s output

The model’s predicted output: is represented using the function $ \hat{y_i} $ = f*(${x_i}$) where:

  1. $ \hat{y_i} $ is the predicted output/model’s prediction

NOTE: if the model matches, we say that it was accurately predicted, else we say that it made an error.

Dataset Example (NOTE: IMPORTANT TO UNDERSTAND)

  • Each row represents a data instance (ex: certain flower)
  • Column represents the characteristics
  1. N = # of data instances/examples in a dataset (training data)
  2. D = # of input features (D = 5 since ${x_1} - {x_5} $)

table here

Ex:

  1. $ {x_{2,5}} $ = 5th input feature of the 2nd data instance
  2. $ {y_{10}} $ = output of the 10th data instance

Dimensions is N * (D + 1) so… 1,000 * (5 + 1) = 1,000 * 6 = 6000

Supervised Learning Continued…

  • Features are also known as independent variables
  • Outputs are also known as dependent variables