Regression concepts simplified

Regression modelling technique is widely used in analytics and perhaps easiest to understand. In this post I am sharing my findings about the concept in simple words.

What is Simple Linear Regression?

A Simple Linear Regression allows you to determine functional dependency between two sets of numbers. For example, we can use regression to determine the relation between ice cream sales and average temperature.

Since we are talking about functional dependency between two sets of variables, we need an independent variable and one dependent variable. In the example above, if change in temperature leads to change in ice cream sales then, temperature is independent variable and sales is dependent variable.

Dependent variables is also called as criterion, response variable or label. It is denoted by Y.

The independent variable is also referred as covariates, predictor or features. It is denoted by X.

Statistical Modeling vs Machine Learning

I have often used the terms Statistical modeling techniques and Machine learning techniques interchangeably but was not sure about the similarities and differences. So I went through few resources and sharing my findings here.


Lets start with basic definition,

A statistical model is a formalization of relationships between variables in the form of mathematical equations.

Machine learning is a subfield of computer science and artificial intelligence which deals with building systems that can learn from data, instead of explicitly programmed instructions.


Lets explore what books and courses say in their first chapter/lecture about both fields.

From book “An introduction to statistical learning” by Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani

Knowledge Discovery and Machine Learning

Knowledge Discovery or Data mining cycle: Following steps constitutes the entire cycle and used to solve a data problems.

• Data cleansing: This is used to remove noise and inconsistent data.
• Data integration: In this step we can combine data from multiple sources.
• Data selection: We might be interested in a specific subset of the data to solve the problem.
• Data transformation: In this step, we would like to format and convert data set in to form suitable for mining. 
• Data mining: In this step we would like to use specific algorithms on formatted data set to get some patterns/ expected output.

Deep Learning: What is that?

What is it?

"Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence." [1]

What does that mean?

"Teaching machines to think has been a dream/nightmare of scientists for a long time. Rather than teaching a machine explicitly, Deep Learning uses simpler, core ideas and then builds upon them — much as a baby learns sounds, then words, then sentences." [2] 

So just another technology?