SUMMER TRAINING Free Tutorials  Go To Your University  Placement Preparation 
Project Based Best Summer Training Courses in Jaipur
Investing in yourself is easy. Udemy online courses now from INR 455
0 like 0 dislike
168 views
in Artificial Intelligence(AI) & Machine Learning by Goeduhub's Expert (3.1k points)
edited by
This article deals with the basic machine learning concepts that is overfitting, underfitting, bias and variance as well as how these concept vary with model complexity.

1 Answer

0 like 0 dislike
by Goeduhub's Expert (3.1k points)
edited by
 
Best answer

To understand the concept of  regularization in machine learning first we need to understand underfitting, overfitting, bias and variance.

To understand these concept consider the below diagram:

underfitting-overfitting

We know that in regression problem we try to find a best fit line for all distributed data points. 

In the above diagram the first diagram (fig1) shows that our model is linear but the data distribution is polynomial. As you can see from the diagram, if we calculate prediction error /error for this diagram (fig1) we will get high prediction error/ error.

This the situation of underfitting , where we are getting high prediction error /error for distributed data. The prediction error /error will be high for both training data as well as test data (random data). 

And the reason for prediction error/ error is not best fit line for distributed data points , to correct this problem, we need a polynomial regression model to get a best fit line for distributed data as our data distribution is polynomial.

In the fig2, we try a polynomial regression model to get a best fit line for the distributed data points. Let's suppose the polynomial regression we used here is polynomial of degree 2 (p=2) and get a best fit line as shown in diagram (fig2). 

For this diagram (fig2) we will get low prediction error/ error as our data mostly lies on best fit line. But we can improve this model if we increase the degree of polynomial of regression model.

In fig3 we increased the degree of polynomial to get a very best fit line for our distributed data and prediction error /error for this regression model will be very less as our data points lies on best fit line. 

But the problem with this regression model is the prediction error/ error is less for training data but not for test data. Meaning in this type of modeling the model perform very well when its come to training data but perform very bad with unseen data or test data or random data. This is called overfitting and model is called overfitted model.

To deal with this type of problems we have concept of regularization in machine learning. 

Bias  and Variance : Bias and variance both are forms of prediction error /error in machine learning. 

Bias: Difference between predicted values and actual values. Model with high bias pays very little attention to the training data and oversimplifies the model. It always leads to high prediction error /error on training and test data. 

Variance: Variance is also difference between predicted values and actual values. But model with variance pays more attention to training data and doesn't generalize on the data which it hasn't seen such as test data or random data.

Underfitting is a case of high variance and high bias.

Overfitting is a case of low bias and high variance. 

Appropriate fitting is a case of low bias and and low variance which is a good regression model.

bias-variance

Bias and Variance Trade off : 

Bias and variance trade off can be defined in terms of complexity of models. For example Low variance and high bias algorithms are less complex.

Examples : Linear regression , Naive Bayes etc..

And High variance and low bias algorithms are more complex and tend to flexibility. 

Examples: Non-linear regression , decision tree , Nearest Neighbor etc...

To build a good model we need a balance between bias and variance means models shouldn't underfit and overfit and total error should be minimum. 

bias-variance

You can see in the diagram as model complexity increases , model tends to overfitting and with low complexity model tend to underfitting. Hence bias and variance also change with model complexity. The optimal model is shown with red circle.

Concept of Regularization in machine learning 

Our Mentors(For AI-ML)


Sharda Godara Chaudhary

Mrs. Sharda Godara Chaudhary

An alumna of MNIT-Jaipur and ACCENTURE, Pune

NISHA (IIT BHU)

Ms. Nisha

An alumna of IIT-BHU

Related questions

 Goeduhub:

About Us | Contact Us || Terms & Conditions | Privacy Policy || Youtube Channel || Telegram Channel © goeduhub.com Social::   |  | 
...