Feature Selection in Machine Learning with Python

Feature Selection in Machine Learning with Python

Over 20 methods to select the most predictive features and build simpler, faster, and more reliable machine learning models.

About the Book

Feature selection is the process of selecting a subset of features from the total variables in a data set to train machine learning algorithms. Feature selection is an important aspect of data mining and predictive modelling.

Feature selection is key for developing simpler, faster, and highly performant machine learning models and can help to avoid overfitting. The aim of any feature selection algorithm is to create classifiers or regression models that run faster and whose outputs are easier to understand by their users.

In this book, you will find the most widely used feature selection methods to select the best subsets of predictor variables from your data. You will learn about filter, wrapper, and embedded methods for feature selection. Then, you will discover methods designed by computer science professionals or used in data science competitions that are faster or more scalable.

First, we will discuss the use of statistical and univariate algorithms in the context of artificial intelligence. Next, we will cover methods that select features through optimization of the model performance. We will move on to feature selection algorithms that are baked into the machine learning techniques. And finally, we will discuss additional methods designed by data scientists specifically for applied predictive modeling.

In this book, you will find out how to:

  • Remove useless and redundant features by examining variability and correlation.
  • Choose features based on statistical tests such as ANOVA, chi-square, and mutual information.
  • Select features by using Lasso regularization or decision tree based feature importance, which are embedded in the machine learning modeling process.
  • Select features by recursive feature elimination, addition, or value permutation.

Each chapter fleshes out various methods for feature selection that share common characteristics. First, you will learn the fundamentals of the feature selection method, and next you will find a Python implementation.

The book comes with an accompanying Github repository with the full source code that you can download, modify, and use in your own data science projects and case studies.

Feature selection methods differ from dimensionality reduction methods in that feature selection techniques do not alter the original representation of the variables, but merely select a reduced number of features from the training data that produce performant machine learning models.

Using the Python libraries Scikit-learn, MLXtend, and Feature-engine, you’ll learn how to select the best numerical and categorical features for regression and classification models in just a few lines of code. You will also learn how to make feature selection part of your machine learning workflow.

  • Share this book

  • Categories

    • Python
    • Artificial Intelligence
    • Machine Learning
    • Programming Cookbooks
  • Feedback

    Email the Author(s)

About the Author

Soledad Galli
Soledad Galli, PhD

Soledad Galli is a data scientist, instructor, and software developer with more than 10 years of experience in world-class academic institutions and renowned businesses. She has developed and put into production machine learning models to assess insurance claims, credit risk, and prevent fraud.


Sole teaches online courses on machine learning, which have enrolled 40,000+ students worldwide and consistently receive good student reviews. She is also the developer and maintainer of the open-source Python library Feature-engine, which is currently downloaded about 100k+ times per month.


Sole received a Data Science Leaders' award in 2018 and was recognized as one of LinkedIn's voices in data science and analytics in 2019. She is passionate about sharing her machine learning knowledge. She gave talks at data science conferences and wrote several publications about data science and machine learning, including one on the misuse of data and artificial intelligence.

The Leanpub 60-day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms

Do Well. Do Good.

Authors have earned$11,845,077writing, publishing and selling on Leanpub, earning 80% royalties while saving up to 25 million pounds of CO2 and up to 46,000 trees.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub