4. Data Wrangling#
By now, we have become familiar with our dataset and identified its key shortcomings. The next step is to address these issues and prepare the data so it can be used effectively for modeling or for drawing meaningful insights.
This stage is commonly referred to as data wrangling, and it can be broadly divided into two main categories:
Data Cleaning In this step, we focus on improving the quality and consistency of the dataset by:
Handling missing or null values
Detecting and managing outliers and noise
Correcting inconsistent or invalid entries
Feature Engineering Here, we transform and create features to make the dataset more suitable for analysis or modeling:
Encoding categorical variables (e.g., one-hot encoding)
Scaling or normalizing numerical features
Creating new derived features based on existing ones
Throughout the data wrangling process, we may circle back to Data Exploration techniques to validate our transformations and ensure we are improving the dataset without introducing bias or errors.
By the end of this stage, our dataset should be clean, consistent, and enriched with meaningful features, ready for analysis or predictive modeling.