What does feature engineering involve?
Translating records into numbers that carry the signal. A machine learning model sees columns, not customers, so someone decides which columns exist. A feature is a hypothesis about what predicts the outcome, written as code. The common moves, which feed classification and regression models on structured data:
| Technique | What it does | Business example |
|---|---|---|
| Encoding | Turns categories into numbers | Country becomes one column per country |
| Scaling | Puts numbers on a comparable range | Revenue and order count on similar scales |
| Aggregation | Rolls many rows into one per entity | Orders in the last 90 days per customer |
| Time features | Pulls patterns out of dates | Day of week, days since last purchase |
| Missing-value flags | Records that a value was absent | No phone number given at signup |
What is data leakage in feature engineering?
A feature that would not exist at prediction time. The scikit-learn documentation describes leakage as using information “that would not be available at prediction time” when building the model, which produces “overly optimistic performance estimates” (scikit-learn 1.9.1, Common pitfalls, read 5 November 2026). Classic cases: a “refund issued” field in a churn model, or scaling computed over the whole dataset before it is split. The same page says to split train and test data first, “particularly before any preprocessing steps”. Leakage looks like a great model until production, where it fails the way overfitting does, and only honest model validation on a time-ordered holdout catches it.
Why does feature engineering still matter for business data?
Because most business prediction runs on tables, and on tables the older methods still win. A 2022 benchmark found that “tree-based models remain state-of-the-art on medium-sized data”, meaning about 10,000 samples, ahead of deep learning (Grinsztajn, Oyallon and Varoquaux, arXiv:2207.08815, read 5 November 2026). Those models learn only from the columns they are given. Domain knowledge enters the model through its features: a credit analyst knows a recent change in payment behavior can say more than the balance. The hours go into the data, not the algorithm, so plan a predictive analytics project around data preparation rather than model selection.
How did deep learning and LLMs change feature engineering?
They moved it rather than ended it. Deep learning learns its own features from raw images, audio and text, so nobody hand-codes edge detectors anymore. Language models went further: an embedding turns a support ticket or a product description into a vector a tabular model can take as input, and an LLM can read unstructured data such as contracts and emails and extract fields like notice period or complaint category into new columns. The leakage rules still apply to features an LLM produces: a field extracted from a document written after the outcome leaks just as badly as a hand-built one.