Feature store is a fundamental component of the ML stack and of any robust data infrastructure because it enables efficient feature engineering and management. It also allows for simple re-use of features, feature standardization across the company, and feature consistency between offline and online models. A centralized, scalable feature store allows organizations to innovate faster and drive ML processes at scale..

From a business perspective, feature stores offer two major benefits:

1. Better ROI from feature engineering through reduction of cost per model, which facilitates collaboration, sharing, and re-using of features
2. Faster time to market for new models, driven by increased productivity of ML Engineers. This enables organizations to decouple storage implementation and features serving API from ML engineers, freeing them to work on models, not on latency problems, for more efficient online serving.

Some use cases do not require a centralized, scalable feature store. Feature stores work for projects where models need a stateful, ever-changing representation of the system. Examples of such use cases include demand forecasting, personalization and recommendation engines, dynamic pricing optimization, supply chain optimization, logistics and transportation optimization, fraud detection, and predictive maintenance.

A standardized feature store has certain key concepts that revolve around it. Those concepts are:

1. Online Feature Store. Online applications look for a feature vector that is sent to an ML model for predictions.
2. ML Specific Metadata. Enables the discovery and re-use of features.
3. ML Specific API and SDK. High-level operations for fetching training feature sets and online access.
4. Materialized Versioned Datasets. Maintains versions of feature sets used to train ML models.

All of the concepts are represented in the above image. Analytical data is taken from a data lake and pushed through the feature engineering pipeline, where output is stored in the feature store. From there, ML engineers can discover the features, use them for training new models, and then re-use the features for serving.

Why Feature stores:

Let’s begin with an example. Suppose I am working in the data science team of a hotel booking platform company. I am tasked with building a hotel capacity prediction system. I have a data table of when customers check-in, and a table of when customers check out. I want to predict if a given hotel is going to have spare capacity. The first step will be for me to try and define useful ways of looking at the data. In our example, comparing check-in date and check out date per customer will yield whether a given room in a hotel is vacant or not. I can aggregate that at the hotel level and know how many rooms are available/free at a given time. From that feature, I can try and predict if there will be free rooms given capacity.

To get there, I have explored data, have written code to extract a potentially useful feature — including code to query, clean, filter, and aggregate data from the hotel booking database. Only after completing that, did I start working on my model.

Turns out that the feature (the number of free rooms in a hotel) is probably useful to many other folks in other data science teams across the company: the one that predicts high-level financial outcomes, the one that builds a product to help hotels forecast cleaning requirements, etc. But each of those teams is re-implementing the same feature computation pipeline so that they can use the same feature in their algorithms.

Worse, to ship the model to production, the data science team typically hands-off the model to the data engineering team, which will make it production-ready (executable at scale, faster, etc.). That team will typically rewrite the feature computation code (e.g., add more efficient ways to compare tables in real-time as new bookings are added)

At the end of the day, you may end up with many ways a feature such as the number of free rooms in a hotel is defined.

“As data scientists, you and I will choose different ways to look at things, say round things up differently. It can have a big impact. And every time we have to recompile every feature when we go to prod.

There are several issues here:

• there is no single point of truth, which means models might behave differently in prod vs. training
• a lot of rework is induced by several teams rewriting stuff, either by lack of a centralized way to define features or by lack of awareness that a feature is already defined somewhere else
• teams end up copying data a lot, especially “in a big org with a lot of legacy, you have to query from many places. People end up copying a lot of data to bring it all together”, Davit Bzhalava added.

That’s where feature stores come in.

A feature store is a central place where data features are defined, curated, consistently served, and shared. It offers traceability of the data from the source to the model.

Architecturally speaking, “it’s a dual database”.You have a low latency database for getting low latency access to feature data in online applications. And then you have a scale-out database that can store large volumes of feature data for training bigger and better models.”

Beyond that, feature stores redefine the way teams work and sit at the very center of the data science workflow, which is probably why they took a while to hit the market: they are complex products to build.

Feature Store vs Data Warehouse:

Data warehouses democratized access to Enterprise data by centralizing data in a single platform and then empowering business analysts with visual tools, such as Tableau and Power BI. No longer did they need to know what data resides where and how to query that data in that platform. They could derive historical insights into the business using BI tools.

Data scientists, in contrast, build predictive models to derive business insights. The Feature store is the data warehouse for Data Science – it is a central vault for storing documented, curated, and access-controlled features that can be used across many different models. The feature store ingests data from the Enterprise’s many different sources after transforming, aggregating, and validating the data.

Feature pipelines need to be written to ensure that data reliably flows from existing sources and is available in a format ready to be consumed by ML training pipelines and models.

Most Data Scientists currently do not have a feature store. They spend most of their time looking for, cleaning, and featurizing data. Hence, the (very real) cliche that 80% of data science is data wrangling. Data Scientists without a feature store work in an era akin to how business analysts worked before the advent of data warehouses, with low individual and organizational productivity.

• Track and share features between data scientists including a version-control repository
• Process and curate feature values while preventing data leakage
• Ensure parity between training and inference data systems
• Serve features for ML-specific consumption profiles including model training, batch and real-time predictions
• Accelerate ML innovation by reducing the data engineering process from months to days
• Monitor data quality to rapidly identify data drift and pipeline errors
• Empower legal and compliance teams to ensure compliant use of data
• Bridging the gap between data scientists and data & ML engineers
• Lower total cost of ownership through automation and simplification
• Faster Time-To-Market for new model-driven products
• Improved model accuracy: the availability of features will improve model performance
• Improved data quality via data ->feature -> model lineage

In this notebook we will,

1. Load the Iris Flower dataset
2. Do feature engineering on the dataset
3. Save the features to the feature store
4. Read the feature data from the feature store
5. Train a KNN Model using SkLearn

Imports

Load Dataset

Feature Engineering

The dataset is already quite well prepared, the only thing we need to for feature engineering is to convert the variety column to numeric and save a lookup table so that we later on can convert the numeric representation back to the categorical representation.

Save Features to the Feature Store

We can save two feature groups (hive tables), one called iris_features that contains the iris features and the corresponding numeric label, and another feature group called iris_labels_lookup for converting the numeric iris label back to categorical.

Note: To be able to run the feature store code, you first have to enable the Feature Store Service in your project. To do this, go to the “Settings” tab in your project, select the feature store service and click “Save”.

Read the Iris Training Dataset from the Feature Store

Train a KNN Model using the Feature Data