Managing a feature store in Feast involves modeling entities and features, registering them in a feature repository, materializing data into offline and online stores, and operating the store with CI/CD, monitoring, and governance.
Feast (Feature Store) is an open-source, cloud-agnostic feature store that bridges data engineering and machine-learning operations by providing a single source of truth for features. It manages feature definitions, histories, and online serving, allowing data scientists to focus on modeling while ensuring feature consistency between training and inference.
Feature drift, training–serving skew, and slow ML release cycles often stem from ad-hoc feature pipelines. A well-managed Feast deployment gives you:
driver_id
).feast init my_feature_repo
This scaffolds a
cd my_feature_repofeature_store.yaml
and example Python files.
# driver_features.py
from datetime import timedelta
from feast import Entity, Field, FeatureView, FileSource
from feast.types import Float32
# 1️⃣ Entity
driver = Entity(name="driver_id", value_type="INT64", description="Driver identifier")
# 2️⃣ Data source
driver_stats_source = FileSource(
path="gs://my-bucket/driver_stats.parquet",
timestamp_field="event_timestamp",
created_timestamp_column="created",
)
# 3️⃣ Feature view
driver_stats_fv = FeatureView(
name="driver_stats",
entities=["driver_id"],
ttl=timedelta(days=1),
schema=[
Field(name="conv_rate", dtype=Float32),
Field(name="acc_rate", dtype=Float32),
Field(name="avg_daily_trips", dtype=Float32),
],
online=True,
batch_source=driver_stats_source,
)
feast apply
The command diff-checks the registry and applies changes safely (useful for CI).
Backfill to the offline & online stores:
# backfill last 90 days into offline store only
date_to=$(date +%F)
date_from=$(date -v-90d +%F)
feast materialize-incremental $date_to --start $date_from
For near-real-time data, schedule an incremental job (Airflow, Prefect) every few minutes.
from feast import FeatureStore
store = FeatureStore(".")
training_df = store.get_historical_features(
entity_df=my_events_df,
features=["driver_stats:conv_rate", "driver_stats:acc_rate"]
).to_df()
feature_vector = store.get_online_features(
Latency is typically <10 ms when using Redis or DynamoDB.
features=[
"driver_stats:conv_rate",
"driver_stats:acc_rate",
],
entity_rows=[{"driver_id": 1001}],
).to_dict()
driver_stats_v2
) or Git tags for reproducibility.ttl
to protect against stale online features.feast apply
and materialize
in CI pipelines with dry-run checks.store.get_historical_features(..., full_feature_names=True)
to avoid name collisions.store.validate()
) in your CI.event_timestamp
column and letting Feast handle the join.materialize
jobs every minute for slow-moving data wastes compute. Right-size the cadence to the freshness SLA.ttl
lets stale cache linger, causing skew. Set an appropriate TTL (e.g., hours) matching data velocity.Feast’s offline store is usually a SQL-based warehouse (Snowflake, Redshift, Postgres). Galaxy’s modern SQL editor can help data engineers:
FileSource
or BigQuerySource
.A mature Feast deployment uses Git for source control, CI for registry changes, a scheduler for materialization, and monitoring for freshness. By following the workflow above, you guarantee that every feature your model sees in production was computed exactly the same way during training—eliminating one of the biggest causes of ML model degradation.
Consistently reproducing feature values at both training and inference time is essential for reliable ML. A mis-managed feature pipeline introduces data leakage, stale features, and training-serving skew that silently erodes model accuracy and business trust. Feast provides the abstraction to solve these problems, but only if you manage entities, feature views, and materialization correctly with CI/CD, monitoring, and governance.
Use feast materialize
with a start and end date to load historical feature values into the offline store. For example: feast materialize 2022-01-01 2022-12-31
. For continuous updates, schedule materialize-incremental
.
Yes. Define a StreamSource
(Kafka, Kinesis) and attach a transformation (Flink, Spark). Feast will write the output to your online store for low-latency reads.
Export metrics (e.g., last materialization timestamp, row counts) from the online store and visualize them in Prometheus/Grafana. Alert if freshness exceeds your SLA.
Galaxy focuses on SQL editing. While it does not manage Feast registries, you can explore and validate offline store tables in Galaxy, then embed the approved SQL into Feast pipelines.