Building a feature store with Kedro and Feast

At Every Cure we apply machine learning to identify drug repurposing candidates; drugs that may treat diseases they weren’t originally developed for. To this end, we engineer lots of features for drugs, diseases, and novel pairs we ask our medical reviewers to look at. This is where Kedro comes in, it empowers us to transfer a wide variety of biomedical data sources into integrated datasets with features that are used to power our workflows. We’ve struggled for a while to reconcile the batch/analytical workloads our Kedro pipelines produce with the real-time needs of our application and agentic workflows.

This post is about how we closed that gap by adopting Feast.

What we had before, and why it wasn't enough

We were running two separate systems, one for batch workloads and another for transactional feature access. However, neither could serve both use-cases.

For batch workloads, we had a datasets repository that versioned feature files in git: lightweight, auditable, and Kedro-friendly, but purely file-based, with no entity-level view and no transactional access. 

For interactive access, an "evidence store" inside our application loaded a handful of features into Postgres, allowing reviewers to filter and sort. It served real-time queries well, but lacked analytical capabilities. State was hand-loaded, hard to recover, and the store only understood a fixed set of entities.

This resulted in two half-answers to the same question. If a reviewer wanted every feature we had for a given drug-disease pair, there was no single place to look, and every new entity or feature type meant plumbing changes in two places.

Why Feast

What drew us to Feast compared to other feature stores is its open-source nature, the fact it does not lock you in into a cloud platform, and its relatively low operational complexity. Features are registered once, in one place, and can then be consumed by any Kedro pipeline. This matters when you have more than one project and more than one team producing and consuming features. More broadly, it promotes feature management from individual, opaque datasets in the catalog to a shared, well-defined registry with explicit schemas and semantics. The registry has two components, an offline store for training and analytics, and an online store for low-latency serving. Feast has no server component: it delegates straight into BigQuery for the offline (analytical) store and our existing Postgres for the online (serving) store.

That offline/online split is the part that unlocked the most for us. The offline store holds everything; the online store holds a curated subset, materialized on a schedule, that we can query interactively. Instead of constantly re-stitching the underlying base files to answer a question about one pair, we ask the online layer and get the latest values back in a single call.

The rest of this post walks through the feature lifecycle steps; register, write, materialize and read, and highlights the Kedro glue code we wrote in between. Together, Kedro and Feast gave us a single feature registry for both batch pipelines and real-time applications, enabling faster review of drug-repurposing candidates and consistent access to engineered features across analytical, application, and agentic workflows.

1. Register entities and features

Entities and features are declared once in a feature_definitions.py. An entity is a dimension of your application domain a feature is attached to:

1drug = Entity(name="drug", join_keys=["drug_kg_node_id"], value_type=ValueType.STRING)
2disease = Entity(name="disease", join_keys=["disease_kg_node_id"], value_type=ValueType.STRING)
3

A source typically maps to the output of one Kedro pipeline:

1python
2drug_batch_source = BigQuerySource(
3    name="drug_online_push_batch_source",
4    table="ec-features-prod.feast.drug_features",
5    timestamp_field="event_timestamp",
6)

And a FeatureView ties features to an entity and source, with typed, documented fields that Feast can validate writes against:

1python
2drug_features_fv = FeatureView(
3    name="drug_feature_view",
4    entities=[drug],
5    schema=[
6        Field(name="name", dtype=String, description="The name of the drug."),
7        Field(name="is_steroid", dtype=Bool, description="True when the drug is a steroid."),
8        # ...
9    ],
10    online=True,  # expose this view to the online store
11    source=drug_batch_source,
12    enable_validation=True,
13)

Feast also ships a UI (feast ui) for inspecting features and lineage.

EveryCure

2. Write features from Kedro

Since Kedro is our framework of choice, we wrote a small custom dataset, FeastDataset, so pipelines can write to the store through the catalog like any other output:

1yaml
2#catalog.yml
3drug_features:
4  type: kedro_utils.datasets.FeastDataset
5  save_args:
6    feature_view_name: drug_feature_view
7    if_exists: append
8  load_args:
9     # as will be shown later, can also point to a service to load bigger feature sets
10    feature_view_name: drug_feature_view
11  metadata:
12    kedro-viz:
13      layer: feature

The dataset infers the schema and table location from the Feast definition and delegates the actual write to Kedro's GBQDataset. That guarantees the data lands exactly where Feast expects it, so materializing to the online store later is friction-free. Data is written to the underlying table incrementally. This has the advantage that pipelines can materialize feature values for distinct subsets of the output domain on each execution. This is especially relevant when the target domain is large (as for drug repurposing) and feature computation is expensive, e.g., LLMs are used to derive features.

Reading is where the integration gets interesting. Unlike a typical dataset that retrieves data on load, this one returns a queryable handle, giving the user control over the entities, and feature set, to load for.

1python
2def enrich_drug_features(drug_list: pd.Dataframe, drug_feature_view):
3    """
4    Query the offline feature store for drug features.
5    Args:
6        drug_list: list of drug identifiers to get features for
7        drug_feature_view: feature view to use
8    """
9    return drug_feature_view.get_historical_features(drug_list[["drug_kg_node_id"]])
10
11def create_pipeline(**kwargs) -> Pipeline:
12    return Pipeline(
13        [
14            Node(
15                func=enrich_drug_features,
16                inputs=["drug_list", "drug_features"],
17                outputs="drug_features_read",
18                name="enrich_drug_features",
19            )
20        ]
21    )

This enriches the input drug dataframe with every feature registered on the view. By default it retrieves the latest values; for reproducible experiments you pass a timestamp to get_historical_features, since every feature is timestamp-enriched.

3. Materialize the features you want served

Feast exposes the features available online through a FeatureService, a named bundle of feature views:

1python
2application_feature_service = FeatureService(
3    name="application_feature_service",
4    features=[drug_features_fv], # add other feature views here to expose more fields
5    tags={"scope": "application"},
6)

This became our single control point: adding a feature view to the service exposes it to reviewers in our internal application, with no other code changes. Two CLI commands keep things in sync:

1bash
2feast apply                  # register new entities / feature views
3feast materialize-incremental "$(date -u +%Y-%m-%dT%H:%M:%S)"   # copy offline → online, incrementally

materialize-incremental`tracks the last materialized timestamp per view, so each run only moves what changed. A nice side effect: the online store is now ephemeral, i.e., a single materialize command rebuilds it from scratch, which made our disaster-recovery story trivial.

4. Query features online

This is the payoff. Instead of stitching base files together, the app asks the online layer via get_online_features and gets the latest values back for the requested entities in a single request:

1python
2if (store := settings.feature_store) is not None:
3    feature_service = store.get_feature_service("application_feature_service")
4    feature_vector = await asyncio.to_thread(
5        store.get_online_features,
6        features=feature_service,
7        full_feature_names=True,
8        entity_rows=[{
9            "drug_kg_node_id": drug_kg_node_id,
10            "disease_kg_node_id": disease_kg_node_id,
11        }],
12    )
13    raw = feature_vector.to_dict(include_event_timestamps=True)
14    # ... map raw values back onto the response, per feature view

One request, fetching evolving drug, disease, and pair feature sets together. No manual joins, no file wrangling.

Where we landed

Putting Feast behind Kedro gave us one definition of a feature, written from our existing pipelines, served both analytically (BigQuery) and interactively (Postgres). The fact that the store lives outside any single Kedro project is what makes it reusable across teams and the online layer is what finally let us stop re-deriving the same features from base files on every question.