Online Feature Computation

"In training I knew everything about the user. In production I had a hundred milliseconds and a key. Nobody warned me they were supposed to agree."

A Feature That Drifted Between Offline and Online

The previous section put a streaming engine to work on the event stream: windows, watermarks, and continuous aggregation in Section 9.6 turned a never-ending log into a table of running results. This section asks what those results are for. The answer, for most production AI, is features: the inputs a model consumes to make a decision. When a fraud model scores a card swipe, it does not see the raw event alone; it sees engineered signals such as "how much has this card spent in the last five minutes," and that signal must be fresh, correct, and identical to the one the model learned from. Producing such signals continuously, over a distributed stream, and serving them under a latency budget is the discipline of online feature computation.

1. From Events to Features in Real Time Beginner

A feature is a function of data that a model takes as input. In batch machine learning that function runs over a static table; the distributed-preprocessing machinery of Section 8.7 computes it once per training run and stores the result. An online feature is the same idea with time added: the function runs over an event stream and its value changes as new events arrive. The most common form is a windowed aggregate. Given a stream of events $e_1, e_2, \dots$ each carrying a timestamp $t_i$, a sliding-window feature for a key (say a user) at query time $\tau$ is

where $W$ is the window length and $\mathrm{agg}$ is a sum, count, mean, max, or similar. The defining property is recency: as $\tau$ advances, old events fall out of the window and new ones enter, so the feature is never computed once and frozen; it is maintained. The engineering goal is to maintain it incrementally, in $O(1)$ amortized work per event, rather than rescanning the window on every query, because at production volumes a rescan per event is hopeless.

Two distinct moments matter, and conflating them is where skew is born. There is the write moment, when an event arrives and the streaming engine updates the aggregate, and the read moment, when a model asks for the current feature value to make a prediction. The write side runs continuously across the cluster; the read side must be a fast key lookup, because the model is on a latency budget. Separating these two responsibilities is exactly what a feature store does, and it is the subject of the next section.

2. The Feature Store and Its Two Halves Beginner

A feature store is the system that computes feature values, stores them, and serves them to both training and inference. Its defining design choice is a split into two physical stores fed, ideally, by one logical definition. The offline store holds the full history of feature values, partitioned and columnar, optimized for the large scans that training needs; it lives on the distributed storage and batch machinery of Chapter 8. The online store holds only the latest value per key in a low-latency key-value system (Redis, DynamoDB, Cassandra) so that an inference request can fetch every feature it needs with a handful of point lookups inside its millisecond budget. Figure 9.7.1 shows how the same event stream feeds both halves.

The split is what makes both workloads fast: training gets cheap sequential scans over history, serving gets cheap random lookups of the present. The danger is also born here. Two physical pipelines computing "the same" feature, one in a batch engine over the offline store and one in a streaming engine over the live stream, will drift apart unless they are forced to share a definition. That forced sharing is the central requirement of the whole section.

3. Train/Serve Consistency, or You Are Training a Different Model Intermediate

Training-serving skew is the situation where the feature value a model sees in production differs from the value it would have seen for the same input during training. It is insidious because nothing crashes: the model returns predictions, the dashboards stay green, and accuracy quietly collapses because the model is being asked about points it never learned. The skew usually comes from one of a few sources: the offline feature is computed by different code than the online feature; the online feature includes events the offline one excluded (or vice versa) because of a window-boundary or timezone mismatch; or the offline feature accidentally peeks at the future. The defense is a single rule, stated as a key insight.

The second subtle requirement is temporal honesty in the offline path. When you build a training set, each training row pairs a label observed at time $\tau$ with the feature values as they were at $\tau$, not as they are now. Joining a label to a feature value computed from events that occurred after $\tau$ leaks the future into the past, and the model learns a relationship that cannot exist at serving time. This is the streaming face of the leakage and correctness discipline introduced for batch pipelines in Section 8.8; here it has a name of its own.

4. Point-in-Time Correct Joins Intermediate

The mechanism that enforces temporal honesty is the point-in-time correct join (also called an as-of join). Given a set of label events, each a pair $(\text{key}, \tau)$, and an offline store of timestamped feature values, the join attaches to each label the feature value whose timestamp is the latest one not exceeding $\tau$:

which is the very same window definition from Section 1 evaluated at the label time $\tau$ rather than at "now." The inequality $t_i \le \tau$ is the whole game: it admits only events the system could actually have observed by $\tau$, and it excludes anything later. A naive join on key alone, ignoring time, would attach the current feature value to every historical label and reintroduce exactly the leakage the as-of join exists to prevent. Production feature stores implement this join as a first-class operation over the offline store so that a correct training set is the default rather than a thing you must remember to do by hand.

5. Building It From Scratch Intermediate

The code below makes all of this concrete with no framework at all. It maintains a five-minute sliding-window sum of spend per user incrementally as a stream of card swipes arrives (the streaming path), writes the latest value per user into a dictionary that stands in for the online key-value store, and serves a toy flag-if-over-100 model by a single key lookup. It then recomputes the same feature the offline way, directly from the full event log, evaluated point-in-time as of each user's last event, and checks that the two agree. The match is the property that the rest of this section has been arguing for.

import time
from collections import deque, defaultdict

# --- 1. A stream of card-swipe events: (user, unix_ts_seconds, amount) ---
events = [
    ("u_alice", 1000, 12.0), ("u_bob", 1001, 5.0),  ("u_alice", 1100, 40.0),
    ("u_alice", 1180, 7.0),  ("u_bob", 1200, 9.0),  ("u_alice", 1250, 30.0),
    ("u_bob", 1260, 4.0),    ("u_alice", 1330, 22.0),("u_alice", 1400, 11.0),
    ("u_bob", 1500, 6.0),    ("u_alice", 1520, 50.0),
]
WINDOW = 300  # a 5-minute (300-second) sliding window

# --- 2. STREAMING path: maintain a per-user sliding window incrementally ---
# This is what runs in production: one pass, O(1) amortized per event, and after
# each event the freshest feature value is written to the online store.
online_store = {}                       # user -> latest feature value (key-value)
windows = defaultdict(deque)            # user -> deque of (ts, amount) still in window
running_sum = defaultdict(float)        # user -> sum of amounts in the window

def streaming_feature(user, ts, amount):
    dq = windows[user]
    dq.append((ts, amount)); running_sum[user] += amount
    while dq and dq[0][0] <= ts - WINDOW:        # evict events that fell out
        old_ts, old_amt = dq.popleft(); running_sum[user] -= old_amt
    return running_sum[user]

for user, ts, amount in events:
    feat = streaming_feature(user, ts, amount)
    online_store[user] = feat            # serve-time read will hit this dict

# --- 3. SERVE: an inference call reads the feature by key, no recomputation ---
def serve(user):
    x = online_store.get(user, 0.0)      # single low-latency key lookup
    return 1 if x > 100.0 else 0         # toy "flag if >$100 spent in 5 min" model

print("online store (amount spent, last 5 min, as of latest event):")
for u in sorted(online_store):
    print(f"  {u:8s} feature={online_store[u]:6.1f}  ->  model says {serve(u)}")

# --- 4. OFFLINE path: same feature DEFINITION, recomputed from the full log ---
# This is the training-time computation. If it disagrees with the streaming value
# for the same (user, as-of time), the model trains and serves on skewed inputs.
def offline_feature(user, as_of_ts):
    return sum(a for (u, t, a) in events
               if u == user and as_of_ts - WINDOW < t <= as_of_ts)

# point-in-time correct check: recompute each user's feature as of THEIR last event
last_ts = {u: max(t for (uu, t, a) in events if uu == u) for u in online_store}
print("\ntrain / serve consistency (online vs offline recomputation):")
ok = True
for u in sorted(online_store):
    online_v = online_store[u]
    offline_v = offline_feature(u, last_ts[u])
    match = abs(online_v - offline_v) < 1e-9
    ok = ok and match
    print(f"  {u:8s} online={online_v:6.1f}  offline={offline_v:6.1f}  match={match}")
print("\nno training-serving skew:" , ok)

online store (amount spent, last 5 min, as of latest event):
  u_alice  feature= 113.0  ->  model says 1
  u_bob    feature=  10.0  ->  model says 0

train / serve consistency (online vs offline recomputation):
  u_alice  online= 113.0  offline= 113.0  match=True
  u_bob    online=  10.0  offline=  10.0  match=True

no training-serving skew: True

The win is that the offline number was produced by the same window definition as the online number, evaluated at the same instant per user, so the model trains and serves on one distribution. The deque in the streaming path is what keeps the maintenance cheap: each event is pushed once and popped at most once, giving the $O(1)$ amortized cost that Section 1 demanded, while the offline recomputation rescans the log because at training time the full history is available and a one-pass scan is acceptable. Two implementations, one definition, identical values.

# pip install feast
from feast import FeatureStore

store = FeatureStore(repo_path=".")

# SERVING: one call fetches the latest values from the online key-value store
features = store.get_online_features(
    features=["user_spend:amount_5min"],
    entity_rows=[{"user": "u_alice"}],
).to_dict()                              # -> {'amount_5min': [113.0], ...}

# TRAINING: the SAME definition, joined point-in-time to labels (no future leak)
training_df = store.get_historical_features(
    entity_df=labels_df,                 # columns: user, event_timestamp, label
    features=["user_spend:amount_5min"],
).to_df()

6. From One Feature to a Real-Time Personalization Engine Advanced

A single windowed sum is the atom; a recommendation system is the molecule. Real-time personalization reads dozens of online features per request (recent clicks, dwell time, category affinities computed over sliding windows) and combines them with embeddings to rank items in milliseconds. The same online/offline split governs it: the ranking model trains on point-in-time correct histories from the offline store and serves on the latest values from the online store, and any skew between them degrades recommendations exactly as it degrades fraud scores. The embeddings such systems consume are themselves a distributed feature, sharded across machines as built in Chapter 11; we develop the full real-time personalization stack, online features and sharded embeddings together, as a case study in Chapter 38.

With features computed and served consistently, the remaining question is how the model that consumes them is itself deployed across machines under the same latency budget. That is where this chapter turns next: a feature lookup is one stage of a real-time inference pipeline, and the pipeline as a whole, feature fetch, model call, and post-processing, must be distributed to meet its deadline. Section 9.8 builds that distributed real-time inference pipeline.