Design a Fraud Detection System: ML Interview Answer Framework

The direct answer is: build a layered decision system that combines hard rules, a low-latency risk model, and manual review, then optimize thresholds against the cost of fraud, customer friction, and review capacity. Train with point-in-time features and delayed outcomes, monitor both system health and decision quality, and keep a path to update defenses as behavior changes.

That answer is a starting map, not a complete design. A strong interview performance makes the decision, data, and feedback loops precise before discussing model architecture.

1. Scope the Fraud Decision

Assume the prompt is: "Design fraud detection for card-not-present payments." Clarify what the system controls. At authorization time, can it approve, decline, request additional verification, or send a transaction to review? Is the goal to detect stolen payment instruments, account takeover, merchant abuse, or several categories?

Ask about constraints that change the architecture:

• Online traffic and peak load
• End-to-end decision latency
• Regions, currencies, and payment methods
• Available history for new and returning users
• Timing and reliability of fraud labels
• Manual review capacity
• Requirements for reason codes, audit, and appeals

State the costs explicitly. A false negative can create fraud loss and downstream operational work. A false positive can block a legitimate customer, lose a sale, and damage trust. A review decision delays the customer and consumes analyst capacity. The right operating point depends on these costs and may vary by segment.

For a general ML design structure, see the ML system design interview guide. Fraud adds adversarial behavior, delayed labels, and unusually important threshold decisions.

2. Define Metrics Before Models

Do not use accuracy as the headline metric. Fraud is typically an imbalanced problem, and a high accuracy can coexist with an unusable detector.

Discuss metrics at three levels:

• Model ranking: precision-recall curves, recall at a chosen precision, or ranking quality within review capacity
• Decision policy: false-positive rate, fraud caught, legitimate value approved, verification rate, and review yield
• System: p95 or p99 latency, availability, feature freshness, and decision errors

Where values differ substantially, amount-weighted measures can complement transaction counts. A model that catches many low-value attempts while missing fewer high-value attempts may look good by count and poor by loss.

Thresholds should follow a cost matrix or constrained objective, not a default probability of 0.5. You might require extremely high confidence for an automatic decline, use a lower threshold for additional verification, and route the uncertain middle to review when capacity allows.

3. Design the Decision Path

A practical online path is:

1. Receive the payment request and validate required fields.
2. Fetch trusted account, device, merchant, and recent activity features.
3. Apply deterministic safety and policy rules.
4. Score the transaction with the risk model.
5. Combine rule outcomes, score, uncertainty, and operational constraints in a policy engine.
6. Approve, verify, review, or decline with a recorded reason.
7. Emit an immutable decision event for monitoring, training, and audit.

Keep the model and policy conceptually separate. The model estimates risk from available evidence. The policy translates that score into an action based on cost, capacity, and current risk appetite. This separation allows threshold changes without retraining and makes business rules easier to audit.

Why Keep Rules?

Rules handle known invariants, emergency blocks, and patterns for which you have too little training data. They are fast and explainable. Models combine weak signals and generalize beyond exact patterns. Neither should silently override everything else.

Manage rules as versioned configuration with owners, tests, expiration reviews, and measured impact. Otherwise rules accumulate, overlap, and create unexplained customer friction.

4. Build Point-in-Time Features

Useful feature families include:

• Transaction context: amount, currency, merchant type, channel, and time
• Account history: account age, prior successful payments, typical amount range, and recent profile changes
• Velocity: attempts, distinct merchants, cards, devices, or locations over several recent windows
• Device and network: known-device status, coarse network risk signals, and consistency with prior use
• Relational signals: shared devices or payment instruments across accounts, represented carefully at scale
• Merchant behavior: recent authorization and dispute patterns relative to an appropriate baseline

The interview-critical phrase is available at decision time. A training row for a payment at 10:00 must not use a chargeback filed weeks later as a feature, or an aggregate recomputed with future events. Build point-in-time joins and validate offline-online feature parity.

Velocity features need fresh state. A streaming aggregator can update windows while an online feature store serves them at low latency. Define fallback behavior for missing or stale features: use safe defaults, a simpler model, additional verification, or fail according to the product's risk policy. "The feature store will always work" is not a design.

Concrete Example

A returning customer makes a purchase slightly above their usual range from a known device. A simple amount rule might block it. A model can combine device familiarity, account history, merchant context, and recent velocity to estimate lower risk. Conversely, several small attempts across new merchants in a few minutes may be suspicious even though each amount is ordinary.

This example demonstrates why interacting signals matter and why one rule should not represent the entire policy.

5. Create Labels Without Fooling Yourself

Fraud labels may come from chargebacks, customer reports, analyst investigations, or confirmed account takeover. They arrive late, can be disputed, and do not cover every bad event. Legitimate labels are also uncertain until enough time has passed.

Use a label maturity window. Train and evaluate only on examples old enough for the chosen outcome to be reasonably observed. Track the source and confidence of labels rather than collapsing every signal into a supposedly perfect target.

Selection bias is a major concern. Analysts review transactions selected by the current system, so reviewed outcomes are not a random sample. Declined transactions often lack the outcome that would have occurred if approved. The model's decisions therefore shape its future training data.

Mitigations can include carefully designed exploration where risk permits, separate evaluation samples, delayed outcome analysis, propensity-aware methods, and multiple label sources. In an interview, acknowledge that no technique removes all uncertainty. The key is recognizing the feedback loop and measuring it.

Split training and validation chronologically. Random splitting can leak repeated entities and future behavior into the past. Also evaluate by important segments and on a recent out-of-time set because adversarial patterns change.

6. Select and Train the Model

Begin with a defensible baseline: rules plus logistic regression or a tree-based model over tabular features. These models train quickly, expose feature problems, and can meet strict serving budgets. A more complex architecture should earn its cost through measured improvement on the target decision.

Handle imbalance through suitable losses, sampling, and evaluation, but preserve probability meaning when the policy needs calibrated risk. If training uses aggressive sampling, correct for the changed class prior or calibrate on representative data.

Evaluate more than an aggregate curve. Inspect performance by new versus established accounts, transaction type, geography where appropriate, device familiarity, and time. Segment analysis can expose a model that shifts cost onto a particular user group or fails on a new attack pattern.

Feature importance or local explanations can support debugging and reason generation, but they do not prove causality or fairness. Use them as one diagnostic input.

7. Deploy Safely

Version the model, features, rules, and policy together in every decision record. Before a new model controls payments:

1. Validate offline data and point-in-time correctness.
2. Replay it on recent traffic where possible.
3. Run in shadow mode to compare decisions and latency without affecting users.
4. Start with limited exposure or advisory actions.
5. Monitor guardrails and keep a tested rollback path.

Online experiments in fraud require care because treatment changes attacker behavior and determines which outcomes become observable. Randomization units, information leakage, and risk limits need explicit review. Not every policy is suitable for a conventional A/B test.

Design for overload. If the review queue is full, the policy cannot continue routing the same volume indefinitely. It may raise the review threshold, prioritize expected risk, request verification, or use a predefined fallback. This is a product and operations decision, not only an infrastructure concern.

8. Monitor Decisions and Adversaries

Immediate monitoring should include service latency, errors, missing features, score distributions, action rates, rule triggers, and review queue health. Break these down by stable segments to catch localized failures.

Outcome monitoring arrives later. Track matured fraud outcomes, false-positive indicators, amount-weighted loss, review yield, and customer friction against the exact model and policy versions that made each decision.

Watch for drift in inputs and relationships. A sudden rise in new-device traffic might be a product change, instrumentation bug, or attack. Model drift alarms cannot decide which. Investigation needs dashboards that connect feature changes, rules, scores, actions, and eventual labels.

Attackers adapt to defenses. Maintain channels for analysts to report emerging patterns, add temporary rules quickly, generate candidate features, and feed confirmed cases into retraining after label checks. Red-team abuse cases and protect model details that would make evasion easier.

Follow-ups an Interviewer May Ask

"What happens for a brand-new user?"

Account-history features are absent, so rely more on transaction, device, network, merchant, and verification signals. Use missingness explicitly and evaluate cold-start traffic as its own segment. Do not substitute global averages and assume equal certainty.

"How would you use graph information?"

Represent relationships among accounts, devices, payment instruments, and merchants. Start with scalable aggregates such as number of linked accounts, recent risky neighbors, or component-level behavior. More complex graph models may help, but discuss freshness, leakage, latency, and the risk that shared infrastructure creates innocent links.

"What if labels take months?"

Use mature windows for final evaluation, leading indicators for operations, and analyst-confirmed outcomes for faster but potentially biased feedback. Retraining frequency should reflect label availability rather than an arbitrary daily schedule.

"How do you explain a decline?"

Generate reason codes from approved policy and feature logic, not free-form model text. Keep an audit record, expose only information appropriate for the user, and avoid revealing details that enable evasion. Exact obligations depend on product and jurisdiction, so confirm them with the relevant specialists.

For related finance-focused reasoning questions, see machine learning in finance interview questions.

A Strong Two-Minute Summary

"I would separate risk estimation from action policy. The online system uses versioned rules and a low-latency model over point-in-time transaction, history, velocity, and relationship features. The policy maps risk to approve, verify, review, or decline based on expected cost and review capacity. Training uses chronologically split, mature labels and explicitly accounts for selective feedback. I would shadow and gradually roll out new versions, monitor immediate system signals plus delayed outcomes, and keep a fast rule path for emerging attacks."

That summary gives the interviewer a complete system and several productive follow-up paths. It also avoids the common mistake of treating fraud detection as an imbalanced classification notebook.

Practice delivering this design and responding to changing constraints with NeuraPrep's ML system design interview questions at neuraprep.com.