say hello

Fraud Detection Platform

ML risk system · Feb–Mar 2026 · Local case study

A production-shaped fraud scorer with chronology-safe evaluation, calibrated probabilities, cost-aware thresholds, a browser review console, drift checks, challenger reports, and proof artifacts.

All work

What it is

The product is a fraud review service, not just a classifier. It needs risk scores, reasons, thresholds, latency metadata, monitoring, and evidence for why a transaction was flagged.

The business readout matters because fraud decisions trade off caught fraud against customer friction; the shipped threshold catches about 57.9% of fraud with about 5.6% customer friction under the README's assumptions.

The loop it has to make believable

  1. 01

    Separated model fitting, calibration, threshold selection, and final test reporting on chronological slices.

  2. 02

    Selected threshold 0.0700 from explicit cost assumptions rather than a default 0.5 cutoff.

How it’s built

The system demonstrates chronology-safe evaluation, feature contracts, calibration, cost-based thresholding, bundle-based serving, drift checks, and proof-pack generation.

The API serves single and batch predictions, health, metrics, model info, demo summary, and a browser console from the checked-in bundle. Render config exists for permanent hosting; Cloudflare Tunnel is only an ephemeral screen-share path.

Architecture

Data path
IEEE-CIS transaction/identity merge with raw and processed contracts
Protocol
Separate chronological model fitting, calibration, threshold selection, and final test reporting under train_calibration_test_v1
Decisioning
Platt calibration plus threshold 0.0700 selected from $10 false-positive and $100 false-negative costs
Serving
FastAPI loads deploy bundle with model, feature pipeline, schema, operating point, metadata, manifest, and browser console

What has to hold up

  • Packaged model, feature pipeline, schema, operating point, metadata, and manifest into a coherent deploy bundle.
  • Kept calibration, challenger, drift, monitoring, proof-pack, and business readout evidence visible, including a stronger CatBoost offline result.

How I would present it

I would show the latest shipped snapshot and business readout, then inspect the deploy bundle and one `/predict` response.

The honest claim is production-shaped ML engineering. It is not a live fraud product, and the drift/challenger evidence should be discussed rather than buried.

Proof

Fraud is compelling because the repo exposes uncomfortable operational evidence too: CatBoost is the offline challenger leader, drift is visible, and hosted latency should not be claimed without a fresh deployed benchmark.

VerifyRepository, checks, stack, status
  1. 01

    Start with the project status and available evidence.

  2. 02

    Read the architecture and edge-case sections.

  3. 03

    Follow the documented proof path: README, evaluation report, deploy bundle, proof pack, drift report.

Role
Feature path, calibration, thresholding, serving, monitoring artifacts
Scope
Production-shaped fraud scoring system
Proof
README, evaluation report, deploy bundle, proof pack, drift report
Stack
Python, LightGBM, FastAPI, Optuna, MLflow, Prometheus, Grafana
AUC-ROC
0.8658
AUC-PR
0.4806
Threshold
0.0700
Net savings
$308k