Risk Data / Feature Engineering
iOS Risk Data Foundry
Risk-data collection and feature-engineering infrastructure, with a reproducible benchmark that measures what the engineered features actually add on a public fraud dataset — including where they cost you.
01 — The problem
A risk model is only as defensible as the features underneath it.
Risk teams inherit feature sets nobody can reproduce: derived columns written once, tuned by hand, and never measured against the baseline they replaced. When the model drifts, there is no way to tell whether the features, the data or the world changed.
The question a reviewer actually asks is narrow and awkward: what did this feature set add, on what data, measured how — and does the improvement survive someone else running it?
02 — The system
Collection, engineering and measurement in one reproducible pipeline.
Raw risk data is ingested into a typed pipeline, engineered features are derived deterministically, and the same harness scores a base feature set against the engineered one on a frozen public dataset. Every stage is tested, linted, containerised and gated in CI — so the benchmark is something a reviewer can rerun rather than take on trust.
Typed ingestion
Raw risk records validated at the boundary and stored before any derivation runs.
Deterministic feature derivation
Engineered features are pure functions of the input, with explicit seeds where randomness is involved.
Synthetic generation
Seeded synthetic fraud records for pipeline exercise, kept structurally separate from observed data.
Benchmark harness
Base versus engineered scored on the same split, by the same scorers, in one command.
Containerised runs
The whole pipeline runs in Docker, so the benchmark environment is part of the artefact.
CI gating
Tests, lint, dependency validation and the Docker build all run on every change.
03 — Architecture
How a benchmark run executes
04 — Engineering decisions
Choices that make the number worth quoting.
One public dataset, named and pinned
The benchmark runs on Kaggle mlg-ulb/creditcardfraud at a pinned harness commit. A result on an unnamed internal dataset is not evidence anyone outside the company can check.
Base and engineered share everything else
Same split, same model family, same scorers. The only variable is the feature set, which is the only way the delta means anything.
Report the metric that got worse
Precision fell while recall rose. Publishing the full metric table rather than the flattering subset is the difference between a benchmark and a marketing number.
Synthetic data is seeded and labelled
Synthetic fraud records are deterministic under explicit seeds and never mixed into observed data or presented as production evidence.
The environment ships with the result
Docker image plus a pinned XGBoost version, because a benchmark that only reproduces on the author’s laptop has not been reproduced.
CI is the gate, not a courtesy
Both the Python and Docker jobs must pass. A feature change that breaks the pipeline cannot reach the benchmark.
05 — Reliability & controls
What the pipeline does when things go wrong.
06 — Evidence
Engineering verification
- 33 automated tests passed on Python 3.11.15
- Ruff lint and formatting checks passed
- Dependency validation passed
- Docker image built successfully
- Container test command passed all 33 tests
- GitHub Actions passed both the Python and Docker jobs
Verified results
None published. No client or institutional production results are claimed here — this is a public-dataset benchmark, not a deployment record.
Benchmarked results
Public dataset: Kaggle mlg-ulb/creditcardfraud
Dataset size: 284,807 transactions
Evaluation harness commit: 5144f8c
XGBoost version: 3.2.0
| Metric | Base | Engineered | Delta |
|---|---|---|---|
| Average precision | 0.8510 | 0.8588 | +0.0079 |
| ROC AUC | 0.9747 | 0.9803 | +0.0055 |
| F1 | 0.8415 | 0.8586 | +0.0171 |
| Precision | 0.9059 | 0.8817 | −0.0242 |
| Recall | 0.7857 | 0.8367 | +0.0510 |
The engineered features improved average precision, ROC AUC, F1 and recall, while precision declined. This is a measured trade-off on one public dataset — not proof of production superiority.
Simulated results
Synthetic fraud generation is implemented and tested for deterministic output using explicit random seeds. Synthetic records are clearly separated from observed public data and are not presented as production evidence.
Projected business value
Not measured or claimed. Production financial impact would require representative institutional data, operational cost modelling and live validation.
Evidence links
Reproduce locally
The benchmark is not run in your browser — it downloads a large dataset, installs native ML dependencies and trains two models. Run it yourself instead:
git clone https://github.com/Etherlabs-dev/ios-risk-data-foundry.git
git clone https://github.com/Etherlabs-dev/eval-harness.git
cd ios-risk-data-foundry
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements-validation.txt
mkdir -p data/raw
curl -fL \
https://www.kaggle.com/api/v1/datasets/download/mlg-ulb/creditcardfraud \
-o data/raw/creditcardfraud.zip
unzip data/raw/creditcardfraud.zip -d data/raw
PYTHONPATH=../eval-harness/ios-risk-eval-harness:. \
python scripts/validate_features.pyOn macOS, XGBoost additionally requires the OpenMP runtime, usually installed with `brew install libomp`.
Verified, benchmarked, simulated and projected figures are kept separate on purpose. A benchmarked number is never presented as a verified client result.
07 — Technology
Technology supports the story. The architecture and the controls are the story.
08 — Repository & demo
Can you defend your feature set in review?
We can put a baseline, a held-out split and an acceptance metric behind the features you already ship.