Adaptive Graph-Enhanced Real-Time Fraud Detection Platform
Real-time fraud detection using streaming transaction processing and graph-derived structural features.
Project Overview
The Adaptive Graph-Enhanced Real-Time Fraud Detection Platform is a streaming fraud-detection system designed to identify suspicious financial transactions as they move through a real-time processing pipeline.
Rather than analysing transactions only as isolated events, the system incorporates graph-derived structural features that capture relationships between entities involved in the transaction network. This provides additional context for identifying suspicious patterns that may be difficult to detect using transaction attributes alone.
Apache Kafka handles real-time transaction ingestion, while Spark Structured Streaming continuously processes incoming events and generates transaction-level features. Graph-based features are then derived from the evolving transaction network and combined with transactional attributes for machine-learning-based fraud classification.
The project demonstrates the integration of stream processing, graph-based feature engineering, and machine learning within an end-to-end fraud analytics pipeline.
Technology Stack
- Python
- Apache Kafka
- Apache Spark Structured Streaming
- Machine Learning
- Graph-Based Feature Engineering
- Real-Time Stream Processing
System Architecture
The system uses a streaming architecture that transforms incoming transaction events into enriched feature representations for fraud prediction.
- Transaction Producer — generates and publishes incoming financial transaction events.
- Apache Kafka — provides the real-time messaging layer for transaction ingestion.
- Spark Structured Streaming — continuously validates, transforms, and processes transaction events.
- Feature Engineering Layer — generates transaction-level behavioural features from the incoming stream.
- Graph Feature Engine — derives structural information from relationships within the transaction network.
- Fraud Detection Model — combines transactional and graph-derived features to classify transactions and estimate fraud risk.
System Workflow
- Financial transactions enter the system as streaming events.
- Apache Kafka publishes the events to the transaction stream.
- Spark Structured Streaming continuously consumes and processes the incoming data.
- Transaction-level features are generated from transaction attributes and behavioural history.
- Graph-based features capture structural relationships and activity patterns within the transaction network.
- Transactional and graph-derived features are combined into an enriched feature representation.
- The machine-learning model evaluates the resulting features and produces a fraud prediction for each transaction.
Key Features
- Real-time ingestion of financial transaction streams using Apache Kafka.
- Continuous transaction processing with Spark Structured Streaming.
- Graph-based modelling of relationships within transaction data.
- Integration of transactional and structural features for fraud detection.
- Machine-learning-based fraud classification.
- End-to-end streaming architecture for continuously arriving transaction data.
Results
- Precision: 0.87
- Recall: 0.91
- F1-Score: 0.89
- ROC-AUC: 0.93