Project II

Jane Street Market Prediction

Background: Predictive model optimization for high-frequency trading decisions in a Jane Street - Kaggle competition setting.

Achievement:

12% Utility Score improvement through unified target selection

Black swan detection: identified 30 outlier dates (6% of total trading days) via IQR method on aggregated daily returns

SO WHAT:

Reduced Portfolio Variance

Unified target selection would decrease prediction inconsistency between models, leading to more stable returns in live trading.

Lower Drawdown Risk

Black swan filtering would reduce exposure to extreme market events, limiting maximum drawdown during market stress periods

Solution Analysis & Baseline

This section analyzes the competition's evaluation mechanism to identify optimization directions, and examines data characteristics including distribution shifts and multi-horizon response variable relationships to establish a solid foundation for model improvement.

Utility Score Mechanism & Optimization Directions

The competition uses a Sharpe-like scoring mechanism that rewards high total returns while penalizing return volatility. This reveals three optimization directions:

Maximize returns: Improve prediction accuracy through hyperparameter tuning and target selection

Reduce volatility: Filter out extreme market days (black swan events) that cause large return swings

Threshold optimization: Find the optimal confidence cutoff that maximizes utility score

Data Understanding

Distribution shift: Removed the first 85 days with significant feature variance anomalies to ensure training/testing distribution consistency

Multi-Horizon Response Variables: resp_3 provides optimal balance between short-term noise (resp) and long-term lag (resp_4), selected as prediction target.

Model Optimization

Building on the baseline analysis, this section implements three optimization strategies: black swan detection, XGBoost hyperparameter tuning, and AE-MLP target selection

Data Processing

Black swan detection aggregated weighted returns by day and applied IQR method to identify 30 extreme market days.

NA handling was optimized by switching XGBoost from mean imputation to forward-fill (unified with AE-MLP), better reflecting financial market continuity patterns and avoiding future information leakage.

XGBoost Hyperparameter Configuration (3-Way Comparison)

Three complete configuration sets were designed for systematic comparison: Baseline (depth=11, weight=1, lr=0.05), Conservative (depth=7, weight=10, lr=0.05), and Aggressive (depth=7, weight=10, 300 trees, lr=0.03). Each configuration contains multiple synergistically adjusted hyperparameters, avoiding the limitations of stepwise single-parameter experiments.

AE-MLP Target Selection

Rapid validation strategy used 2 folds with epochs reduced to 30, balancing efficiency and reliability. Five configurations were compared, outputting AUC to assess performance.

Black Swan Analysis

Evaluation

Compared individual model performance across configurations, then evaluated ensemble strategies to quantify improvement from unified target selection.

XGBoost Hyperparameter Comparison

Baseline (depth=11) achieved Utility 2020, outperforming Conservative (-3.1%) and Aggressive (-6.3%). Deeper trees better captured complex feature interactions in this dataset.

AE-MLP Target Comparison

5-fold CV showed resp_3 (AUC 0.528) slightly outperformed 5-targets (AUC 0.524). Selected resp_3 to align with XGBoost target for ensemble consistency.

Final Ensemble Comparison

Unified target ensemble achieved Utility 2675, a 12% improvement over the mixed target baseline (2388) and 20-32% over single models.

Final Performance Comparison