3 March 2023One night, as the model saw it
- Weighted global move
- Shock signals
- Model predicted
- NASDAQ opened
Real days from the 496-day test set. Asian markets close before New York opens; European markets are still trading, so the model only uses their previous close. Using the same day would leak the future into the forecast.
Problem
Major markets in Asia and Europe trade while New York sleeps. For Move Tickers, a financial-data company, the question was whether those overnight moves can signal how the NASDAQ-100 will open, so its users know what to expect at the opening bell.
What we built
We combined ten years of daily data (2015 to 2025) from seven global indices with NASDAQ-100 prices, engineered percentage moves, volatility "shock" flags and a weighted global move, and labelled each day's opening gap. We compared four model types against a baseline, tuned XGBoost with time-series cross-validation to avoid look-ahead bias, and packaged the pipeline as a modular, tested Python project.
My role
I was the team lead, coordinating our planning and project board, and I owned the data foundation: I designed and built the PostgreSQL database that every model read from and wrote its results back to. I also contributed to model development, modular code and unit tests, and led the final presentation to the client.
Result
Always predicting "flat" scores 53.4% accuracy, and tells you nothing. Our tuned XGBoost scored lower on accuracy, 47.6%, but it was the model that actually caught moves: it identified 59% of down-gap days, against 0% for the baseline, and reached the best macro ROC-AUC of the models we tested (0.63).
The accuracy trap
Scores on the same 496 held-out trading days. Flat days make up 53% of the test set, which is what makes accuracy misleading here.
What I'd do now
Daily closing prices were a deliberate starting point. With what I know now, I would push the signal closer to the opening bell:
- Use Europe's live session. European markets are halfway through their day when New York opens. Prices from just before 9:30 in New York would add the most recent information without leaking the future.
- Add pre-market signals. NASDAQ futures, the VIX and the dollar-yen rate react overnight and are likely stronger predictors than a weighted average of indices.
- Judge it like a trader would. Backtest a simple strategy with transaction costs, and tune the decision threshold for each class, instead of relying on accuracy alone.
- Retrain as markets change. A walk-forward scheme that retrains regularly would handle shifts like 2020 and 2022 better than a single fixed model.
Skills gained
- Database design: building a PostgreSQL database with SQL and SQLAlchemy as the team's single source of truth
- Time-series features: percentage moves, rolling volatility, shock flags and time-zone alignment
- Model evaluation: choosing macro ROC-AUC over accuracy for imbalanced classes, with time-series cross-validation
- Software engineering: modular packaging with Poetry, linting, unit tests and documentation
- Leadership: planning and coordinating a team of four, and presenting the results to the client
- AI regulation: classifying a proposed system's risk under the EU AI Act and assessing GDPR implications
Contact
Looking for a data science or AI intern?
I'm open to internships in data science, machine learning and data engineering for February 2027. Email is the quickest way to reach me.