Welcome to Braedyn Thompson's portfolio!
See the roles I'm targeting →

Now playing
Berkeley Lab SeismicSoCal BearLM
ROLE I'M TARGETING

Data Scientist

Design honest experiments and find out what the data can support.

Experiment designCausal inferenceForecastingUncertainty
SKILL CHECK . . . OK+4.2%eBART riders, pre-registered0.004A/B result, McNemar p~0.10R² data ceiling at CBU0.9998detection ROC-AUC

Skill map01

Each row points to the section below that shows it.

SkillWhereProof
Causal inferenceBayRidershippre-registered difference-in-differences against 34 control stations, with placebo intervals
A/B testing and powerBayRidershipa power analysis before arm B ran; exact McNemar p = 0.004
Forecast uncertaintyBayRidershipquantile LightGBM with a split-conformal band; 73% coverage reported against an 80% target
Validation without leakageBerkeley Lab, SeismicSoCalrolling-origin CV; chronological splits; thresholds frozen on validation days
Hypothesis testingBerkeley Labtwo explanations for missed peaks (five losses, percentile forecasts), both rejected
IdentifiabilityCBU Researcha paired clean-vs-noisy design and a noise sweep
Confidence intervalsSeismicSoCalevent-clustered bootstrap; a 10-seed study separates seed from sampling variance
Probabilistic modelsAcademy of TestersBKT + IRT over 8 skills; the essay grader scored by QWK

BayRidership · causal study, A/B test, forecasts02

A Bay Area transit analytics platform I built end to end. Three of its questions needed statistics: did the eBART extension bring new riders, do retrieved examples make the SQL agent better, and how sure is a two-week forecast?

+4.2%eBART net ridersplacebo interval +1.6% to +7.1%
27 → 50%Agent accuracyMcNemar p = 0.004
48%Lower forecast errorMAE 283 vs 549
73%Band coverageagainst an 80% target

  • 01

    Chose the unit to separate new riders from switchers. A rider who switches from Pittsburg/Bay Point to Antioch leaves the line's total unchanged; a new rider raises it. So the study compares the whole line and the three old stations with 34 far-away stations, with station and week fixed effects on log weekly entries: +4.2% for the line, −23.9% for the old stations.

  • 02

    Led with placebo inference: reassign the extension to all 5,984 sets of 3 control stations and see how often noise alone looks this big (RI p = 0.0007). Intervals from clustered standard errors are reported too, but with only 3 treated stations they under-cover on synthetic data, so the placebo intervals are the headline.

  • 03

    Pre-registered the design in a hashed file, then validated it on 200 simulations, which caught a too-narrow interval (coverage 0.84, fixed to 0.925 by one dated amendment). The post-hoc trend check gives +1.0% and is labelled post-hoc.

  • 04

    Ran the agent A/B by the book: a frozen question set, a power analysis after arm A (+14 points detectable at 80% power), a three-part decision rule, then arm B. It won by +22.7 points (95% CI +9.1 to +36.4); near-twin examples in the pool mean unfamiliar questions will gain less, and the write-up says so.

  • 05

    Back-tested the forecasts walk-forward with a paired bootstrap CI on the improvement (200 to 340 entries a day), and named the band a “calibrated model range” because it held 73%, not 80%.

Berkeley Lab · forecasting and testing why peaks are missed03

Data Science & ML Research Intern at Lawrence Berkeley National Laboratory (ESnet), Sep 2026 – present, forecasting traffic at the regional data caches that serve CMS physicists. I test how predictable that traffic is, peaks included.

24%Lower RMSE than persistenceevery variable and horizon
5Losses testedMAE to MSE, same spikes missed
100Day lookbackbeat every 3-day setup on RMSE

  • 01

    Compared PatchTST with persistence and linear regression on a chronological hold-out, never a random split, which would leak the future.

  • 02

    Noticed RMSE hid the real failure: the model tracked the everyday level but smoothed through every spike, so I scored each experiment on both average error and peaks.

  • 03

    Picked the lookback with rolling-origin cross-validation, then checked it wasn't luck: on RMSE every 100-day configuration beat every 3-day one, and re-runs across 5 seeds barely moved.

  • 04

    Turned the missed peaks into two hypotheses (the loss is to blame; forecast high on purpose) and rejected both. The information isn't in past traffic, so the team's question became what signal would make peaks predictable.

  • 05

    Split one cache's history into a calm and an erratic era: calm traffic rewards longer history, erratic traffic the last few days. The answer is to detect the regime and use the model tuned for it.

CBU Research · proving a data ceiling04

ML Research Intern at California Baptist University, May 2026 – present.
A model predicting a structure's stiffness from its measured shape (an inverse finite-element problem) was stuck at R² ≈ 0.10 on one target. I tested whether any model could do better.

7Model configurationssame cross-validation
~0.10R² ceilinga limit in the data

  • 01

    Audited the evaluation first (leak-free scaling, out-of-fold scores, a predict-the-mean baseline, the overfitting gap) and dropped the chained setup, which passed noise from one target to the next.

  • 02

    Benchmarked seven configurations, from a baseline random forest to boosted and ExtraTrees ensembles and PCA pipelines. All hit the same ceiling on the hard target.

  • 03

    Isolated noise as the one variable with a paired clean-vs-noisy design: the target was recoverable from clean data and gone once realistic noise was added.

  • 04

    Ruled out the obvious bottlenecks: bypassing PCA and training on only the cleanest inputs didn't rescue it.

  • 05

    Trained on clean simulations and tested on noisy measurements, then swept the noise level to get an identifiability curve. For the hard target a quantile prediction interval is more useful than a point estimate.

SeismicSoCal · deep vs. classical, with confidence intervals05

A live earthquake detection system on 19 seismometers across Southern California, built during my CBU internship. Each model is measured against the classic seismology method it replaces.

0.9998Detection ROC-AUCvs 0.816 STA/LTA
0.951Magnitude R²vs 0.886 amp + distance
94%Within one MMI level28 held-out quakes

  • 01

    Split every dataset chronologically (70/15/15), the 2,062 hard negatives of real live-stream noise included, so the test set is the future.

  • 02

    Bootstrapped confidence intervals by event, so a quake's correlated station windows resample together. The deep and classical intervals don't overlap.

  • 03

    Ran a nearest-station ablation: with only the closest station, magnitude R² falls to 0.808, worse than the classic formula. The accuracy comes from combining stations.

  • 04

    Separated seed variance from sampling variance with a 10-seed study. Ensembling added only +0.003, inside the sampling CI, so the live system keeps 5 seeds.

  • 05

    Fitted the shaking-intensity offset on the older 29 quakes and scored it once on the newer 28 (1,892 report cells). The ground-type term gave no measurable gain, and I report that too.

Academy of Testers · modelling what a student knows06

A full-stack AP/SAT study platform I built as ML & AI Software Developer (Jan 2023 – Jun 2026). Its adaptive SAT engine estimates what each student knows and picks the question that will tell it the most.

8Skills modelled per studentBKT + IRT
0.62QWK vs. official scoresfrom 0.590
22%Lower error on held-out APUSHnever tuned on

  • 01

    Tracked mastery with Bayesian Knowledge Tracing and damped each step (25% of a gain, 50% of a loss), so one lucky guess can't fake mastery; a guard makes sure a wrong answer never raises a skill.

  • 02

    Chose the next question with IRT: each item's 3PL Fisher information at the student's ability, blended with the skill gap and recency, then sampled from the top 5 with a softmax.

  • 03

    Modelled forgetting as exponential decay toward a 0.20 floor (λ = 0.0112/day, so the part above the floor halves in about 62 days).

  • 04

    Designed the essay-grader A/B so nothing leaks: test essays from 2023/2024, the retrieval corpus from 2025.

  • 05

    Withheld difficulty until after submission, because knowing it changes how students answer and would contaminate the data used to recalibrate it.