Welcome to Braedyn Thompson's portfolio!
See the roles I'm targeting →

Now playing
Berkeley Lab SeismicSoCal BearLM
INTERNSHIP · RESEARCH

CBU Research

Machine Learning Research Intern at California Baptist University. I diagnosed why a structural inverse-FEA predictor stalled, then built SeismicSoCal: live deep-learning earthquake detection, location, sizing and alerts for Southern California.

May 2026 – presentRiverside, CAseismicsocal.duckdns.org
PSsynthetic trace · 30 s window

Overview01

7ML models benchmarkedinverse-FEA diagnosis
~0.10R² ceilingset by the data
0.9998Detection ROC-AUCvs 0.816 STA/LTA
0.951Magnitude R²vs 0.886 baseline

This internship has two research problems. In both, I started by finding out what the data could support.

  • 01

    Diagnosed the root cause of a stalled inverse-FEA predictor by benchmarking 7 ML models (Random Forest, XGBoost, HGB/ExtraTrees ensembles, PCA pipelines), running paired clean-vs-noisy analysis and a noise-identifiability sweep. This proved the R² ≈ 0.10 ceiling was a data limitation rather than a model limitation, gave the team the measurement precision each target needs, and redirected them away from futile model tuning.

  • 02

    Raised earthquake detection to 0.9998 ROC-AUC (vs. 0.816 STA/LTA) and magnitude R² to 0.951 (vs. 0.886) by training PyTorch CNN→Transformer and CNN→GNN→Transformer ensembles on 50,743 windows and 6,243 quakes from 19 live stations, with chronological splits and event-clustered bootstrap CIs.

  • 03

    Built and deployed a real-time SeedLink pipeline that detects, picks, locates (≥ 3 stations, negative evidence) and sizes quakes, then sends two-stage FCM alerts. On a replay of 80 held-out days: 145 push alerts, every one a real quake (chance baseline 0%), median location error 3–4 km, and 83% of isolated M3+ quakes caught.

Diagnosing the inverse-FEA ceiling02

An inverse finite-element problem runs simulation backwards: given a structure's measured response, recover the material properties that produced it. Here, the measured shape of a deformed structure (landmark positions, summarised with PCA) is used to predict the stiffness of several of its components. Two targets were predicted well. One was stuck at R² ≈ 0.10, which was holding up the team's IEEE research paper, and the instinct was to keep tuning models. I set out to find out whether tuning could help at all.

This work is part of an unpublished paper, so its figures and detailed results are held back until publication. The method is below.

01Auditmake the evaluation leak-free first
02Benchmark7 models, same CV
03Clean vs. noisypaired, one variable
04BottlenecksPCA? noisy inputs?
05Sim-to-realtrain clean, test noisy
06Noise sweephow much noise each target tolerates

Step 1 · Make the evaluation trustworthy

Before comparing models I fixed how they were judged, so a small difference couldn't be an artefact:

Leak-free scalingScalers fit on the training fold only, never on the rows being scored.
Out-of-fold scoresEvery reported R² comes from rows the model never trained on.
Predict-the-meanEach target is compared with a trivial baseline, the real test for a hard target.
Overfitting gapTrain-minus-out-of-fold gap reported per target, so a good score can be shown to be real.

I also dropped the chained setup (feeding one target's prediction into the next). The targets are drawn independently, so chaining only passes noise along.

Step 2 · Benchmark seven models

If the bottleneck is the model, different inductive biases should give different answers. I benchmarked seven model configurations under the same cross-validation: a baseline random forest, chained random forests, and my own models, including boosted / ExtraTrees voting ensembles and PCA pipelines. Every one landed on the same ceiling for the hard target, while the easier targets were predicted well by all of them.

Seven very different models hit the same ceiling.

Step 3 · Paired clean vs. noisy data

The decisive test was to train on clean data and on noisy data side by side. If a target is recoverable from clean data but not once realistic noise is added, then no model trained on noisy data will get it back, however it's tuned. That's exactly what happened.

Step 4 · Rule out the obvious bottlenecks

  • Is PCA throwing the signal away? I bypassed it and trained on the raw landmark coordinates, alone and alongside the PCA features. Neither rescued the hard target, so the signal isn't hiding in the discarded components.
  • Are the noisiest inputs drowning it? I measured each landmark group's noise relative to its signal and ran an ablation on only the cleanest group. Dropping the noisiest inputs didn't rescue the target either.

Step 5 · Sim-to-real

The realistic deployment is a model trained on clean simulations and used on noisy measurements. Trained that way it failed on every target until I mean-centred the noisy features onto the clean ones, which fixed a systematic offset and recovered the easy targets. The hard target stayed unrecoverable.

Step 6 · Noise sweep: how much noise each target tolerates

Finally I rebuilt the feature generator, injected Gaussian landmark noise at increasing levels, re-projected through the clean PCA and cross-validated each target. The result is an identifiability curve: the noise level at which each target stops being predictable. At the real measurement noise the hard target is far past that point, and the curve says how precise the measurements would need to be for it to work.

Could an eighth model have fixed it?

No. On clean data the target is recoverable. With the real measurement noise, no model gets it back.

So what does the team do instead?

Fix the measurements. The noise sweep says how much cleaner they need to be.

The outcome

Every line of evidence pointed the same way: the R² ≈ 0.10 ceiling comes from the data. That redirected the team from model tuning to the inputs: what's measured, and at what signal quality. In the meantime, the useful outputs are the targets that are identifiable, plus a calibrated prediction interval (quantile models) for the hard one instead of a misleading point estimate.

Building SeismicSoCal03

The second half of the internship is SeismicSoCal, deep-learning seismology for Southern California, deployed live. It listens to 19 stations in real time, detects, locates and sizes each quake, and sends two-stage alerts to people who follow nearby sensors. Everything measured offline is exactly what runs live, and a replay of real archived days is the acceptance test.

Held-out chronological test set, 2022–2026 (higher is better)0.0000.2500.5000.7501.000Classical seismology baselineDeep model (5-seed ensemble)Detect · ROC-AUCDetect · ROC-AUC — Classical seismology baseline: 0.8160.816Detect · ROC-AUC — Deep model (5-seed ensemble): 0.99980.9998Size · R²Size · R² — Classical seismology baseline: 0.8860.886Size · R² — Deep model (5-seed ensemble): 0.9510.95195% CIs (event-clustered bootstrap): AUC 0.9997–0.9999 vs 0.805–0.828 · R² 0.943–0.959 vs 0.871–0.898
FigureHeld-out chronological test (2022–2026). Detection AUC 0.9998 vs 0.816 STA/LTA; magnitude R² 0.951 vs 0.886 amplitude + distance, with non-overlapping 95% CIs.
01Stream19 SeedLink stations
02DetectCNN→Transformer, every 2 s
03Locate≥ 3 picks, no silent nearer station
04SizeCNN→GNN→Transformer ensemble
05Alerttwo-stage FCM push
Replay of 80 held-out daysResult
Push alerts145, every one a real quake (chance baseline 0%)
Confirmed events that were real86% on busy days, 79% on random days (chance 4% / 0%)
Median location error3–4 km
Pushed magnitudes vs. catalogueMAE 0.12
Isolated M3+ quakes caught83% with 3+ sensors online

It also estimates the shaking at each user's home (MMI), validated on USGS "Did You Feel It?" reports (94% within one intensity level), and it is a production ML system: QuakeOps tracks every training run in MLflow, only promotes a retrained model through a six-rule statistical + replay gate, and checks the live stream for drift every day.

The case study covers the live pipeline, the replay acceptance test and QuakeOps.

Read the full SeismicSoCal case study

How I approach research04

Both halves of this internship (and my other research) follow the same loop. Here is each step, with what it looked like in practice.

The eight steps run as a loop.

  1. 01

    Frame the question

    Turn a vague problem into one that evidence can answer.

    • CBU: is the R² ≈ 0.10 ceiling a model problem or a data problem?
    • SeismicSoCal: not "is the test AUC high?" but "would this live system have alerted correctly on real days?"
  2. 02

    Know the field & pick baselines

    Start from what practitioners already use, so a result means something.

    • STA/LTA for detection, amplitude + distance for magnitude, and a time-shifted chance baseline for every live precision.
    • Persistence for forecasting (Berkeley Lab); a baseline random forest and predict-the-mean for inverse-FEA.
  3. 03

    Design the experiment

    Decide how results will be judged before running anything.

    • Chronological 70/15/15 splits, hard negatives split by date, thresholds calibrated on validation days and test days scored once.
    • Leak-free, out-of-fold cross-validation, and a paired clean-vs-noisy design that isolates noise as the one variable.
  4. 04

    Collect & check the data

    Most bad results start as bad data.

    • 50,743 SCEDC waveform windows and 6,243 quakes labelled against the USGS catalogue, on one shared station list.
    • A quality gate that drops gap-fill zeros, stuck runs, clipping and glitch spikes; three years of XCache logs explored before modelling.
  5. 05

    Run experiments & ablations

    Change one thing at a time and see what actually carries the result.

    • A nearest-single-station ablation (R² 0.951 → 0.808) that proves the graph fusion matters; a 10-seed study; five alert variants replayed over 20 days.
    • A lookback sweep across forecast horizons, and an ablation on only the cleanest inputs for the inverse-FEA model.
  6. 06

    Analyze honestly

    Use metrics that can't be gamed, and write down the limits.

    • ROC-AUC and MCC on imbalanced data, event-clustered bootstrap CIs, and precision always next to its chance baseline.
    • Known limits stated plainly, e.g. off-network quakes are located from one side, and sizes below M2 read high.
  7. 07

    Conclude & communicate

    Say exactly what the evidence supports, to the people who need it.

    • The inverse-FEA ceiling is a data limit, with the measurement precision each target needs: written up for the team's IEEE paper.
    • SeismicSoCal's results published with their baselines and CIs on the live site and its /health page.
  8. 08

    Iterate

    Every answer sets up the next question.

    • Score shadow mode against USGS before switching live pushes on; run the first gated monthly retrain.
    • Test a ~120-day lookback at the 30-day horizon.

Step 8 feeds straight back into step 1.