Study F: Pre-Smoothing

Registration Benchmark v2

1 Overview

Study F: 32,400 runs | 9 DGPs | 6 pre-processing variants | 0.0% failures

  • Weak DGPs (SRVF degrades under noise in Study A): D01, D02, D05, D07, D09, D10
  • Control DGPs (wiggly templates, harder to smooth): D03, D12, D14
  • Method: SRVF only (all variants)
  • Design: 9 DGPs x 6 variants x 3 noise x 2 severity x 100 reps
  • Evaluation: Warps estimated from pre-smoothed curves, then transferred back to raw curves for comparable metrics

2 Primary Estimand: Paired Log-Ratio

The primary estimand is the paired log-ratio of warp MISE (pre-smoothed vs raw SRVF) on the same (dgp, rep) seed. Negative values mean pre-smoothing helps.

2.1 All Variants: Warp MISE Ratio Distribution

Median warp MISE ratio (variant / raw SRVF) and % of runs improved, by noise level.
preproc_id median_ratio_0 median_ratio_0.1 median_ratio_0.3 pct_improved_0 pct_improved_0.1 pct_improved_0.3
lowess_f010 5.03 0.44 0.30 2.44 64.9 84.9
lowess_f015 11.41 0.57 0.24 1.78 58.0 73.8
spline_local_k15 4.25 0.43 0.22 6.72 61.8 76.2
spline_local_k25 1.48 0.40 0.22 18.50 70.8 91.9
spline_global_k25 1.46 0.38 0.22 22.22 71.7 94.7

2.2 Primary Variants: Cell-Level Ratio Distribution

3 Study-Level Decision Rule

Study-level decision rule for the two primary variants.
preproc_id rescue_pass rescue_total control_pass control_total plausible_default
lowess_f015 22 24 1 30 FALSE
spline_local_k25 23 24 11 30 FALSE

LOWESS (f=0.15): 22/24 rescue cells pass, 1/30 control cells pass non-inferiority. Not plausible as unconditional default.

Spline local (k=25): 23/24 rescue cells pass, 11/30 control cells pass non-inferiority. Not plausible as unconditional default.

4 Cross-Over Analysis

  • Control DGPs, severity=0.5: cross-over between noise 0.1 and 0.3
  • Control DGPs, severity=1: cross-over between noise 0.1 and 0.3
  • Weak DGPs, severity=0.5: cross-over between noise 0 and 0.1
  • Weak DGPs, severity=1: cross-over between noise 0 and 0.1

5 Secondary Metrics

5.1 Template Quality

Median template MISE by variant and noise level.
preproc_id 0 0.1 0.3
none 0.0050 0.0740 0.1423
lowess_f010 0.0333 0.0158 0.0186
lowess_f015 0.0836 0.0491 0.0382
spline_local_k15 0.0125 0.0131 0.0195
spline_local_k25 0.0059 0.0068 0.0131
spline_global_k25 0.0057 0.0067 0.0141
Median template elastic distance by variant and noise level.
preproc_id 0 0.1 0.3
none 0.150 0.717 1.929
lowess_f010 0.347 0.263 0.424
lowess_f015 0.531 0.445 0.411
spline_local_k15 0.204 0.216 0.277
spline_local_k25 0.159 0.202 0.263
spline_global_k25 0.158 0.215 0.307

5.2 Alignment Error

Median alignment error by variant and noise level.
preproc_id 0 0.1 0.3
none 0.0077 0.0513 0.301
lowess_f010 0.0440 0.0394 0.115
lowess_f015 0.0722 0.0573 0.113
spline_local_k15 0.0403 0.0488 0.114
spline_local_k25 0.0230 0.0319 0.103
spline_global_k25 0.0231 0.0321 0.103

6 Per-DGP Breakdown

6.1 Warp MISE by DGP and Variant

6.2 Per-DGP Improvement at Highest Noise

Median warp MISE ratio by DGP (noise=0.3, severity=1.0). Values < 1 indicate improvement.
dgp dgp_type lowess_f015 spline_local_k25 spline_global_k25
D01 Weak 0.048 0.046 0.046
D02 Weak 0.175 0.188 0.211
D03 Control 0.890 0.639 0.583
D05 Weak 0.059 0.055 0.057
D07 Weak 0.184 0.186 0.223
D09 Weak 0.117 0.108 0.113
D10 Weak 0.341 0.313 0.349
D12 Control 1.260 0.699 0.738
D14 Control 1.343 0.550 0.511

7 Variant Comparison

Mean rank across cells by variant and noise level.
preproc_id 0 0.1 0.3
none 1.33 4.17 5.33
lowess_f010 4.67 3.44 4.22
lowess_f015 6.00 5.11 3.83
spline_local_k15 4.22 3.78 3.11
spline_local_k25 2.56 2.44 2.22
spline_global_k25 2.22 2.06 2.28

8 Ratio Distributions by DGP Group

9 Comparison with Study A

9.1 Warp MISE

9.2 Alignment Error

9.3 Template MISE

9.4 Template Elastic Distance

9.5 Computation Time

9.6 Summary Table

Pooled median warp_mise across 9 Study F DGPs by noise level.
method 0.1 0.3
SRVF (raw) 0.0017 0.0065
SRVF + spline(k=25) 0.0008 0.0016
cc_default 0.0045 0.0036
cc_crit1 0.0043 0.0041
affine_ss 0.0527 0.0433
landmark_auto 0.0110 0.0123
Pooled median alignment_error across 9 Study F DGPs by noise level.
method 0.1 0.3
SRVF (raw) 0.0533 0.307
SRVF + spline(k=25) 0.0304 0.101
cc_default 0.0959 0.142
cc_crit1 0.1769 0.196
affine_ss 0.2163 0.209
landmark_auto 0.3438 0.466
Pooled median template_mise across 9 Study F DGPs by noise level.
method 0.1 0.3
SRVF (raw) 0.0702 0.1528
SRVF + spline(k=25) 0.0064 0.0127
cc_default 0.0322 0.0329
cc_crit1 0.0840 0.0675
affine_ss 0.0842 0.0644
landmark_auto 0.0466 0.0655
Pooled median elastic_dist across 9 Study F DGPs by noise level.
method 0.1 0.3
SRVF (raw) 0.706 1.909
SRVF + spline(k=25) 0.188 0.262
cc_default 0.309 0.937
cc_crit1 0.289 0.917
affine_ss 0.875 1.004
landmark_auto 0.333 0.830
Pooled median time across 9 Study F DGPs by noise level.
method 0.1 0.3
SRVF (raw) 4.494 4.442
SRVF + spline(k=25) 4.699 4.641
cc_default 8.053 6.263
cc_crit1 10.991 9.924
affine_ss 5.874 4.724
landmark_auto 0.172 0.172

10 Timing

Pre-smoothing adds ~0.2s overhead per run:

Variant Median time (s)
lowess_f010 4.48
lowess_f015 4.48
none 4.49
spline_global_k25 4.54
spline_local_k15 4.61
spline_local_k25 4.68

11 Conclusions

  1. Pre-smoothing strongly rescues SRVF under noise: The best variant (spline_local_k25) reduces median warp MISE by 73% under noise (pooled across noise > 0).

  2. Pre-smoothing hurts in clean settings: Even the least harmful variant (spline_global_k25) inflates warp MISE by 1.46x at noise = 0.

  3. Neither primary variant qualifies as an unconditional default: Both fail too many control cells for non-inferiority, primarily because of the clean-data penalty.

  4. Noise-adaptive recommendation: Pre-smoothing with penalized splines (k=25) is strongly beneficial when noise is known to be present (noise >= 0.1 for weak DGPs, noise >= 0.3 for all DGPs). It should be offered as an option with guidance, not as the default.

  5. Spline variants dominate LOWESS: Penalized splines (k=25) produce less clean-data harm and comparable or better noise rescue than LOWESS smoothing.

  6. Timing is not a concern: Pre-smoothing adds ~0.2s overhead per run.