Before-and-After Measurements that Actually Compare

[Pavel]

Short answer: A valid comparison changes one variable while holding signal, time window, gain, bandwidth, antenna position and metric constant—or explicitly normalizes unavoidable differences. Screenshots alone are not evidence when display range and conditions differ.

Before you start​

• Define the project's purpose and one measurable success criterion before listing hardware.
• Document the complete RF and data path, including antenna, feedline, filters, gain, SDR, sample rate, software and processing output.

Step-by-step method​

1. Step 1: Write a hypothesis and primary metric before testing, such as a filter will increase valid ADS-B messages while reducing overload products.
2. Step 2: Capture baseline over a duration suited to variability and save raw settings plus environmental conditions.
3. Step 3: Change one component, repeat the same duration and use identical software scaling. Alternate A/B/A when time variation may bias the result.
4. Step 4: Report raw values, percentage change, sample count and limitations. Keep unsuccessful or neutral results rather than selecting only the best screenshot.

Concrete example​

Antenna A measured Monday for ten minutes and antenna B Friday for one hour is not a fair ADS-B comparison. Alternating A/B across matched 24-hour periods with fixed gain and reporting aircraft plus MPS is much stronger.

How to judge the result​

The comparison is credible when reversing the change moves the result back or repeated trials show the same direction beyond normal variation.

What to record​

• Use equal observation windows and unchanged receiver settings for before-and-after comparisons.
• Include raw counts, signal-to-noise ratio or error rate alongside screenshots so the improvement is measurable.
• Record failed approaches and constraints; they often make a build log more useful than a polished final photograph.

Common mistakes​

• Changing several parts between baseline and result, then crediting one component without evidence.
• Publishing a polished result without the versions, settings, failures and measurement needed to reproduce it.
• Autoscaled waterfall colours can make identical RF levels look dramatically different.

Final check​

Repeat the baseline or rebuild from the written log. Publish the result only when another member can follow the RF chain, settings and measurement without relying on private files or unstated changes.