1. Background & Regulatory Context
1.1 The Alternate Test Procedure Program
The EPA Alternate Test Procedure (ATP) program, codified at 40 CFR 136.4 (nationwide) and 40 CFR 136.5 (limited-use, facility-specific), provides a pathway for developers to submit new analytical methods for regulatory approval as alternatives to methods listed in 40 CFR Part 136. These are the required methods for Clean Water Act compliance reporting. This document describes the study design and statistical analysis plan for a limited-use ATP application under 40 CFR 136.5, submitted by the City of Boulder Utilities to EPA Region 8.
1.2 Colorado Regulation 93 / 303(d)
Boulder Creek is listed as impaired under Colorado Regulation 93 (the state implementation of the federal 303(d) program for impaired waters) for E. coli exceedances. The applicable water quality standard is a geometric mean of 126 CFU/100 mL over a 30-day period, with a single-sample maximum of 235 CFU/100 mL. The City of Boulder Utilities currently monitors compliance at 6 designated monitoring locations on Boulder Creek using IDEXX Colilert as the reference method. This study will determine whether the Virridy Lume tryptophan-like fluorescence (TLF) sensor produces results statistically equivalent to Colilert at these compliance monitoring sites.
1.3 The Case for Continuous Monitoring
Current Reg 93 compliance monitoring on Boulder Creek relies on discrete grab samples processed by a laboratory with an 18–24 hour delay before results are available. This creates a structural monitoring gap: contamination events driven by storm runoff, upstream discharges, or wildlife activity may occur and resolve entirely between sampling events, leaving the public exposed without warning. The Lume sensor provides continuous, in-situ E. coli estimates at 15-minute intervals, enabling real-time detection of exceedance events and substantially increasing the density of compliance-relevant data.
1.4 Relationship to Existing Data
Virridy has previously collected 856 paired Lume–Colilert observations across multiple sites and waterbodies (global dataset, 2021–2025) (R² = 0.67, Cohen’s kappa = 0.82–0.84, MAPE = 7.1% in log-transformed space). This existing dataset is used to support the pre-submission consultation with EPA Region 8 and demonstrates technical feasibility, but does not constitute the formal regulatory validation dataset. The prospective study described in this document will generate the regulatory-grade paired dataset required for the 136.5 application, under a pre-specified, agreed study protocol.
Modeling after Colilert: This study design is modeled on the validation framework used by IDEXX to obtain EPA approval for Colilert, which included: (1) precision studies within and across laboratories, (2) comparative paired testing against existing approved methods across multiple matrices, (3) interference characterization, (4) method detection limit determination, and (5) pre-specified statistical acceptance criteria. Those same elements are incorporated here, adapted for continuous in-situ monitoring.
2. Study Objectives & Regulatory Claim
2.1 Primary Objective
Demonstrate that the Virridy Lume tryptophan-like fluorescence sensor, using a multivariate linear regression model with TLF intensity, turbidity, and temperature as inputs, produces E. coli concentration estimates that are statistically equivalent to IDEXX Colilert results at Boulder Creek’s 6 Regulation 93 TMDL monitoring sites across all seasons, flow conditions, and concentration ranges relevant to Reg 93 compliance.
2.2 Secondary Objectives
- Demonstrate correct binary classification of samples above and below the Reg 93 geometric mean threshold (126 CFU/100 mL) with pre-specified sensitivity and specificity targets
- Characterize method precision (sensor-to-sensor reproducibility) across the 6 monitoring sites
- Determine the method detection limit (MDL) in both TLF units and translated CFU/100 mL
- Characterize known interferences (dissolved organic carbon, humic fluorescence, turbidity, temperature) and their effect on the prediction equation
- Generate a complete method document suitable for inclusion in a 40 CFR 136.5 application
2.3 Regulatory Claim Hierarchy
The primary regulatory claim is direct replacement of Colilert grab sampling for Reg 93 compliance reporting. Fallback claims are listed in order of preference if the primary claim cannot be fully supported by the dataset:
Direct Replacement of Colilert Grab Sample
The Lume reading at the time of a scheduled compliance sampling event replaces the Colilert result as the official Reg 93 compliance value. City of Boulder no longer needs to submit grab samples for laboratory Colilert analysis at approved monitoring sites. This is the strongest regulatory claim and requires the tightest statistical equivalence demonstration.
Continuous Compliance Determination
A rolling geometric mean of Lume readings over a defined period constitutes the compliance measurement, independent of grab sampling. Requires agreement with EPA and CDPHE on the aggregation period and how continuous data maps to the Reg 93 geometric mean standard.
Screening with Colilert Confirmation
The Lume is the primary continuous monitor. A Lume exceedance above a defined threshold triggers an immediate Colilert grab sample for confirmation before any official compliance action. The Lume is the compliance tool; Colilert is the safety net.
Supplemental / Advisory Use
Lume data supplements but does not replace Colilert for official compliance reporting. Used for real-time public advisories, operational decisions, and interim monitoring between scheduled Colilert events. Lowest regulatory bar; does not require equivalence demonstration.
3. Method Description
3.1 Principle of the Method
The Lume sensor measures tryptophan-like fluorescence (TLF) in situ at 275 nm excitation and 340 nm emission. Tryptophan is an amino acid present in the cell walls of bacteria, including E. coli, and its fluorescence signal is a proxy for fecal contamination. The sensor simultaneously measures turbidity (NTU) and water temperature (°C) as co-variates. A multivariate linear regression model, fit to paired Lume–Colilert calibration data and validated on a held-out test set, converts the three sensor outputs to an E. coli concentration estimate in CFU/100 mL.
3.2 Prediction Equation
The regulatory prediction equation takes the following form, fit in log10 space to account for the log-normal distribution of environmental E. coli concentrations:
Where TLF is in ppb tryptophan equivalents, turbidity is in NTU, and temperature is in °C. The model coefficients (β0–β3) will be estimated by ordinary least squares regression on the training partition of the study dataset (see Section 8). The fitted coefficients will be locked before evaluation on the holdout test set and will not be modified thereafter. The final coefficient values will be reported in the ATP application and become fixed parameters of the approved method.
Why linear regression: A multivariate linear regression model was selected because: (1) the equation is fully transparent and auditable by EPA on a spreadsheet; (2) there are no versioning or retraining concerns once coefficients are fixed; (3) the regulatory precedent for linear correction equations in approved methods is well-established (e.g., turbidity corrections in UV absorbance methods, temperature corrections for dissolved oxygen); and (4) no site-specific calibration is required, making the method universally applicable.
3.3 Sensor Specifications
| Parameter | Specification |
|---|---|
| TLF excitation wavelength | 275 nm |
| TLF emission wavelength | 340 nm |
| TLF units | ppb tryptophan equivalents |
| Turbidity measurement | Nephelometric (NTU) |
| Temperature measurement | °C (thermistor) |
| Measurement interval | 15 minutes (configurable) |
| Deployment type | Submersible in-situ sensor |
| Data transmission | Cellular (primary); on-board SD card (backup) |
| Power | Solar or line power with battery backup |
| Operating temperature range | 0–40 °C |
3.4 Method Scope and Applicability
This method is applicable to freshwater surface water for the measurement of E. coli in the range of approximately 1–10,000 CFU/100 mL. The method is designed for continuous in-situ monitoring and is not applicable to discrete grab sample laboratory analysis. The method as validated is specific to Boulder Creek, Colorado under the 40 CFR 136.5 limited-use approval; nationwide applicability under 40 CFR 136.4 will require additional multi-site validation.
4. Study Design
4.1 Study Sites
The study will be conducted at the 6 City of Boulder Utilities TMDL compliance monitoring locations on Boulder Creek. These are the same sites at which Boulder currently collects Colilert grab samples for Regulation 93 reporting to CDPHE. Using existing compliance monitoring sites ensures that the validation dataset directly reflects the conditions and concentration ranges relevant to the regulatory claim.
| Site | Description | Expected Range | Regulatory Significance |
|---|---|---|---|
| BC-Can | Upstream reference / background | Low (<10 CFU/100 mL) | Establishes baseline TLF and natural organic matter background; critical for MDL characterization |
| BC-Eben | Canyon mouth / Eben Fine Park | Low–moderate | Mixed urban/natural signal; entry point for canyon runoff |
| BC-13 | 13th Street compliance point | Variable — low to high | Central compliance point; demonstrates performance at and above the 126 CFU/100 mL Reg 93 threshold |
| BC-CU | CU campus compliance point | Moderate | Mixed urban stormwater signal; seasonal variability important |
| BC-30 | 30th Street compliance point | Moderate–low | Natural attenuation zone; demonstrates performance during dilution and recovery |
| BC-55 | 55th Street / watershed outlet | Variable | Integrates full watershed; irrigation diversions may affect flow dynamics seasonally |
Note: Site descriptions and names are provisional. Exact site IDs, GPS coordinates, and Reg 93 permit associations to be confirmed with City of Boulder Utilities prior to study start.
4.2 Study Duration
The study will run for 12 months from the formal study start date (defined as the date all 6 sensors are deployed and the reference method sampling program is operating at the required frequency). Twelve months is the minimum duration required to capture all four seasons and, critically, two complete spring runoff events on Boulder Creek. Spring runoff (typically April–June) produces the highest E. coli concentrations of the year and is the period most likely to generate exceedances of the Reg 93 threshold. It must be represented in both the training and holdout partitions of the dataset.
Minimum duration rationale: A 9-month study (fallback) is statistically achievable in terms of sample counts but risks missing adequate representation of spring runoff in the holdout period. EPA is likely to scrutinize performance at high concentrations near the Reg 93 threshold specifically. A 12-month study captures spring runoff twice and eliminates this vulnerability. The existing 856-observation dataset may be submitted to EPA Region 8 as supplementary evidence if a reduced prospective study duration is requested.
4.3 Sensor Deployment
| Task | Timing | Details |
|---|---|---|
| Deploy Lume sensors at all 6 sites | Pre-study | Each sensor co-located within 2 meters of the designated Colilert grab sample collection point at the same depth and cross-section position. GPS coordinates, installation photographs, and sensor serial numbers documented at deployment. |
| Firmware and model version lock | Study Day 0 | Firmware version and prediction equation coefficients (once fit per Section 8) are locked for the study duration. Any firmware updates require a protocol amendment and notification to EPA Region 8. |
| Data pipeline validation | Week 1 | Confirm all 6 sensors are transmitting reliably. Verify measurement interval, data completeness, and sensor-to-server latency. Automated alerts configured for signal dropout, out-of-range readings, and battery/power status. |
4.4 Operators and Responsibilities
| Party | Role |
|---|---|
| Virridy | Sensor deployment, maintenance, calibration verification, data management, and statistical analysis. Contracted to City of Boulder for the duration of the study. |
| City of Boulder Utilities | Applicant on the 136.5 ATP application. Oversight of the study. Coordinates Colilert grab sample collection at the 6 monitoring sites per existing Reg 93 SOPs. |
| Reference Laboratory | State-certified laboratory for E. coli analysis. Processes all Colilert samples for the study. [See Open Question B-1: confirm certified lab and willingness to increase sampling frequency] |
| CDPHE | State regulatory oversight. Reviews study design and provides letter of support for the 136.5 application. |
| Independent Statistical Reviewer | Per EPA §4.4, data compilation must not be performed by the analysts conducting the sample analyses. City of Boulder Utilities will independently verify and certify all statistical outputs before ATP submission. An independent third-party statistician (role to be confirmed; CU Boulder faculty is a candidate) will review SAP execution on the holdout dataset before submission to EPA Region 8. [See Open Question G-1] |
5. Reference Method Protocol
5.1 Reference Method
The reference method for this study is IDEXX Colilert for E. coli quantification in freshwater, analyzed using the Quantitray/2000 MPN method per the manufacturer’s protocol and consistent with 40 CFR Part 136 Table IA. The specific Colilert formulation (standard 24-hour vs. Colilert 18-hour) will be confirmed with City of Boulder Utilities and will match the version currently used for Reg 93 compliance reporting. Both are EPA-approved; consistency with Boulder’s existing compliance program is required so that the ATP validation dataset is directly comparable to the regulatory baseline.
Critical prerequisite — Laboratory Certification: All Colilert reference samples used in the regulatory validation dataset must be analyzed by a laboratory holding current state certification under the Colorado Laboratory Certification Program (CLCP) for E. coli analysis. This is a prerequisite, not a preference. Uncertified reference data is grounds for application rejection. [See Open Question B-1 and B-2]
5.2 Sample Collection Protocol
| Parameter | Requirement |
|---|---|
| Collection frequency | Minimum 3 times per week per site during the study period. Event-based enhanced sampling (see 5.3) increases this frequency during high-flow events. |
| Collection timing | Exact time of grab sample recorded to the minute by the collector. Matched against nearest Lume 15-minute reading (pairing window: ≤15 minutes; see Section 6). |
| Collection location | Within 2 meters of the Lume sensor, same depth and cross-section position. |
| Collection containers | Sterile, single-use 100 mL bottles per Part 136 / Colilert protocol. |
| Preservation and holding time | Per approved Colilert protocol: chilled at 4 °C, analyzed within 6 hours of collection. |
| Chain of custody | Formal chain of custody documentation for every sample, from collection through laboratory reporting. COC retained for minimum 10 years. |
| Field conditions log | At each sampling event: weather (clear/rain/snow/runoff), visual water clarity, estimated stream flow condition (baseflow/elevated/flood), any visible upstream disturbance, air temperature, and collector initials. |
| Laboratory duplicate | Every 10th paired sample: collect a field duplicate grab (two bottles from the same sampling event). Submitted blind to the laboratory. Provides reference method precision estimate (relative percent difference). |
5.3 Enhanced Sampling During High-Flow Events
Spring runoff and storm events are the primary drivers of Reg 93 exceedances on Boulder Creek. The Lume’s continuous monitoring will automatically detect elevated TLF signals. When any Lume sensor reads above a pre-defined alert threshold (to be determined during initial deployment, approximately 2× baseline TLF), the Colilert sampling frequency at that site will be increased to daily for the duration of the event. This generates additional paired observations at high concentrations, which are the most important for demonstrating performance at and above the 126 CFU/100 mL threshold.
5.4 Source Water Characterization (per EPA §7.1.1)
EPA §7.1.1 of the Microbiological ATP Protocol (EPA-821-B-10-001) requires source water characterization data to be collected at the time of sample collection to identify potential interferences. The following ancillary parameters will be collected throughout the study and included in Appendix C of the study report:
| Parameter | Frequency | Method / Notes |
|---|---|---|
| pH | Every paired sample event | Handheld pH meter (±0.1 unit accuracy). Recorded on field conditions log alongside each Colilert grab. Important context for E. coli viability and Colilert enzyme substrate activity at temperature extremes. |
| Dissolved organic carbon (DOC) | Monthly baseline + during high-turbidity events | Grab sample collected concurrently with the Colilert sample. Preserved per SM 5310 (H2SO4 to pH <2, chilled), analyzed by certified laboratory. DOC is the primary driver of humic fluorescence interference in the TLF signal (see §11.1). Monthly baseline at all 6 sites; additional samples during elevated-turbidity and high-flow events. |
| Heterotrophic plate count (HPC) | Quarterly (all 6 sites) | Per SM 9215B (spread plate) or equivalent. Characterizes background non-target bacterial density. Required by EPA §7.1.1; provides context for interpreting TLF signal where high heterotrophic bacteria may contribute non-E. coli fluorescence. |
| Free and total chlorine residual | Every paired sample event at BC-13; quarterly at other sites | DPD colorimetric method (SM 4500-Cl G or equivalent USEPA-approved method). BC-13 is the site most likely to receive WWTP effluent with residual chlorine (§11.1). Elevated chlorine residual is documented in the field conditions log and included in the interference analysis for that site. |
6. Paired Data Collection Protocol
6.1 Definition of a Valid Paired Observation
A paired observation is defined as a Colilert result and a Lume sensor reading that meet all of the following criteria:
- The Colilert grab sample was collected within ±15 minutes of a Lume 15-minute measurement interval (i.e., the Lume reading is the single reading closest in time to sample collection, within one measurement interval)
- The Lume sensor was operating normally at the time of the reading (no QC flags applied; see Section 7)
- The Colilert sample was analyzed by the certified reference laboratory and has a valid COC
- The grab sample was collected within 2 meters of the sensor at the same depth and cross-section
- No documented maintenance activity (sensor cleaning, calibration, removal) was performed within 30 minutes of the reading
Pairing rationale: The ±15 minute window is more conservative than the ±30 minute window used in preliminary analyses. In a flowing stream, water conditions can change meaningfully over 30 minutes during events. A tighter pairing window ensures that the Lume reading and the Colilert sample represent the same parcel of water, which is the fundamental assumption underlying the equivalence claim.
6.2 Minimum Sample Size Requirements
| Metric | Minimum Target | Rationale |
|---|---|---|
| Total valid paired observations | ≥600 | 6 sites × ≥100 per site. Substantially exceeds Colilert’s original multi-lab validation sample count. Provides sufficient power for equivalence testing and holdout evaluation. |
| Paired observations per site | ≥80 | Minimum for site-level performance statistics. Lower-bound sites may have fewer observations due to site-specific access constraints. |
| Observations above 126 CFU/100 mL | ≥30 total (≥5 per site) | The Reg 93 threshold must be represented in the dataset. Classification performance at the threshold is meaningless without samples above it. Enhanced event sampling (Section 5.3) is designed to meet this requirement. |
| Observations above 235 CFU/100 mL | ≥10 total | The Reg 93 single-sample maximum. High-end performance must be characterized. |
| Seasonal coverage | All 4 seasons at all 6 sites | Spring runoff, summer baseflow, fall recession, and winter low-flow represent distinct hydrologic and biological conditions. All must be represented. |
| Continuous data completeness | ≥90% uptime per sensor per month | Demonstrates operational reliability of the method as deployed. |
6.3 Data Recording and Management
All paired observations will be stored in a centralized, version-controlled database. Each record will include: site ID, sample date and time (UTC), Lume TLF reading (ppb), turbidity (NTU), temperature (°C), raw Lume E. coli estimate (CFU/100 mL), Colilert MPN result (CFU/100 mL), data quality flags, field conditions log entries, laboratory COC reference, and collector/operator ID. Raw data will be archived in original format with all transformations and QA flag applications fully documented and reproducible. Minimum retention: 10 years.
7. QA/QC Requirements
7.1 Sensor Calibration and Verification
| QC Check | Frequency | Details |
|---|---|---|
| Field blank | Monthly (per site) | Submerge sensor in laboratory-grade DI water before and after each maintenance visit. TLF reading must be ≤[MDL to be determined]. Any reading above MDL triggers investigation and possible recalibration before data collection resumes. |
| Tryptophan standard verification | Monthly (per site) | Verify sensor response against a prepared tryptophan standard at a known concentration (e.g., 100 ppb). Acceptable response: within ±15% of the expected value. Out-of-range triggers recalibration. |
| Optical window cleaning | Bi-weekly | Clean optical window with approved cleaning solution. Verify post-cleaning blank reading before returning sensor to service. |
| Negative control (DI water blank) | Monthly (per site, with field blank) | In addition to the field blank confirming zero-TLF response, a separate DI water solution prepared with zero tryptophan (reagent-grade water, not field DI) is measured as an explicit negative control. Acceptable reading: below MDL. Documents that the sensor’s blank response is not an artifact of the DI water source used. Per EPA §5.2.12 (positive/negative controls analog). |
| Laboratory matrix spike analog | Quarterly | The Lume sensor is submerged in a prepared tryptophan standard solution of known concentration (e.g., 50 ppb, 200 ppb, 500 ppb) in the laboratory. Measured response compared against expected value. Acceptable recovery: within ±20% of expected. Provides a spike-recovery analog (per EPA §5.2.7) that verifies sensor linearity across the concentration range without requiring in-situ spiking of the creek. Results documented in calibration log. |
7.2 Data Quality Flags
| Flag | Trigger | Action |
|---|---|---|
| Fouling suspected | TLF drift >20% over 24 hours without corresponding Colilert change; post-cleaning blank >MDL | Flag all readings during fouling period. Exclude from paired dataset. Document in maintenance log. |
| Out-of-range turbidity | Turbidity >[upper operating limit NTU — to be determined from initial deployment] | Flag readings. High turbidity may saturate optical signal. Include flagged readings in sensitivity analysis but exclude from primary dataset. |
| Maintenance window | Any period during which sensor was removed, cleaned, or calibrated | Exclude all readings within ±30 minutes of maintenance activity from paired dataset. |
| Power interruption | Gap in data >30 minutes | Flag readings immediately before and after gap. Investigate cause. Exclude interpolated values from paired dataset. |
| Temperature out of range | Water temperature <0 °C or >35 °C | Flag readings. Outside characterized operating range. Include in sensitivity analysis; exclude from primary dataset. |
7.3 Reference Method QC
| QC Check | Frequency | Details |
|---|---|---|
| Laboratory field duplicate | Every 10th sample | Blind field duplicate submitted to certified laboratory. Acceptable lab RPD: ≤[to be confirmed with reference laboratory; typically ≤30% for environmental E. coli samples]. |
| Laboratory positive control | Per laboratory QA plan | Certified laboratory runs positive and negative controls per their accredited QA plan. Control results reported with each analytical batch. |
| Colilert holding time compliance | Every sample | Sample analyzed within 6 hours of collection. Samples exceeding holding time excluded from paired dataset. |
| Colilert method blank (per EPA §5.2.9) | Per laboratory QA plan (minimum: 1 per every 20 samples or 1 per week, whichever is more frequent) | Sterile reagent-grade buffered water analyzed exactly as a field sample (Colilert added, Quanti-Tray sealed, incubated at 35°C for 24 hours). Result must be negative for both total coliform and E. coli. Contaminated method blanks trigger rejection of all samples in the affected batch. Results reported with each analytical batch. |
7.4 Data Completeness Requirements
For a calendar month’s data to count toward the study dataset, the following minimum completeness thresholds must be met: (a) ≥90% of expected Lume 15-minute readings must be present and unflagged for each site; (b) ≥80% of planned Colilert grab samples must have valid paired Lume readings within the ±15 minute pairing window. Months not meeting these thresholds will be documented and excluded from the primary dataset; if exclusion brings total paired observations below the Section 6.2 minimums, the study will be extended accordingly.
7.5 Rationale for Ambient-Only Study Design (No Sample Spiking)
EPA §7.0 of the Microbiological ATP Protocol (EPA-821-B-10-001) describes sample spiking and stressing procedures for bacteriological method comparisons. EPA §7.2 explicitly states: “Depending on the matrix and the analyte of interest, it may not always be necessary to spike samples prior to analysis. Range finding analyses should be performed to assess the ambient concentration of target organism(s) in the matrix of interest to determine whether sample spiking will be useful.”
This study uses an ambient-only design for the following reasons, each of which should be reviewed and accepted by EPA Region 8 in the pre-submission consultation [Open Question A-3]:
- Physical impracticality of in-situ spiking. The Lume is a continuous in-situ sensor deployed in a flowing stream channel. There is no practicable method to spike a stream reach with known concentrations of E. coli under controlled conditions. The spiking procedures in EPA §7.2 are designed for discrete laboratory samples and batch culture; they are not transferable to continuous field deployment.
- Natural concentration range is sufficient. Boulder Creek naturally spans the full regulatory concentration range across seasons: <1 CFU/100 mL (winter low-flow at BC-Can) to >1,000 CFU/100 mL (spring runoff events at BC-13, BC-CU, and BC-55). The existing 856-observation global dataset from multiple sites confirms this coverage. Range-finding analyses conducted during sensor deployment will be documented in the study report.
- Ambient conditions are directly regulatory-relevant. The ATP claim is for compliance monitoring in ambient Boulder Creek surface water. Method performance on naturally occurring concentrations with real environmental interfering substances (humic matter, sediment, algae) is more directly applicable to the regulatory claim than performance on spiked laboratory samples, which may not reflect the chemical and biological matrix of the creek.
- Event-based enhanced sampling addresses high-concentration coverage. The enhanced sampling protocol (§5.3) ensures adequate paired observations at and above the 126 CFU/100 mL Reg 93 threshold by increasing Colilert grab frequency during storm events and spring runoff. The minimum requirement of ≥30 paired observations above 126 CFU/100 mL (§6.2) is designed to be met through this mechanism.
Contingency: If EPA Region 8 requires spiking data, a supplementary laboratory spike experiment can be added without altering the field study design: the Lume sensor is submerged in prepared E. coli suspensions of known concentration (enumerated by Colilert) and measured. This generates spike-recovery data at controlled concentrations. This is distinct from the quarterly matrix spike analog in §7.1, which uses tryptophan standards rather than live bacteria.
8. Model Development & Validation
8.1 Model Form
The prediction equation (Section 3.2) will be fit by ordinary least squares (OLS) regression to log10-transformed E. coli concentrations. Below-detection-limit Colilert results will be treated as one-half the detection limit for regression purposes (standard left-censored data convention). The model form is additive with three predictors — TLF, turbidity, and temperature — and no site-specific or temporal covariates. No interaction terms will be included unless pre-specified below and justified by exploratory analysis on the training partition only.
8.2 Training / Holdout Split — Pre-Specified Protocol
Critical: The train/test split methodology described below is pre-specified prior to data collection and may not be modified after data collection begins. This prevents any risk of split selection that optimizes apparent model performance. EPA will be provided this document before the study start date as evidence that the split was pre-defined.
| Partition | Data Period | Purpose |
|---|---|---|
| Training set | Months 1–9 of the study (all 6 sites) | Fit the OLS regression coefficients (β0–β3). Model is built entirely on this partition. The coefficients are fixed after training and not updated. |
| Holdout test set | Months 10–12 of the study (all 6 sites) | Independent evaluation of the fixed model. All primary performance statistics (see Section 9) are calculated on this partition only. The holdout period is chosen to include spring runoff (if the study starts in January, this is the second spring; otherwise adjusted accordingly). |
A temporal split is used rather than a random split because it mimics real-world deployment: the model is trained on historical data and applied forward in time. This is more conservative than a random split and eliminates any possibility of the training data “seeing” future conditions.
8.3 Model Acceptance Criteria (Training Phase)
Before the model is locked for holdout evaluation, it must meet the following minimum acceptance criteria on the training set:
- R² ≥ 0.50 (log10-transformed concentrations)
- All three predictors (TLF, turbidity, temperature) must have non-zero coefficients with the expected sign (TLF positive; turbidity positive or negative depending on correlation with organic matter; temperature positive)
- No predictor with a statistically significant coefficient of opposite sign to that expected (would indicate collinearity or overfitting)
- Residuals approximately normally distributed (Shapiro-Wilk test, p > 0.05) and homoscedastic across the fitted range
If these criteria are not met on the training set, the model form will be reviewed (e.g., log10 transformation of predictors, addition of one pre-specified interaction term: TLF × turbidity) with notification to EPA Region 8. The holdout set will not be examined until the model form is finalized.
8.4 Model Coefficient Reporting
The final fitted coefficients (β0, β1, β2, β3), standard errors, 95% confidence intervals, and p-values will be reported in the ATP application. The complete prediction equation with numerical coefficients will be included in the method documentation (Section 3.2) such that any laboratory can implement the method using a standard spreadsheet without proprietary software.
9. Statistical Analysis Plan
Pre-specified SAP: All statistical tests and acceptance thresholds in this section are pre-specified. No additional tests will be added and no thresholds will be modified after data collection begins. This section is the binding statistical analysis plan for the ATP application.
9.1 Primary Endpoint — Continuous Equivalence
Evaluated on the holdout test set only (Months 10–12). All analyses on log10-transformed values unless otherwise specified.
| Test | Method | Pre-Specified Threshold |
|---|---|---|
| Pearson correlation | r between log10(Lume) and log10(Colilert) on holdout set | r ≥ 0.80 [proposed; to be confirmed with EPA Region 8] |
| Coefficient of determination | R² on holdout set | R² ≥ 0.65 [proposed] |
| Mean absolute percentage error | MAPE in log-transformed space on holdout set | MAPE ≤ 15% [proposed] |
| Systematic bias | Geometric mean ratio: median(Lume/Colilert) on holdout set. Test for bias using paired t-test on log differences. | Geometric mean ratio between 0.67 and 1.50 (i.e., within a factor of 1.5×); paired t-test p > 0.05 for no significant bias [proposed] |
| Limits of agreement | Bland-Altman analysis: mean difference ± 1.96 SD of differences (log10 scale) | 95% of differences within ±1.0 log10 unit (factor of 10×); reported for transparency [proposed] |
| Formal equivalence test | Two one-sided t-tests (TOST) on log10 differences. Equivalence margin: ±0.5 log10 units (≈ factor of 3×; comparable to Colilert’s own inter-lab variability of 20–30%) | Both one-sided tests significant at p < 0.05 within ±0.5 log10 equivalence margin [proposed] |
9.2 Secondary Endpoint — Classification at Reg 93 Threshold
Binary classification performance at the Regulation 93 geometric mean threshold of 126 CFU/100 mL, evaluated on the holdout test set.
| Metric | Definition | Pre-Specified Threshold |
|---|---|---|
| Sensitivity (true positive rate) | P(Lume ≥126 | Colilert ≥126) | ≥0.85 [proposed — protective; missed exceedances are the higher public health risk] |
| Specificity (true negative rate) | P(Lume <126 | Colilert <126) | ≥0.80 [proposed] |
| Cohen’s kappa | Agreement beyond chance on binary classification | ≥0.70 [proposed — “substantial agreement”] |
| Overall accuracy | (TP + TN) / N | ≥0.85 [proposed] |
| Area under ROC curve | AUC from continuous Lume predictions vs. binary Colilert threshold outcome | ≥0.80 [proposed] |
| False negative rate | P(Lume <126 | Colilert ≥126) | ≤0.15 [proposed — complement of sensitivity; protects against under-reporting exceedances] |
Classification performance will also be evaluated at the Reg 93 single-sample maximum (235 CFU/100 mL) and reported as supplementary analyses. Confusion matrices for each site and season will be reported in the full comparability report.
9.3 Precision Analysis
| Analysis | Data Source | Metric |
|---|---|---|
| Temporal within-sensor precision | Readings during stable baseflow periods (identified post-hoc by <5% CV over 24 hours) | Coefficient of variation (CV) of consecutive readings; characterizes instrument noise floor vs. real environmental signal. |
| Reference method precision | Field duplicate Colilert samples (every 10th sample) | RPD between field duplicates. Documents Colilert’s own measurement uncertainty at Boulder Creek, providing context for the equivalence margin. |
9.4 Stratified Performance Analyses
All primary and secondary endpoint analyses will be repeated for the following stratifications, reported as supplementary analyses in the comparability report:
- By site (BC-Can, BC-Eben, BC-13, BC-CU, BC-30, BC-55): identifies site-specific performance differences or outliers
- By season (spring/summer/fall/winter): documents seasonal performance; spring runoff reported separately
- By flow condition (baseflow / rising limb / peak / recession): defined using USGS stream gauge data from the nearest Boulder Creek gauge
- By concentration range (<10 / 10–126 / 126–235 / >235 CFU/100 mL): performance at each regulatory decision threshold
- By turbidity range (<10 / 10–50 / >50 NTU): documents turbidity interference effects
9.5 Statistical Software and Reproducibility
All statistical analyses will be performed in R (version ≥4.3) or Python (scikit-learn, scipy, statsmodels). Analysis code will be version-controlled and provided to EPA Region 8 upon request. All results will be reproducible from the archived raw data using the provided code. No analyses will be performed in Excel alone; Excel may be used for data entry and visualization only.
9.6 EPA-Required Comparability Statistics (EPA-821-B-10-001 §8.5)
In addition to the primary and secondary endpoints in §9.1–9.2, the following statistical tests are explicitly specified in EPA-821-B-10-001 (§8.5) and its decision flowcharts (Figures 8-1 and 8-2). These tests constitute the formal comparability determination framework for the ATP application. All will be calculated on the holdout test set and reported in the comparability report.
Open action — statistical test equivalency: The modern statistical tests in §9.1–9.2 (TOST, Bland–Altman, AUC) are more informative than the 2010 EPA-specified tests but use a different framework. EPA Region 8 should confirm whether the tests below are strictly required in addition to §9.1–9.2, or whether the pre-specified SAP satisfies the comparability determination under §8.5. [See Open Question A-4]
| EPA-Specified Test | EPA Reference | Application to This Study |
|---|---|---|
| Normality test (Shapiro–Wilk or Kolmogorov–Smirnov) | §8.5.2.1 | Applied to log10-transformed Lume and Colilert values separately. Determines whether parametric (F-test / ANOVA) or non-parametric (Conover / WMW) tests are used in the precision and recovery evaluations below. Already required in §8.3 for model acceptance; repeated here on the holdout set. |
| F-test or Conover Squared-Rank test for precision | §8.5.2.2, Fig. 8-1 | Compares variance (spread of replicate readings) between Lume and Colilert. If data are normally distributed (per normality test above), F-test is used; otherwise the non-parametric Conover Squared-Rank test is used. Evaluated separately for each Boulder Creek site (to detect matrix effects) and then pooled across all 6 sites. Failure condition: Lume variance significantly greater than Colilert variance at the majority of sites. |
| ANOVA or Wilcoxon–Mann–Whitney (WMW) test for recovery | §8.5.2.3, Fig. 8-2 | Tests whether mean recovery (the ratio of Lume estimate to Colilert result) differs significantly between methods overall and by site. If the normality and precision tests pass, ANOVA is used with a method-by-site interaction term tested first. If the interaction is significant, methods are compared separately per site and the proportion of sites where recovery is not significantly lower is reported (≥80% of sites required). If normality fails, a single WMW test is run disregarding matrix stratification. |
| Chi-square test for false positive / false negative rate comparison | §8.5.1.1 | Constructs a 2×2 contingency table (Lume positive/negative × Colilert positive/negative relative to the 126 CFU/100 mL threshold) and uses the chi-square test to determine whether the false positive rate and false negative rate differ significantly between Lume and Colilert. Run over all holdout observations pooled (or separately by site if the Breslow–Day test below is significant). These tables and rates are already calculated for §9.2; this test adds the formal chi-square significance evaluation. |
| Breslow–Day test for method-by-site interaction | §8.5.1.2 | Tests whether the method effect on false positive and false negative rates is consistent across the 6 Boulder Creek sites. A significant Breslow–Day result means the method performs differently at different sites in terms of misclassification rate, which would require site-specific chi-square tests rather than a single pooled test. An interaction here would be informative about which sites present the most challenging conditions for the Lume. |
The decision logic for these tests follows Figures 8-1 and 8-2 of EPA-821-B-10-001 exactly. The comparability conclusion will be stated per the EPA protocol’s criteria: the Lume is acceptable if it is not significantly less precise than Colilert (for ≥80% of sites), not significantly lower in recovery, and does not have significantly higher false positive or false negative rates than Colilert. The TOST and AUC analyses in §9.1–9.2 provide supplementary evidence and may exceed these minimum criteria.
10. Method Detection Limit
10.1 TLF Detection Limit
The instrument detection limit for TLF will be determined per 40 CFR Part 136 Appendix B: minimum 7 replicate measurements of a tryptophan standard near the expected detection limit, run in both laboratory (controlled tryptophan solution in DI water) and field (low-concentration upstream reference site BC-Can) conditions. MDL = t-value (α = 0.01, n−1 degrees of freedom) × standard deviation of replicates.
10.2 E. coli MDL (Translated)
The TLF MDL will be translated to an E. coli MDL in CFU/100 mL by applying the fitted prediction equation. The method reporting limit (MRL) will be set at 3× the MDL, consistent with standard analytical chemistry practice. Colilert results below the Lume MRL will be reported as <MRL in the paired dataset and treated as left-censored in regression analyses.
10.3 Relationship to Regulatory Thresholds
The MDL and MRL will be compared against the Reg 93 thresholds (126 CFU/100 mL geometric mean; 235 CFU/100 mL single-sample maximum) and the existing literature MDL of approximately 10 CFU/100 mL for TLF-based methods. The method must demonstrate that the MRL is substantially below the regulatory threshold to support the compliance monitoring claim.
11. Interferences & Limitations
11.1 Known Interferences
| Interference | Mechanism | Mitigation in This Method |
|---|---|---|
| Dissolved organic carbon (DOC) / humic-like fluorescence | Humic and fulvic acids can fluoresce at wavelengths overlapping the TLF signal, causing false positive E. coli estimates | Partially mitigated by turbidity correction in the prediction equation. The magnitude of humic interference at Boulder Creek is characterized during the study by comparing TLF at low-E. coli / high-DOC conditions (e.g., late fall leaf litter input) against Colilert results. |
| High turbidity | Very high turbidity (>[threshold NTU]) can attenuate fluorescence signal (inner filter effect) or scatter light into detector | Turbidity is included as a model predictor. Readings above the identified interference threshold are flagged (Section 7.2). Performance at high turbidity conditions is characterized separately. |
| Temperature | TLF signal intensity is temperature-dependent; fluorescence efficiency decreases with increasing temperature | Temperature included as a model predictor. Study spans all four seasons, ensuring temperature effects are represented in the calibration dataset. |
| Algal fluorescence | Phytoplankton can produce fluorescence signals in the TLF spectral region, particularly during algal blooms | Algal events documented in field conditions log. Performance during identified algal bloom periods analyzed separately. Boulder Creek turbid spring runoff typically limits algal growth. |
| Biofouling | Biological growth on the optical window attenuates signal over time | Bi-weekly cleaning protocol (Section 7.1); fouling detection via daily drift monitoring (Section 7.2); maintenance rotation minimizes gaps. |
| Chlorinated effluent | WWTP effluent may carry chlorine residual that partially inactivates E. coli but does not immediately destroy TLF signal, causing over-prediction | Site BC-13 may receive residual-chlorinated effluent intermittently. Performance at BC-13 analyzed separately; any systematic bias at this site documented and disclosed. |
11.2 Method Limitations
- The prediction equation is calibrated to Boulder Creek water chemistry. Performance at sites with substantially different DOC, humic content, or conductivity is not validated by this study and should not be assumed without site-specific evaluation.
- The method measures total TLF, not E. coli specifically. The prediction equation uses turbidity and temperature to correct for non-E. coli TLF sources, but cannot distinguish E. coli fluorescence from all potential interfering substances.
- The method does not distinguish viable from non-viable E. coli. Post-chlorination over-prediction (documented at BC-13) reflects this limitation.
- Sensor performance during ice formation or under ice cover has not been characterized. Data collected when water temperature is <1 °C should be considered provisional.
11.3 Verification of Results — Applicability Note (EPA §7.6)
EPA §7.6 of the Microbiological ATP Protocol requires biochemical confirmation of positive and negative results using tests such as LTB/BGLB and LTB/EC-MUG (§7.6.1.3). This requirement applies to culture-based methods where isolated colonies must be biochemically confirmed as the target organism.
This requirement is not applicable to the Lume method. The Lume does not produce cultures or isolated colonies requiring identification. Its output is a continuous fluorescence-based E. coli concentration estimate. Verification of that estimate is provided by the paired IDEXX Colilert reference method result, which is itself biochemically specific for E. coli via MUG (4-methylumbelliferyl-β-D-glucuronide) fluorescence hydrolysis by β-glucuronidase. The Colilert result serves as the “independent standard” in the sense of EPA Table 8-1 — it is the ground truth against which Lume readings are evaluated. No further biochemical verification step is applicable to an in-situ fluorescence measurement that does not generate discrete biological cultures.
This determination will be stated explicitly in the method documentation (Section 9.8 of the study report, per EPA §3.0 Section 17 method format).
12. Deliverables & Schedule
12.1 Pre-Study Prerequisites
The study clock does not start until all prerequisites are met. Paired data collected before these conditions are satisfied does not count toward the regulatory dataset.
- [Priority 1 — Blocking] EPA Region 8 confirmation that Virridy’s role as method developer and sensor operator satisfies the independence requirements for a 40 CFR 136.5 limited-use application, and written agreement on the data compilation independence structure (§4.4 and G-1). This is the single highest-risk open item. The study clock does not start until this is resolved in writing. [See Open Questions A-1 and G-1]
- All 6 Lume sensors deployed and transmitting at Boulder Creek TMDL monitoring sites
- Certified reference laboratory confirmed and under contract for increased Colilert sampling frequency
- Colilert version (standard / Colilert) confirmed and consistent with Boulder’s existing Reg 93 program
- This study design document submitted to and acknowledged by EPA Region 8 (study design pre-registration)
- Statistical analysis plan (Section 9) accepted by EPA Region 8 in writing, including pre-specified thresholds and confirmation on §9.6 test requirements [See Open Question A-4]
- Draft 17-section EMMC method document (per EPA §3.0) submitted to EPA Region 8 for initial review. Sections 1–12 can be substantially drafted before the study begins; Sections 13 (Method Performance) and 17 (Validation Data) are completed post-study.
- Written rationale for ambient-only study design (no spiking, §7.5) reviewed and accepted by EPA Region 8 [See Open Question A-3]
- Chain-of-custody and field conditions logging protocols in place and tested
12.2 Study Timeline
Confirm certified reference lab; confirm Colilert version; deploy all 6 sensors; submit study design to EPA Region 8; receive written acknowledgment; define study Day 0.
Continuous Lume data + 3×/week Colilert grabs at all 6 sites. Monthly QC checks. Event-based enhanced sampling. Quarterly interim reports to EPA Region 8 and CDPHE. All data to training partition.
Fit OLS regression to training data. Evaluate model acceptance criteria (Section 8.3). Lock coefficients. Report locked equation to EPA Region 8 before holdout evaluation begins. Holdout set remains unseen until this step is complete.
Continue Colilert + Lume paired collection. At Month 12, evaluate locked model on holdout set against pre-specified SAP thresholds (Section 9). Holdout evaluation is blinded — model coefficients are fixed, analysis plan is fixed.
Compile full statistical analysis report on holdout set. Draft Boulder Creek site-specific comparability report. Prepare complete 40 CFR 136.5 application package: method documentation, comparability report, QA/QC records, SAP results.
City of Boulder submits limited-use ATP application to EPA Region 8 Administrator.
12.3 Deliverables
| Deliverable | Recipient | Timing |
|---|---|---|
| Study design document (this document) | EPA Region 8, CDPHE, City of Boulder | Pre-study |
| Draft 17-section EMMC method document (Sections 1–12 pre-data; Sections 13 and 17 post-data) | EPA Region 8 | Pre-study (draft); Month 15 (final) |
| Quarterly interim data reports | EPA Region 8, CDPHE | Months 3, 6, 9 |
| Model fitting report (locked coefficients) | EPA Region 8 | Month 9 |
| Full statistical analysis report (holdout) | EPA Region 8, CDPHE, City of Boulder | Month 13 |
| Boulder Creek comparability report (organized per EPA §9.0 study report structure) | EPA Region 8, CDPHE | Month 14 |
| Complete method documentation (EPA §3.0, 17-section format, final) | EPA Region 8 | Month 15 |
| 40 CFR 136.5 application package | EPA Region 8 | Month 16 |
12.4 Method Document Preparation Status (17-Section EMMC Format)
EPA §3.0 requires a method document in the 17-section EMMC format. The table below tracks which sections can be drafted before the study (from existing data and this study design) vs. which require post-study performance data.
| # | EMMC Section | Status | Source |
|---|---|---|---|
| 1 | Scope and Application | Draftable now | §3.4 of this document |
| 2 | Summary of Method | Draftable now | §3.1 of this document |
| 3 | Definitions | Needs drafting | Define TLF, PPB-TE, MPN, paired observation, Reg 93 threshold, etc. |
| 4 | Interferences | Draftable now | §11 of this document |
| 5 | Safety | Needs drafting | Electrical safety (submersible sensor), UV light source handling, E. coli-positive sample handling |
| 6 | Equipment and Supplies | Draftable now | §3.3 of this document |
| 7 | Reagents and Standards | Partial | Tryptophan standard preparation; DI water specification; no reagents for sensor itself |
| 8 | Sample Collection, Preservation, Storage | Draftable now | §5.2 and §17 of this document |
| 9 | Quality Control | Draftable now | §7 of this document |
| 10 | Calibration and Standardization | Draftable now | §7.1 of this document |
| 11 | Procedure | Draftable now | §14.2 of this document |
| 12 | Data Analysis and Calculations | Draftable now | §8 and §9 of this document |
| 13 | Method Performance | Post-study | Holdout evaluation results (R², MAPE, sensitivity, specificity, MDL) |
| 14 | Pollution Prevention | Needs drafting | Minimal waste generation; tryptophan standard disposal; no hazardous reagents |
| 15 | Waste Management | Needs drafting | Colilert waste disposal per laboratory SOP; sensor battery disposal |
| 16 | References | Needs compiling | All cited methods, standards, and publications |
| 17 | Tables, Diagrams, Flowcharts, and Validation Data | Post-study | Final paired dataset, performance tables, confusion matrices, regression equation with coefficients |
13. Regulatory Engagement
Ongoing coordination with CDPHE and EPA Region 8 throughout the project. Front-loaded in Q1–Q2 to align study design with regulatory expectations before committing to the full field deployment. The regulatory context is Colorado Regulation 93—CDPHE’s 303(d) program for impaired waters—under which Boulder Creek carries E. coli impairment. The initial ATP application will be scoped to support Reg 93 compliance monitoring at Boulder’s six monitoring points.
13.1 CDPHE Initial Engagement
| Task | Timing | Details |
|---|---|---|
| Introductory briefing to CDPHE Water Quality Control Division | Month 1 | Present the Lume technology, existing Boulder Creek validation data, and ATP intent. Identify CDPHE staff lead for ongoing coordination. |
| Review Colorado Regulation 93 / 303(d) requirements for Boulder Creek | Month 1–2 | Confirm the Reg 93 E. coli impairment listing and TMDL threshold (126 CFU/100 mL geometric mean) for Boulder Creek. Identify Boulder’s existing monitoring obligations at the 6 monitoring points and determine how continuous Lume data maps to Reg 93 compliance reporting requirements. |
| CDPHE feedback on study design | Month 2–3 | Share proposed data collection protocol (see Section 17). Get CDPHE input on sample counts, seasonal coverage, and any Colorado-specific requirements. |
13.2 EPA Region 8 Engagement
| Task | Timing | Details |
|---|---|---|
| Pre-submission meeting request to EPA Region 8 | Month 1 | Contact EPA Region 8 Water Technical Unit in Denver. Request formal pre-submission consultation per ATP program guidance. |
| Prepare briefing package | Month 1–2 | Package includes: technology summary, published validation data, proposed method scope (E. coli in freshwater and enterococci in marine water via TLF-based linear regression for recreational water monitoring), proposed study design, draft QA/QC framework, and specific questions for EPA. |
| Pre-submission meeting | Month 3–4 | Formal meeting with EPA Region 8. Present briefing package, receive guidance on study design, statistical requirements, and any Region 8-specific considerations. |
| Parallel consultation with EPA OST (headquarters) | Month 3–4 | Contact Lemuel Walker at EPA Office of Science and Technology (walker.lemuel@epa.gov, 202-566-1077). OST handles nationwide ATP review; early engagement ensures Region 8 and OST expectations are aligned. |
13.3 Key Questions for Regulators
Continuous vs. Discrete Output
How should continuous sensor readings map to discrete compliance determinations? Options: instantaneous readings at defined intervals, rolling averages, time-weighted composites. This is the fundamental framing question.
ML Model Classification
Does EPA consider the ML component a “black box” concern? How should model versioning, retraining, and site-specific calibration be documented in the method?
New Method vs. Modified Method
Confirm that the Lume method is classified as a “new method” (new determinative technique) rather than a “modified method.” This determines which validation protocol applies.
Existing Data Applicability
Can the existing Boulder Creek paired dataset supplement the formal validation study, or must all regulatory data be collected under the finalized protocol?
Multi-Lab Requirement for Limited-Use
For limited-use approval at Boulder’s sites, is single-operator (Virridy + Boulder) data sufficient, or does EPA Region 8 require independent laboratory operation even for facility-specific approval?
Colorado-Specific Standards
Does CDPHE impose additional requirements beyond federal ATP? Are there Colorado water quality standards for E. coli that differ from federal thresholds?
A-1 — Independence / Conflict of Interest
EPA §5.0 states: “A laboratory with a vested interest in the method, apparatus, reagents, media, or associated kits may not participate in the side-by-side comparison study.” EPA §6.1.1 states: “the laboratory selected cannot be the method developer’s laboratory and cannot be affiliated with the method developer.”
Both provisions use the word “laboratory” — not “operator,” “developer,” or “organization.” In this study, the entity performing laboratory analyses is the independent, state-certified Colilert reference laboratory. Virridy does not perform laboratory analyses; the Lume sensor runs autonomously in the field and is analogous to a USGS stream gauge or a continuous DO probe — a piece of monitoring infrastructure, not a laboratory.
Furthermore, §5.0’s independence requirement applies to studies “in which only one laboratory is performing analyses” — which here is the Colilert lab, which has no affiliation with Virridy.
Question for EPA Region 8: Given that the protocol’s conflict-of-interest language was written for culture-based laboratory methods, does EPA interpret “laboratory” to encompass a field-deployed autonomous sensor operated by the method developer? Or does the independence requirement apply only to the laboratory conducting the reference method analyses — which is independently certified and unaffiliated? This interpretive question should be resolved in writing before study data collection begins. [Prerequisite — blocking]
A-2 — Matrix Diversity for Limited-Use
EPA Table 6-1 specifies 10 geographically diverse matrices for a nationwide side-by-side comparability study. For a 40 CFR 136.5 limited-use application covering 6 sites on a single waterbody (Boulder Creek, ~10-mile corridor): what is the minimum matrix diversity requirement? Does the site-to-site variability in the Boulder Creek dataset (canyon mouth to watershed outlet, low-concentration to high-event sites) satisfy any matrix diversity requirement, or does EPA require data from multiple distinct waterbodies?
A-3 — Spiking Waiver Rationale
EPA §7.2 notes that spiking may not be necessary when ambient concentrations cover the range of interest. The Lume is an in-situ continuous sensor for which creek-scale spiking is physically impractical. Boulder Creek naturally spans <1–>1,000 CFU/100 mL across seasons. Does EPA Region 8 accept the written no-spiking rationale in §7.5, or is a supplementary laboratory spike experiment (sensor submerged in prepared E. coli suspensions) required as part of the study protocol?
A-4 — Statistical Test Equivalency
The SAP in §9.1–9.2 uses TOST equivalence testing, Bland–Altman analysis, and AUC, which are more modern and arguably more informative than the F-test, WMW test, chi-square, and Breslow–Day tests specified in EPA-821-B-10-001 §8.5 (2010). The §9.6 tests have been added to the SAP to explicitly satisfy the EPA protocol. Does EPA Region 8 confirm that: (a) the §9.6 tests are required in the study report; and (b) the §9.1–9.2 supplementary analyses are acceptable and will be considered in the comparability determination?
G-1 — Data Independence Plan
EPA §4.4 requires that data compilation not be performed by the analysts conducting sample analyses. Virridy operates the sensors and performs data management. Proposed mitigation: (a) City of Boulder Utilities independently verifies and certifies all statistical outputs before submission; (b) an independent third-party statistician reviews SAP execution on the holdout dataset. Is this structure acceptable to EPA Region 8, or is a more formal separation of roles required for 136.5 approval?
14. Field Study Workplan
The core data collection effort. Deploy Lume sensors at all 6 of the City of Boulder Utilities’ recreational water monitoring locations on Boulder Creek. Collect continuous TLF data paired with Boulder’s existing weekly Colilert E. coli grab samples.
14.1 Site Preparation & Deployment
| Task | Timing | Details |
|---|---|---|
| Formalize MOU with City of Boulder Utilities | Month 1 | Define roles, data sharing, access to compliance sites, sampling coordination, and liability. Boulder continues their existing Colilert program unchanged; Virridy adds sensor deployment. |
| Site survey of all 6 compliance locations | Month 1–2 | Visit each site. Assess: mounting infrastructure, power availability, cellular/data connectivity, flow characteristics, security, and seasonal access constraints (ice, high water). |
| Sensor procurement & pre-deployment calibration | Month 1–2 | Prepare 7 Lume sensors (6 primary + 1 spare for maintenance). Factory calibration with tryptophan standards. Document sensor serial numbers, firmware versions, and initial calibration data. |
| Install sensors at all 6 sites | Month 2–3 | Staggered installation over 2–4 weeks. At each site: mount sensor, verify data transmission, confirm measurement interval (15-min default), photograph installation, record GPS coordinates. Begin continuous logging. |
| Validate data pipeline | Month 3 | Confirm all 6 sensors are transmitting reliably to the Lume data platform. Set up automated data quality alerts (signal dropout, out-of-range readings, battery/power status). |
14.2 Ongoing Data Collection
| Task | Frequency | Details |
|---|---|---|
| Continuous Lume TLF measurement | Every 15 min | Automated. Each reading: TLF intensity (ppb), turbidity (NTU), temperature (°C), ML model E. coli estimate (CFU/100 mL), categorical classification, model confidence score. |
| Paired Colilert grab sample (Boulder’s program) | Weekly | Boulder staff collect grab samples at each recreational water monitoring site per their existing reporting schedule. Log exact sample collection time to match against nearest Lume reading. |
| Enhanced sampling during events | As needed | Increase Colilert grab frequency during high-flow events, storm runoff, and any known upstream contamination events. Target: 2–3× weekly during events to increase paired observations at high concentrations. |
| Field QC: blank verification | Monthly | Submerge sensor in DI water before and after each maintenance visit. Verify TLF reading is below MDL. Document any drift. |
| Sensor maintenance | Bi-weekly | Clean optical window, check mounting integrity, download local backup data, verify calibration against tryptophan standard. Log all maintenance activities with timestamps. |
| Data review & QA flagging | Weekly | Review incoming data for each site. Flag and document: fouling events, power interruptions, sensor malfunctions, outliers. Apply data quality flags per QA/QC protocol. |
14.3 Data Collection Targets
| Metric | Target | Rationale |
|---|---|---|
| Study duration | 12–18 months minimum | Must span all four seasons to capture temperature, flow, and biological variability. Two full spring runoff cycles preferred. |
| Paired observations per site | ≥52 (weekly × 12 months) | 52 is the baseline from weekly Colilert. Enhanced event sampling should push this to 70–100+ per site. |
| Total paired observations (all sites) | ≥400–600 | 6 sites × 70–100 each. Substantially exceeds typical EPA multi-lab requirements. |
| Concentration range | <1 to >1,000 CFU/100 mL | Must cover the full range relevant to recreational water quality criteria. |
| Continuous data completeness | ≥90% uptime per sensor | Account for maintenance windows, fouling, and equipment failures. One spare sensor provides coverage during repairs. |
| Duplicate sensor precision data | ≥12 one-week deployments | One per month across different sites = reproducibility data across conditions. |
15. Limited-Use ATP Application
Facility-specific ATP approval under 40 CFR 136.5, submitted by the City of Boulder to EPA Region 8 for Colorado Regulation 93 / 303(d) compliance monitoring on Boulder Creek. This is the near-term regulatory milestone: a site-specific approval that allows Boulder to use Lume data in fulfillment of its Reg 93 impaired-waterbody monitoring obligations, with the TMDL-driven threshold of 126 CFU/100 mL geometric mean as the key compliance metric.
15.1 Application Preparation
| Task | Timing | Details |
|---|---|---|
| Compile Boulder Creek paired dataset | Month 13–14 | After 12 months of data collection. Merge Lume continuous data with Boulder’s Colilert results. Apply QA/QC flags. Generate site-specific performance statistics. |
| Prepare site-specific comparability report | Month 14–15 | Statistical analysis from Workstream 4 (see ATP Overview) tailored to Boulder’s specific sites and permit conditions. Include seasonal breakdown, flow-condition breakdown, and performance at Boulder’s permit limit thresholds. |
| Draft limited-use application | Month 15–16 | Application per 40 CFR 136.5 requirements. Includes: method description, comparability data, QA/QC protocol, proposed reporting framework. |
| City of Boulder internal review & approval | Month 16–17 | Boulder Utilities reviews and approves the application. Boulder is the applicant under 136.5—Virridy provides technical support. |
| CDPHE pre-submission review | Month 16–17 | Share draft with CDPHE for informal feedback before formal submission to EPA Region 8. CDPHE support strengthens the application. |
15.2 Submission & Review
| Task | Timing | Details |
|---|---|---|
| Submit to EPA Region 8 | Month 17 | City of Boulder submits the limited-use ATP application to the EPA Region 8 Administrator. |
| Respond to EPA information requests | Month 17–20 | Anticipate questions and requests for additional data. Have supplementary analyses pre-computed. |
| EPA Region 8 approval | Month 18–21 | If approved, EPA issues a limited-use ATP approval letter. Boulder can begin using Lume data for recreational water quality monitoring on Boulder Creek. |
15.3 Post-Approval Operations
| Task | Details |
|---|---|
| Compliance reporting protocol | Define how Boulder reports Lume data for recreational water quality monitoring. Agree on: averaging period, reporting frequency, data completeness thresholds, and backup procedure (revert to Colilert if sensor data completeness falls below threshold). |
| Parallel monitoring period | Continue Colilert sampling in parallel with Lume for an initial period (6–12 months) after limited-use approval. Builds confidence and provides ongoing comparability data. |
| Performance monitoring & reporting | Quarterly performance reports to EPA Region 8 and CDPHE during the parallel monitoring period. Document ongoing accuracy, precision, and any issues. |
16. Next Steps: Colorado State-Wide Expansion Track A — Phase 2
After the Boulder Creek limited-use ATP is approved, expand the Lume’s recognized use to additional Colorado waterbodies listed under Reg 93 / 303(d). The goal is to build a body of multi-site, multi-condition evidence that supports both CDPHE statewide recognition and the nationwide freshwater ATP submission.
16.1 Additional Colorado Limited-Use ATPs
| Task | Timing | Details |
|---|---|---|
| Identify target Colorado watersheds | Month 18–20 | Select 2–4 additional Colorado 303(d)-listed waterbodies with active E. coli impairments and willing utility/agency partners. Geographic and hydrologic diversity strengthens the broader ATP submission (e.g., mountain streams, plains rivers, reservoirs). |
| Submit additional 40 CFR 136.5 applications | Month 21–24 | Each new site submits its own facility-specific ATP application to EPA Region 8, using the Boulder Creek method documentation and comparability report as the technical foundation. |
16.2 CDPHE State Method Recognition
| Task | Timing | Details |
|---|---|---|
| Briefing to CDPHE on multi-site performance | Month 22–24 | Present aggregated Colorado performance data across all approved sites. Propose CDPHE formally recognize the Lume method for E. coli monitoring on Colorado Reg 93 / 303(d) waterbodies. |
| CDPHE state guidance or policy adoption | Month 24+ | Work with CDPHE to issue guidance that streamlines future Reg 93 monitoring approvals for the Lume across Colorado. Provides a state-level endorsement that supports the nationwide EPA submission. |
17. Data Collection Protocol
Supplementary protocol details for paired data collection at each Boulder Creek site, covering elements not fully addressed in Sections 5–7 above. Designed to generate EPA-quality comparability data.
17.1 Colilert SOP Details
| Parameter | Requirement |
|---|---|
| Colilert formulation | IDEXX Colilert (24-hour incubation at 35°C), matching Boulder’s existing Reg 93 compliance program. Must use consistent formulation throughout study. See Open Question B-2. |
| Quantification method | Quanti-Tray/2000 MPN per IDEXX protocol and 40 CFR Part 136. |
| Incubation control | Incubator temperature verified daily during all active incubation periods. Temperature logs retained as part of chain of custody. |
| Laboratory QC controls | Positive control (E. coli ATCC 25922 or equivalent) and negative control (sterile Colilert reagent in DI water) run with each analytical batch. Results reported with each batch. |
| MPN calculation | Per IDEXX Quanti-Tray/2000 MPN table. Below-detection results reported as <1 MPN/100 mL. Saturated Quanti-Tray (all wells positive) reported as >2419.6 MPN/100 mL. |
17.2 Certified Laboratory Requirements
Critical prerequisite: All Colilert reference samples used in the regulatory validation dataset must be analyzed by a laboratory holding current state certification under the Colorado Laboratory Certification Program (CLCP) for E. coli analysis. Uncertified reference data is grounds for application rejection.
| Requirement | Details |
|---|---|
| CLCP certification scope | Laboratory must hold current CLCP certification specifically for E. coli in surface water by the Colilert / Quanti-Tray method. Verify certification is active at study start and before each annual renewal period. |
| PT sample performance | Laboratory must demonstrate acceptable performance on proficiency testing (PT) samples for E. coli within 12 months prior to study start. PT results requested as part of laboratory selection. |
| Sample turnaround | Laboratory must commit to returning results within 24 hours of receipt to allow timely data management. Results provided as a digital report with sample IDs, collection times, and MPN values. |
17.3 10-Year Data Archival
| Category | Retention Requirement |
|---|---|
| Raw sensor data | All raw TLF, turbidity, temperature readings archived in original format. No deletion or overwrite. Minimum 10-year retention from study end date. |
| Laboratory Colilert reports | All original laboratory reports retained. Digital copies stored in version-controlled database. Physical copies retained by the City of Boulder per their existing record-keeping requirements. |
| Chain of custody records | All COC forms (paper originals and digital scans) retained for minimum 10 years. COC records are EPA records and may not be destroyed without agency notification. |
| QA/QC records | Calibration logs, maintenance logs, field blank results, equipment service records. 10-year minimum retention. |
| Analysis code and models | All statistical analysis scripts, model coefficient files, and data processing code version-controlled and archived. Must be sufficient to fully reproduce all results in the ATP application from raw data. |
17.4 Chain of Custody Specifics
| COC Element | Requirement |
|---|---|
| Sample identification | Each grab sample assigned a unique sample ID at the time of collection: [SiteID]-[YYYYMMDD]-[HHMM]-[CollectorInitials]. Label affixed to sample bottle before leaving the collection site. |
| Collector signature | COC form signed by collector at collection. Field conditions (weather, flow, turbidity estimate, any anomalies) recorded at time of collection. |
| Transfer documentation | COC accompanies sample from field to laboratory. Each transfer (e.g., field to cooler, cooler to lab intake) documented with date, time, and signature. |
| Laboratory receipt | Laboratory intake staff sign and timestamp COC upon receipt. Record condition of samples (temperature, ice presence, container integrity). Reject and document samples received outside holding time or temperature. |
| Holding time compliance | Sample must be analyzed (Colilert added) within 6 hours of collection. Holding time calculated from collection time recorded on COC, not laboratory receipt time. |
| Digital COC backup | Photograph or scan of completed COC uploaded to study database within 24 hours of laboratory receipt. |
