Skip to main content

Data Validation

The validation module provides a Pydantic-based validation system. It ensures data integrity and catches malformed responses from the CDN before they reach the application code. The fetch pipeline calls the pydantic-free validate_*_data() fast paths, which check structure, aliases, and array lengths. The pydantic models back the manual validate_laps(), validate_telemetry(), validate_weather(), validate_race_control(), and validate_drivers() functions.

Overview

The tif1 validation system acts as a quality gate between raw CDN JSON data and the application. It provides structural validation, type checking, and anomaly detection. This ensures that the data is clean and consistent.

What Validation Catches

The validation system detects and handles:
  • Missing Required Fields: Ensures all mandatory data fields are present
  • Incorrect Data Types: Validates that numeric fields contain numbers, booleans are booleans, etc.
  • Inconsistent Array Lengths: Verifies all parallel arrays have matching lengths (critical for DataFrame construction)
  • Null-like String Values: Automatically converts "", “none”, “null”, “nan” to proper None values
  • Out-of-Range Values: Validates numeric constraints on lap payloads (stint >= 1, tire life >= 0). The per-sample TelemetryPoint model also bounds RPM (<= 20,000), speed, gear (<= 8), throttle, and acceleration; the batch telemetry model does not
  • Known Enum Values: TireCompound, SessionType, and LapStatus list the known categorical values for application code to check against. The payload validators do not reject unknown compounds, and SessionData only logs a warning for an unknown session type
  • Data Anomalies: Detects missing laps, duplicate lap numbers, and statistical outliers
  • Field Aliases: Handles both verbose and abbreviated field names from different CDN formats

Performance Considerations

Validation is disabled by default for optimal performance. The library is designed for speed, and validation adds work to every fetch. Enable validation during development, debugging, or when working with untrusted data sources. Validation only runs on payloads fetched from the CDN outside ultra-cold mode. With the default ultra_cold_start=True, a session that is not in the cache yet loads in ultra-cold mode and skips validation. Set ultra_cold_start to False as well (see Configuration).
When to Enable Validation:
  • Development and testing environments
  • First-time data exploration for new seasons/events
  • Debugging data quality issues
  • Working with custom CDN sources (the cdns setting)
  • Building data quality monitoring pipelines
When to Keep Validation Disabled:
  • Production environments with trusted data
  • Performance-critical applications
  • Batch processing large datasets
  • Repeated analysis of the same sessions

Core Validation Functions

validate_laps

Validates lap timing data structure with full field checking and length consistency validation.
Purpose: Validates raw lap timing JSON from the CDN. Ensures that all required fields are present and that arrays have consistent lengths. Values must meet domain constraints (for example, stint >= 1, tire life >= 0). Parameters:
  • data (dict): Raw JSON dictionary from CDN containing lap timing arrays
Returns:
  • LapData: Validated Pydantic model with all fields type-checked and normalized
Raises:
  • pydantic.ValidationError: If validation fails (missing fields, type mismatches, inconsistent lengths)
Validation Rules:
  • All required fields must be present: time, lap, s1, s2, s3, compound, stint, life, pos, status, pb
  • All non-empty arrays must have identical length
  • Stint numbers must be >= 1
  • Tire life must be >= 0
  • Null-like strings ("", “none”, “null”, “nan”) in any list field are converted to None before the model is built
  • Optional fields (session_time, pit times, speed traps, weather) are validated if present
Example:
Common Validation Errors:

validate_telemetry

Validates high-frequency telemetry data in batch mode (faster than point-by-point validation).
Purpose: Validates raw telemetry JSON containing arrays of sensor readings (speed, RPM, throttle, brake, etc.). Handles nested tel objects and performs boolean coercion for brake/DRS fields. Parameters:
  • data (dict): Raw JSON dictionary from CDN containing telemetry arrays
Returns:
  • TelemetryData: Validated Pydantic model with normalized field names and types
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • Required fields: time, speed (minimum 1 element each)
  • All non-empty arrays must have identical length
  • Automatically unwraps nested tel objects if present
  • Boolean coercion for brake and drs fields (handles numeric 0/1 values)
  • Null-like strings converted to None
  • Supports both aliased (DriverAhead, DistanceToDriverAhead, dataKey) and standard field names
Special Handling: The validator handles two common CDN formats:
Example:
Working with Nested Tel Objects:
Performance Note: Batch validation checks whole arrays in one model. Use validate_telemetry() for array data rather than the per-sample TelemetryPoint model.

validate_drivers

Validates driver information data structure.
Purpose: Validates driver roster data from the CDN. Ensures that all driver codes follow the 3-letter format and that required metadata fields are present. Parameters:
  • data (dict): Raw JSON dictionary from CDN with drivers array
Returns:
  • DriversData: Validated Pydantic model containing list of DriverInfo objects
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • driver: Must be exactly 3 uppercase letters (for example, “VER”, “HAM”, “LEC”)
  • team: Team name, 1-100 characters
  • dn: Driver number (string)
  • fn: First name (required)
  • ln: Last name (required)
  • tc: Team color hex code (required)
  • url: Headshot photo URL (required)
Example:
Common Validation Errors:
Integration with Session: Session loading does not call validate_drivers(). The validate_data option validates weather.json and rcm.json (see Configuration).

validate_weather

Validates weather data structure with automatic key normalization.
Purpose: Validates weather sensor data from the CDN, handling both PascalCase and aliased field names. Ensures consistent array lengths for time-series weather data. Parameters:
  • data (dict): Raw JSON dictionary from CDN containing weather arrays
Returns:
  • WeatherData: Validated Pydantic model with normalized field names
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • Required field: time (or alias wT) - timestamp in seconds
  • All non-empty arrays must have identical length
  • Automatically normalizes PascalCase keys (Time, AirTemp) to snake_case
  • Accepts both verbose (air_temp) and aliased (wAT) field names
  • Null-like strings converted to None
Supported Field Formats: The validator accepts three naming conventions:
Example:
Field Mapping Reference: Handling Mixed Formats:

validate_race_control

Validates race control messages (flags, safety car, penalties, etc.).
Purpose: Validates race control message data from the CDN, ensuring message timestamps, categories, and metadata are properly structured. Parameters:
  • data (dict): Raw JSON dictionary from CDN containing race control message arrays
Returns:
  • RaceControlData: Validated Pydantic model with normalized message data
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • Required field: time - message timestamps. The CDN rcm.json stores ISO 8601 wall-clock strings; session-relative seconds are accepted as well
  • All non-empty arrays must have identical length
  • Supports aliased field names (cat, msg, dNum)
  • Null-like strings converted to None
  • Sector field accepts both int and string values
Example:
Message Categories: Race control message categories in the CDN data:
  • Flag: Track flag changes (GREEN, YELLOW, RED, BLUE, etc.)
  • SafetyCar: Safety car and virtual safety car (VSC) deployment and withdrawal
  • Drs: DRS enabled/disabled
  • CarEvent: Incidents of one car (for example, “CAR 5 (VET) SPUN AND CONTINUED AT TURN 3”)
  • Other: Other messages, including penalties and investigations
Field Reference: Filtering Messages by Category:

Pydantic Models

LapData

Pydantic model for lap timing data with automatic length consistency validation. Purpose: Represents validated lap timing data with all required and optional fields. Ensures all parallel arrays have consistent lengths, which is critical for DataFrame construction. Architecture: Inherits from ConsistentLengthsMixin which provides automatic array length validation across all fields. Required Fields: Optional Fields (with aliases): Weather Fields (per-lap): Validation Behavior:
  1. Length Consistency: All non-empty lists must have the same length
  2. Stint Validation: All stint values must be >= 1
  3. Tire Life Validation: All tire life values must be >= 0
  4. Null Normalization: validate_laps() and validate_lap_data() convert null-like strings to None before they build the model; LapData(...) itself does not
  5. Alias Support: Accepts both verbose and aliased field names
Example:
Working with Aliases:

TelemetryData

Pydantic model for high-frequency telemetry data with automatic unwrapping of nested structures. Purpose: Represents validated telemetry sensor data with support for multiple CDN formats. Handles nested tel objects and performs boolean coercion for brake/DRS fields. Architecture:
  • Inherits from ConsistentLengthsMixin for automatic array length validation
  • Implements _unwrap_tel pre-validator to handle nested structures
  • Supports both aliased and standard field names
Required Fields: Optional Fields: Special Handling:
  1. Nested Tel Objects: Automatically unwraps tel nested structures
  2. Boolean Coercion: Converts numeric 0/1 to False/True for brake and drs
  3. Null Normalization: validate_telemetry() converts null-like strings to None before it builds the model
  4. Empty Arrays: Optional fields can be empty arrays
Validation Behavior:
Example:
Handling Nested Structures:
Boolean Coercion:

WeatherData

Pydantic model for session weather data with automatic key normalization. Purpose: Represents validated weather sensor data with support for multiple naming conventions (PascalCase, aliased, snake_case). Architecture:
  • Inherits from ConsistentLengthsMixin for array length validation
  • Implements _normalize_pascalcase_keys pre-validator for key normalization
  • Supports three naming conventions simultaneously
Required Field: Optional Fields: Validation Behavior:
  1. Key Normalization: Automatically converts PascalCase to snake_case
  2. Alias Support: Accepts compact aliased names (wT, wAT, etc.)
  3. Length Consistency: All non-empty arrays must have same length
  4. Null Normalization: validate_weather() converts null-like strings to None before it builds the model
Example:

RaceControlData

Pydantic model for race control messages with flexible field types. Purpose: Represents validated race control message data including flags, safety car deployments, and penalties. Architecture:
  • Inherits from ConsistentLengthsMixin for array length validation
  • Supports aliased field names
  • Flexible sector field (accepts both int and string)
Required Field: Optional Fields: Example:

DriversData

Container model for driver information. Fields:
  • drivers: List of DriverInfo objects
Example:

DriverInfo

Pydantic model for individual driver information with strict validation. Fields: Validation:
  • Driver code must be exactly 3 uppercase letters
  • Team name must be 1-100 characters
  • All fields are required (no None values)
Example:

Enums

TireCompound

Enumeration of valid tire compound values used in F1 sessions.
Usage:
Compound Types:
  • SOFT: Softest compound, fastest but least durable
  • MEDIUM: Middle compound, balanced performance
  • HARD: Hardest compound, slowest but most durable
  • INTERMEDIATE: For damp conditions
  • WET: For wet conditions
  • UNKNOWN: Compound not identified
  • TEST-UNKNOWN: Used in testing sessions

SessionType

Enumeration of valid F1 session types.
Usage:
Session Types:
  • Practice 1/2/3: Free practice sessions
  • Qualifying: Standard qualifying format
  • Sprint: Sprint race (100km race on Saturday)
  • Sprint Qualifying: Qualifying for sprint race (2024+ format)
  • Sprint Shootout: Short qualifying for sprint race (2023 format)
  • Race: Main Grand Prix race

LapStatus

Enumeration of valid lap status values.
Usage:
The status field of CDN lap payloads holds track status codes (“1”, “4”, …), not LapStatus values. Status Types:
  • VALID: Clean lap with no track limits violations
  • INVALID: Lap deleted due to track limits or other violations
  • OUTLAP: Lap exiting pit lane
  • INLAP: Lap entering pit lane

AnomalyType

Enumeration of data anomaly types detected by the validation system.
Usage:
Anomaly Types:
  • MISSING_LAPS: Gaps in lap number sequence (for example, laps 1, 2, 4, 5 - missing lap 3)
  • DUPLICATE_LAPS: Same lap number appears multiple times
  • OUTLIER_TIMES: Lap times far from the average (>3x mean)

Anomaly Detection

detect_lap_anomalies

Detects data quality issues in lap data with structured, actionable results.
Purpose: Analyzes lap data to identify missing laps, duplicate lap numbers, and statistical outliers. Returns structured anomaly objects with severity levels and detailed context. Parameters:
  • laps (list[dict]): List of lap dictionaries with at least one of the lap or time fields
Returns:
  • list[Anomaly]: List of detected anomalies with type, severity, description, and details
Detection Logic:
  1. Missing Laps: Checks for gaps in lap number sequence
    • Severity: medium
    • Details: List of missing lap numbers
  2. Duplicate Laps: Identifies lap numbers that appear multiple times
    • Severity: high
    • Details: List of duplicate lap numbers
  3. Outlier Times: Finds lap times >3x the average (lenient threshold)
    • Severity: low
    • Details: Count of outliers and average lap time
    • Requires: At least 3 laps for meaningful statistics
Example:
Filtering by Severity:
Integration with Session Data: detect_lap_anomalies() reads the lap and time keys, or LapNumber and a numeric LapTime. The LapTime column of tif1 is a timedelta, so pass LapTimeSeconds as time, one driver at a time.

Anomaly

Structured model for detected data anomalies. Fields: Example:
Severity Levels:
  • low: Minor issues that do not affect analysis (for example, a lap time over 3x the average)
  • medium: Moderate issues that may affect completeness (for example, missing laps)
  • high: Critical issues that indicate data corruption (for example, duplicate lap numbers)

Configuration

Validation in tif1 is controlled through the validate_data configuration option and related settings. The system is designed for maximum performance by default, with validation disabled to minimize overhead.

Global Configuration

Enable Validation Globally

Disable Validation (Default)

Environment Variables

Configure validation via environment variables for deployment environments:

Configuration File

Set validation in .tif1rc configuration file:
File Locations (checked in order):
  1. TIF1_CONFIG_FILE environment variable path
  2. ./.tif1rc (if TIF1_TRUST_CWD_CONFIG=true)
  3. ~/.tif1rc (user home directory)

Validation Behavior

What Gets Validated

Validation runs on payloads fetched from the CDN, not on payloads served from the cache. Each payload type has its own configuration key: The pipeline does not validate drivers.json. Validation is also skipped in ultra-cold mode. ultra_cold_start is True by default, and a session that is not in the cache yet then loads in ultra-cold mode. Set ultra_cold_start to False to validate a cold load.

Validation Modes

Non-Strict Mode (Default):
  • Validation errors are logged but do not raise exceptions
  • Original data is returned if validation fails
  • Suitable for production environments
Strict Mode:
  • Validation errors raise InvalidDataError exceptions
  • Suitable for development and testing
  • Ensures data quality is enforced

Advanced Configuration

Selective Validation

Enable validation for specific data types only:

Ultra Cold Start Mode

Validation is automatically disabled in ultra-cold mode for maximum performance. ultra_cold_start is True by default and applies to sessions that are not in the cache yet:

Performance Tuning

Configuration Persistence

Save Configuration

Load Configuration

Validation Integration Points

Automatic Validation (Session Loading)

Manual Validation (Custom Workflows)

Configuration Best Practices

  1. Development: Enable all validation to catch data issues early
  1. Testing: Use strict mode to enforce data quality
  1. Production: Disable validation for maximum performance
  1. CI/CD: Use environment variables for configuration
  1. Monitoring: Enable validation periodically to check data quality

Complete Examples

Custom Validation


Anomaly Detection Workflow


Validation with Logging


Best Practices

  1. Use strict mode during development: Catches data issues early.
  1. Handle validation errors gracefully: Do not crash on bad data.
  1. Run anomaly detection periodically: Monitor data quality over time.
  2. Clean data before validation: Remove obvious errors first, as clean_lap_data() does in Custom Validation.
  3. Use null-like string conversion: validate_laps(), validate_lap_data(), validate_telemetry(), validate_weather() and validate_race_control() convert "", “none”, “null”, “nan” to None. The model constructors (LapData(...), TelemetryData(...)) do not.

Troubleshooting

Validation Errors

Inconsistent Lengths

Performance Issues

Last modified on October 15, 2026