Skip to main content

Data Validation

The validation module provides a comprehensive, Pydantic-based validation system that ensures data integrity and catches malformed responses from the CDN before they reach your application code.

Overview

tif1’s validation system acts as a quality gate between raw JSON data from the CDN and your application. It performs deep structural validation, type checking, and anomaly detection to ensure you’re working with clean, consistent data.

What Validation Catches

The validation system detects and handles:
  • Missing Required Fields: Ensures all mandatory data fields are present
  • Incorrect Data Types: Validates that numeric fields contain numbers, booleans are booleans, etc.
  • Inconsistent Array Lengths: Verifies all parallel arrays have matching lengths (critical for DataFrame construction)
  • Invalid Enum Values: Checks tire compounds, session types, and other categorical data against known valid values
  • Null-like String Values: Automatically converts "", “none”, “null”, “nan” to proper None values
  • Out-of-Range Values: Validates numeric constraints (e.g., RPM < 20,000, gear <= 8, stint >= 1)
  • Data Anomalies: Detects missing laps, duplicate lap numbers, and statistical outliers
  • Field Aliases: Handles both verbose and abbreviated field names from different CDN formats

Performance Considerations

Validation is disabled by default for optimal performance. The library is designed for speed, and validation adds 10-15% overhead. Enable validation during development, debugging, or when working with untrusted data sources.
When to Enable Validation:
  • Development and testing environments
  • First-time data exploration for new seasons/events
  • Debugging data quality issues
  • Working with experimental or beta CDN endpoints
  • Building data quality monitoring pipelines
When to Keep Validation Disabled:
  • Production environments with trusted data
  • Performance-critical applications
  • Batch processing large datasets
  • Repeated analysis of the same sessions

Core Validation Functions

validate_laps

Validates lap timing data structure with comprehensive field checking and length consistency validation.
Purpose: Validates raw lap timing JSON from the CDN, ensuring all required fields are present, arrays have consistent lengths, and values meet domain constraints (e.g., stint >= 1, tire life >= 0). Parameters:
  • data (dict): Raw JSON dictionary from CDN containing lap timing arrays
Returns:
  • LapData: Validated Pydantic model with all fields type-checked and normalized
Raises:
  • pydantic.ValidationError: If validation fails (missing fields, type mismatches, inconsistent lengths)
Validation Rules:
  • All required fields must be present: time, lap, s1, s2, s3, compound, stint, life, pos, status, pb
  • All non-empty arrays must have identical length
  • Stint numbers must be >= 1
  • Tire life must be >= 0
  • Null-like strings ("", “none”, “null”, “nan”) are automatically converted to None
  • Optional fields (session_time, pit times, speed traps, weather) are validated if present
Example:
Common Validation Errors:

validate_telemetry

Validates high-frequency telemetry data in batch mode (significantly faster than point-by-point validation).
Purpose: Validates raw telemetry JSON containing arrays of sensor readings (speed, RPM, throttle, brake, etc.). Handles nested tel objects and performs boolean coercion for brake/DRS fields. Parameters:
  • data (dict): Raw JSON dictionary from CDN containing telemetry arrays
Returns:
  • TelemetryData: Validated Pydantic model with normalized field names and types
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • Required fields: time, speed (minimum 1 element each)
  • All non-empty arrays must have identical length
  • Automatically unwraps nested tel objects if present
  • Boolean coercion for brake and drs fields (handles numeric 0/1 values)
  • Null-like strings converted to None
  • Supports both aliased (DriverAhead, DistanceToDriverAhead, dataKey) and standard field names
Special Handling: The validator handles two common CDN formats:
Example:
Working with Nested Tel Objects:
Performance Note: Batch validation is ~50x faster than point-by-point validation. Always use validate_telemetry() for array data rather than validating individual telemetry points.

validate_drivers

Validates driver information data structure.
Purpose: Validates driver roster data from the CDN, ensuring all driver codes follow the 3-letter format and required metadata fields are present. Parameters:
  • data (dict): Raw JSON dictionary from CDN with drivers array
Returns:
  • DriversData: Validated Pydantic model containing list of DriverInfo objects
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • driver: Must be exactly 3 uppercase letters (e.g., “VER”, “HAM”, “LEC”)
  • team: Team name, 1-100 characters
  • dn: Driver number (string)
  • fn: First name (required)
  • ln: Last name (required)
  • tc: Team color hex code (required)
  • url: Headshot photo URL (required)
Example:
Common Validation Errors:
Integration with Session:

validate_weather

Validates weather data structure with automatic key normalization.
Purpose: Validates weather sensor data from the CDN, handling both PascalCase and aliased field names. Ensures consistent array lengths for time-series weather data. Parameters:
  • data (dict): Raw JSON dictionary from CDN containing weather arrays
Returns:
  • WeatherData: Validated Pydantic model with normalized field names
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • Required field: time (or alias wT) - timestamp in seconds
  • All non-empty arrays must have identical length
  • Automatically normalizes PascalCase keys (Time, AirTemp) to snake_case
  • Accepts both verbose (air_temp) and aliased (wAT) field names
  • Null-like strings converted to None
Supported Field Formats: The validator accepts three naming conventions:
Example:
Field Mapping Reference: Handling Mixed Formats:
Purpose: Validates race control message data from the CDN, ensuring message timestamps, categories, and metadata are properly structured. Parameters:
  • data (dict): Raw JSON dictionary from CDN containing race control message arrays
Returns:
  • RaceControlData: Validated Pydantic model with normalized message data
Raises:
  • pydantic.ValidationError: If validation fails
Validation Rules:
  • Required field: time - message timestamp in seconds
  • All non-empty arrays must have identical length
  • Supports aliased field names (cat, msg, dNum)
  • Null-like strings converted to None
  • Sector field accepts both int and string values
Example:
Message Categories: Common race control message categories:
  • Flag: Track flag changes (GREEN, YELLOW, RED, BLUE, etc.)
  • SafetyCar: Safety car deployment/withdrawal
  • VirtualSafetyCar: VSC deployment/withdrawal
  • Penalty: Driver penalties (time penalties, drive-through, etc.)
  • DRS: DRS enabled/disabled
  • Other: Miscellaneous messages
Field Reference: Filtering Messages by Category:

Pydantic Models

LapData

Comprehensive Pydantic model for lap timing data with automatic length consistency validation. Purpose: Represents validated lap timing data with all required and optional fields. Ensures all parallel arrays have consistent lengths, which is critical for DataFrame construction. Architecture: Inherits from ConsistentLengthsMixin which provides automatic array length validation across all fields. Required Fields: Optional Fields (with aliases): Weather Fields (per-lap): Validation Behavior:
  1. Length Consistency: All non-empty lists must have the same length
  2. Stint Validation: All stint values must be >= 1
  3. Tire Life Validation: All tire life values must be >= 0
  4. Null Normalization: Null-like strings ("", “none”, “null”, “nan”) converted to None
  5. Alias Support: Accepts both verbose and aliased field names
Example:
Working with Aliases:

TelemetryData

Comprehensive Pydantic model for high-frequency telemetry data with automatic unwrapping of nested structures. Purpose: Represents validated telemetry sensor data with support for multiple CDN formats. Handles nested tel objects and performs boolean coercion for brake/DRS fields. Architecture:
  • Inherits from ConsistentLengthsMixin for automatic array length validation
  • Implements _unwrap_tel pre-validator to handle nested structures
  • Supports both aliased and standard field names
Required Fields: Optional Fields: Special Handling:
  1. Nested Tel Objects: Automatically unwraps tel nested structures
  2. Boolean Coercion: Converts numeric 0/1 to False/True for brake and drs
  3. Null Normalization: Converts null-like strings to None
  4. Empty Arrays: Optional fields can be empty arrays
Validation Behavior:
Example:
Handling Nested Structures:
Boolean Coercion:

WeatherData

Pydantic model for session weather data with automatic key normalization. Purpose: Represents validated weather sensor data with support for multiple naming conventions (PascalCase, aliased, snake_case). Architecture:
  • Inherits from ConsistentLengthsMixin for array length validation
  • Implements _normalize_pascalcase_keys pre-validator for key normalization
  • Supports three naming conventions simultaneously
Required Field: Optional Fields: Validation Behavior:
  1. Key Normalization: Automatically converts PascalCase to snake_case
  2. Alias Support: Accepts compact aliased names (wT, wAT, etc.)
  3. Length Consistency: All non-empty arrays must have same length
  4. Null Normalization: Converts null-like strings to None
Example:

RaceControlData

Pydantic model for race control messages with flexible field types. Purpose: Represents validated race control message data including flags, safety car deployments, and penalties. Architecture:
  • Inherits from ConsistentLengthsMixin for array length validation
  • Supports aliased field names
  • Flexible sector field (accepts both int and string)
Required Field: Optional Fields: Example:

DriversData

Container model for driver information. Fields:
  • drivers: List of DriverInfo objects
Example:

DriverInfo

Pydantic model for individual driver information with strict validation. Fields: Validation:
  • Driver code must be exactly 3 uppercase letters
  • Team name must be 1-100 characters
  • All fields are required (no None values)
Example:

Enums

TireCompound

Enumeration of valid tire compound values used in F1 sessions.
Usage:
Compound Types:
  • SOFT: Softest compound, fastest but least durable
  • MEDIUM: Middle compound, balanced performance
  • HARD: Hardest compound, slowest but most durable
  • INTERMEDIATE: For damp conditions
  • WET: For wet conditions
  • UNKNOWN: Compound not identified
  • TEST-UNKNOWN: Used in testing sessions

SessionType

Enumeration of valid F1 session types.
Usage:
Session Types:
  • Practice 1/2/3: Free practice sessions
  • Qualifying: Standard qualifying format
  • Sprint: Sprint race (100km race on Saturday)
  • Sprint Qualifying: Qualifying for sprint race (2023 format)
  • Sprint Shootout: Short qualifying for sprint race (2024+ format)
  • Race: Main Grand Prix race

LapStatus

Enumeration of valid lap status values.
Usage:
Status Types:
  • VALID: Clean lap with no track limits violations
  • INVALID: Lap deleted due to track limits or other violations
  • OUTLAP: Lap exiting pit lane
  • INLAP: Lap entering pit lane

AnomalyType

Enumeration of data anomaly types detected by the validation system.
Usage:
Anomaly Types:
  • MISSING_LAPS: Gaps in lap number sequence (e.g., laps 1, 2, 4, 5 - missing lap 3)
  • DUPLICATE_LAPS: Same lap number appears multiple times
  • OUTLIER_TIMES: Lap times significantly different from average (>3x mean)

Anomaly Detection

detect_lap_anomalies

Detects data quality issues in lap data with structured, actionable results.
Purpose: Analyzes lap data to identify missing laps, duplicate lap numbers, and statistical outliers. Returns structured anomaly objects with severity levels and detailed context. Parameters:
  • laps (list[dict]): List of lap dictionaries with at least lap and/or time fields
Returns:
  • list[Anomaly]: List of detected anomalies with type, severity, description, and details
Detection Logic:
  1. Missing Laps: Checks for gaps in lap number sequence
    • Severity: medium
    • Details: List of missing lap numbers
  2. Duplicate Laps: Identifies lap numbers that appear multiple times
    • Severity: high
    • Details: List of duplicate lap numbers
  3. Outlier Times: Finds lap times >3x the average (lenient threshold)
    • Severity: low
    • Details: Count of outliers and average lap time
    • Requires: At least 3 laps for meaningful statistics
Example:
Filtering by Severity:
Integration with Session Data:

Anomaly

Structured model for detected data anomalies. Fields: Example:
Severity Levels:
  • low: Minor issues that don’t affect analysis (e.g., statistical outliers from pit stops)
  • medium: Moderate issues that may affect completeness (e.g., missing laps)
  • high: Critical issues that indicate data corruption (e.g., duplicate lap numbers)

Configuration

Validation in tif1 is controlled through the validate_data configuration option and related settings. The system is designed for maximum performance by default, with validation disabled to minimize overhead.

Global Configuration

Enable Validation Globally

Disable Validation (Default)

Environment Variables

Configure validation via environment variables for deployment environments:

Configuration File

Set validation in .tif1rc configuration file:
File Locations (checked in order):
  1. TIF1_CONFIG_FILE environment variable path
  2. ./tif1rc (if TIF1_TRUST_CWD_CONFIG=true)
  3. ~/.tif1rc (user home directory)

Validation Behavior

What Gets Validated

When validate_data=True, the following CDN payloads are validated: Note: Lap and telemetry data validation is controlled separately (see below).

Validation Modes

Non-Strict Mode (Default):
  • Validation errors are logged but don’t raise exceptions
  • Original data is returned if validation fails
  • Suitable for production environments
Strict Mode:
  • Validation errors raise InvalidDataError exceptions
  • Suitable for development and testing
  • Ensures data quality is enforced

Advanced Configuration

Selective Validation

Enable validation for specific data types only:

Ultra Cold Start Mode

Validation is automatically disabled in ultra-cold start mode for maximum performance:

Performance Tuning

Configuration Persistence

Save Configuration

Load Configuration

Validation Integration Points

Automatic Validation (Session Loading)

Manual Validation (Custom Workflows)

Configuration Best Practices

  1. Development: Enable all validation to catch data issues early
  1. Testing: Use strict mode to enforce data quality
  1. Production: Disable validation for maximum performance
  1. CI/CD: Use environment variables for configuration
  1. Monitoring: Enable validation periodically to check data quality

Complete Examples

Custom Validation


Anomaly Detection Workflow


Validation with Logging


Best Practices

  1. Use strict mode during development: Catches data issues early.
  1. Handle validation errors gracefully: Don’t crash on bad data.
  1. Run anomaly detection periodically: Monitor data quality over time.
  2. Clean data before validation: Remove obvious errors first using normalization functions.
  3. Leverage null-like string conversion: The validation module automatically converts "", “none”, “null”, “nan” to None.

Troubleshooting

Validation Errors

Inconsistent Lengths

Performance Issues

Last modified on May 8, 2026