Data Validation
The validation module provides a comprehensive, Pydantic-based validation system that ensures data integrity and catches malformed responses from the CDN before they reach your application code.Overview
tif1’s validation system acts as a quality gate between raw JSON data from the CDN and your application. It performs deep structural validation, type checking, and anomaly detection to ensure you’re working with clean, consistent data.What Validation Catches
The validation system detects and handles:- Missing Required Fields: Ensures all mandatory data fields are present
- Incorrect Data Types: Validates that numeric fields contain numbers, booleans are booleans, etc.
- Inconsistent Array Lengths: Verifies all parallel arrays have matching lengths (critical for DataFrame construction)
- Invalid Enum Values: Checks tire compounds, session types, and other categorical data against known valid values
- Null-like String Values: Automatically converts "", “none”, “null”, “nan” to proper None values
- Out-of-Range Values: Validates numeric constraints (e.g., RPM < 20,000, gear <= 8, stint >= 1)
- Data Anomalies: Detects missing laps, duplicate lap numbers, and statistical outliers
- Field Aliases: Handles both verbose and abbreviated field names from different CDN formats
Performance Considerations
Validation is disabled by default for optimal performance. The library is designed for speed, and validation adds 10-15% overhead. Enable validation during development, debugging, or when working with untrusted data sources.
- Development and testing environments
- First-time data exploration for new seasons/events
- Debugging data quality issues
- Working with experimental or beta CDN endpoints
- Building data quality monitoring pipelines
- Production environments with trusted data
- Performance-critical applications
- Batch processing large datasets
- Repeated analysis of the same sessions
Core Validation Functions
validate_laps
Validates lap timing data structure with comprehensive field checking and length consistency validation.
data(dict): Raw JSON dictionary from CDN containing lap timing arrays
LapData: Validated Pydantic model with all fields type-checked and normalized
pydantic.ValidationError: If validation fails (missing fields, type mismatches, inconsistent lengths)
- All required fields must be present:
time,lap,s1,s2,s3,compound,stint,life,pos,status,pb - All non-empty arrays must have identical length
- Stint numbers must be >= 1
- Tire life must be >= 0
- Null-like strings ("", “none”, “null”, “nan”) are automatically converted to None
- Optional fields (session_time, pit times, speed traps, weather) are validated if present
validate_telemetry
Validates high-frequency telemetry data in batch mode (significantly faster than point-by-point validation).
tel objects and performs boolean coercion for brake/DRS fields.
Parameters:
data(dict): Raw JSON dictionary from CDN containing telemetry arrays
TelemetryData: Validated Pydantic model with normalized field names and types
pydantic.ValidationError: If validation fails
- Required fields:
time,speed(minimum 1 element each) - All non-empty arrays must have identical length
- Automatically unwraps nested
telobjects if present - Boolean coercion for
brakeanddrsfields (handles numeric 0/1 values) - Null-like strings converted to None
- Supports both aliased (
DriverAhead,DistanceToDriverAhead,dataKey) and standard field names
validate_telemetry() for array data rather than validating individual telemetry points.
validate_drivers
Validates driver information data structure.
data(dict): Raw JSON dictionary from CDN withdriversarray
DriversData: Validated Pydantic model containing list ofDriverInfoobjects
pydantic.ValidationError: If validation fails
driver: Must be exactly 3 uppercase letters (e.g., “VER”, “HAM”, “LEC”)team: Team name, 1-100 charactersdn: Driver number (string)fn: First name (required)ln: Last name (required)tc: Team color hex code (required)url: Headshot photo URL (required)
validate_weather
Validates weather data structure with automatic key normalization.
data(dict): Raw JSON dictionary from CDN containing weather arrays
WeatherData: Validated Pydantic model with normalized field names
pydantic.ValidationError: If validation fails
- Required field:
time(or aliaswT) - timestamp in seconds - All non-empty arrays must have identical length
- Automatically normalizes PascalCase keys (
Time,AirTemp) to snake_case - Accepts both verbose (
air_temp) and aliased (wAT) field names - Null-like strings converted to None
Handling Mixed Formats:
data(dict): Raw JSON dictionary from CDN containing race control message arrays
RaceControlData: Validated Pydantic model with normalized message data
pydantic.ValidationError: If validation fails
- Required field:
time- message timestamp in seconds - All non-empty arrays must have identical length
- Supports aliased field names (
cat,msg,dNum) - Null-like strings converted to None
- Sector field accepts both int and string values
Flag: Track flag changes (GREEN, YELLOW, RED, BLUE, etc.)SafetyCar: Safety car deployment/withdrawalVirtualSafetyCar: VSC deployment/withdrawalPenalty: Driver penalties (time penalties, drive-through, etc.)DRS: DRS enabled/disabledOther: Miscellaneous messages
Filtering Messages by Category:
Pydantic Models
LapData
Comprehensive Pydantic model for lap timing data with automatic length consistency validation.
Purpose:
Represents validated lap timing data with all required and optional fields. Ensures all parallel arrays have consistent lengths, which is critical for DataFrame construction.
Architecture:
Inherits from ConsistentLengthsMixin which provides automatic array length validation across all fields.
Required Fields:
Optional Fields (with aliases):
Weather Fields (per-lap):
Validation Behavior:
- Length Consistency: All non-empty lists must have the same length
- Stint Validation: All stint values must be >= 1
- Tire Life Validation: All tire life values must be >= 0
- Null Normalization: Null-like strings ("", “none”, “null”, “nan”) converted to None
- Alias Support: Accepts both verbose and aliased field names
TelemetryData
Comprehensive Pydantic model for high-frequency telemetry data with automatic unwrapping of nested structures.
Purpose:
Represents validated telemetry sensor data with support for multiple CDN formats. Handles nested tel objects and performs boolean coercion for brake/DRS fields.
Architecture:
- Inherits from
ConsistentLengthsMixinfor automatic array length validation - Implements
_unwrap_telpre-validator to handle nested structures - Supports both aliased and standard field names
Optional Fields:
Special Handling:
- Nested Tel Objects: Automatically unwraps
telnested structures - Boolean Coercion: Converts numeric 0/1 to False/True for
brakeanddrs - Null Normalization: Converts null-like strings to None
- Empty Arrays: Optional fields can be empty arrays
WeatherData
Pydantic model for session weather data with automatic key normalization.
Purpose:
Represents validated weather sensor data with support for multiple naming conventions (PascalCase, aliased, snake_case).
Architecture:
- Inherits from
ConsistentLengthsMixinfor array length validation - Implements
_normalize_pascalcase_keyspre-validator for key normalization - Supports three naming conventions simultaneously
Optional Fields:
Validation Behavior:
- Key Normalization: Automatically converts PascalCase to snake_case
- Alias Support: Accepts compact aliased names (wT, wAT, etc.)
- Length Consistency: All non-empty arrays must have same length
- Null Normalization: Converts null-like strings to None
RaceControlData
Pydantic model for race control messages with flexible field types.
Purpose:
Represents validated race control message data including flags, safety car deployments, and penalties.
Architecture:
- Inherits from
ConsistentLengthsMixinfor array length validation - Supports aliased field names
- Flexible sector field (accepts both int and string)
Optional Fields:
Example:
DriversData
Container model for driver information.
Fields:
drivers: List ofDriverInfoobjects
DriverInfo
Pydantic model for individual driver information with strict validation.
Fields:
Validation:
- Driver code must be exactly 3 uppercase letters
- Team name must be 1-100 characters
- All fields are required (no None values)
Enums
TireCompound
Enumeration of valid tire compound values used in F1 sessions.
- SOFT: Softest compound, fastest but least durable
- MEDIUM: Middle compound, balanced performance
- HARD: Hardest compound, slowest but most durable
- INTERMEDIATE: For damp conditions
- WET: For wet conditions
- UNKNOWN: Compound not identified
- TEST-UNKNOWN: Used in testing sessions
SessionType
Enumeration of valid F1 session types.
- Practice 1/2/3: Free practice sessions
- Qualifying: Standard qualifying format
- Sprint: Sprint race (100km race on Saturday)
- Sprint Qualifying: Qualifying for sprint race (2023 format)
- Sprint Shootout: Short qualifying for sprint race (2024+ format)
- Race: Main Grand Prix race
LapStatus
Enumeration of valid lap status values.
- VALID: Clean lap with no track limits violations
- INVALID: Lap deleted due to track limits or other violations
- OUTLAP: Lap exiting pit lane
- INLAP: Lap entering pit lane
AnomalyType
Enumeration of data anomaly types detected by the validation system.
- MISSING_LAPS: Gaps in lap number sequence (e.g., laps 1, 2, 4, 5 - missing lap 3)
- DUPLICATE_LAPS: Same lap number appears multiple times
- OUTLIER_TIMES: Lap times significantly different from average (>3x mean)
Anomaly Detection
detect_lap_anomalies
Detects data quality issues in lap data with structured, actionable results.
laps(list[dict]): List of lap dictionaries with at leastlapand/ortimefields
list[Anomaly]: List of detected anomalies with type, severity, description, and details
-
Missing Laps: Checks for gaps in lap number sequence
- Severity:
medium - Details: List of missing lap numbers
- Severity:
-
Duplicate Laps: Identifies lap numbers that appear multiple times
- Severity:
high - Details: List of duplicate lap numbers
- Severity:
-
Outlier Times: Finds lap times >3x the average (lenient threshold)
- Severity:
low - Details: Count of outliers and average lap time
- Requires: At least 3 laps for meaningful statistics
- Severity:
Anomaly
Structured model for detected data anomalies.
Fields:
Example:
- low: Minor issues that don’t affect analysis (e.g., statistical outliers from pit stops)
- medium: Moderate issues that may affect completeness (e.g., missing laps)
- high: Critical issues that indicate data corruption (e.g., duplicate lap numbers)
Configuration
Validation in tif1 is controlled through thevalidate_data configuration option and related settings. The system is designed for maximum performance by default, with validation disabled to minimize overhead.
Global Configuration
Enable Validation Globally
Disable Validation (Default)
Environment Variables
Configure validation via environment variables for deployment environments:Configuration File
Set validation in.tif1rc configuration file:
TIF1_CONFIG_FILEenvironment variable path./tif1rc(ifTIF1_TRUST_CWD_CONFIG=true)~/.tif1rc(user home directory)
Validation Behavior
What Gets Validated
Whenvalidate_data=True, the following CDN payloads are validated:
Note: Lap and telemetry data validation is controlled separately (see below).
Validation Modes
Non-Strict Mode (Default):- Validation errors are logged but don’t raise exceptions
- Original data is returned if validation fails
- Suitable for production environments
- Validation errors raise
InvalidDataErrorexceptions - Suitable for development and testing
- Ensures data quality is enforced
Advanced Configuration
Selective Validation
Enable validation for specific data types only:Ultra Cold Start Mode
Validation is automatically disabled in ultra-cold start mode for maximum performance:Performance Tuning
Configuration Persistence
Save Configuration
Load Configuration
Validation Integration Points
Automatic Validation (Session Loading)
Manual Validation (Custom Workflows)
Configuration Best Practices
- Development: Enable all validation to catch data issues early
- Testing: Use strict mode to enforce data quality
- Production: Disable validation for maximum performance
- CI/CD: Use environment variables for configuration
- Monitoring: Enable validation periodically to check data quality
Complete Examples
Custom Validation
Anomaly Detection Workflow
Validation with Logging
Best Practices
- Use strict mode during development: Catches data issues early.
- Handle validation errors gracefully: Don’t crash on bad data.
- Run anomaly detection periodically: Monitor data quality over time.
- Clean data before validation: Remove obvious errors first using normalization functions.
- Leverage null-like string conversion: The validation module automatically converts "", “none”, “null”, “nan” to None.