Skip to main content
Beyond the core timing and telemetry API, tif1 provides utility functions. They manage internal state, performance optimization, reliability layers, and system configuration. These utilities give fine-grained control over caching behavior, network resilience, logging verbosity, data processing pipelines, and more. This guide is an exhaustive reference for all utility functions. It covers parameters, return values, edge cases, performance implications, thread safety, and real-world usage patterns. Use these utilities when debugging data loading issues, optimizing cache performance, or building production applications.

Overview

The utilities in tif1 are organized into several categories:
  • Logging: Control diagnostic output verbosity
  • Configuration: Manage global settings and preferences
  • Cache Management: Control persistent and in-memory caching
  • Data Utilities: Convert and manipulate time and data structures
  • Reliability & Networking: Manage circuit breakers and CDN fallback
  • Memory Management: Clear caches to free resources
All utilities are designed with performance in mind and are thread-safe where applicable.

Logging

Logging is essential for debugging data loading issues, understanding cache behavior, and diagnosing network problems. The tif1 library uses Python’s standard logging module and provides a convenient setup function to control verbosity across all internal components.

setup_logging

Configure the logging level for all tif1 components.
This function configures the root logger and sets the level for the tif1 logger hierarchy. It also suppresses noisy connection pool warnings from the underlying HTTP library. Parameters:
  • level (int): Standard logging level from Python’s logging module. Defaults to logging.WARNING.
Available Levels: Behavior:
  • Configures logging format: %(asctime)s - %(name)s - %(levelname)s - %(message)s
  • Sets date format: %Y-%m-%d %H:%M:%S
  • Applies to all tif1.* loggers (core, cache, http, retry, cdn, etc.)
  • Suppresses urllib3_future.connectionpool warnings (set to ERROR level)
  • Thread-safe: Can be called from any thread
Basic Example:
Debug Output Example:
Selective Logging Example:
Advanced: Per-Module Logging:
Performance Considerations:
  • DEBUG logging adds ~5-10% overhead due to string formatting
  • INFO logging adds ~1-2% overhead
  • WARNING and above have negligible performance impact
  • Logging is buffered and asynchronous in most Python implementations
Best Practices:
  1. Use DEBUG during development to understand data flow
  2. Use INFO in production for operational visibility
  3. Use WARNING (default) for minimal overhead
  4. Disable logging in performance-critical tight loops
  5. Consider using a logging handler that writes to files for production

Configuration

The tif1 configuration system provides a flexible way to customize library behavior through environment variables, configuration files, or programmatic API calls. Configuration is managed through a singleton Config object that loads settings from multiple sources with a defined precedence order.

Configuration Loading Order

Settings are loaded in the following order (later sources override earlier ones):
  1. Built-in defaults: Hardcoded sensible defaults
  2. Configuration file (~/.tif1rc): JSON file in user’s home directory
  3. Environment variables: Prefixed with TIF1_ (for example, TIF1_LIB=polars)
  4. Programmatic calls: Using config.set() at runtime

get_config

Returns the global configuration singleton instance.
The Config object is a singleton, meaning all calls to get_config() return the same instance. This ensures configuration consistency across the entire application. Returns:
  • Config: Singleton configuration instance
Thread Safety:
  • The Config singleton is thread-safe
  • Multiple threads can safely call get() and set()
  • File operations (save()) use atomic writes
Basic Example:

Config Methods

get(key: str, default: Any = None) -> Any

Retrieve a configuration value by key. Parameters:
  • key (str): Configuration key name
  • default (Any, optional): Value to return if key does not exist. Defaults to None.
Returns:
  • The configuration value, or default if key not found
Example:

set(key: str, value: Any) -> None

Update a configuration value in memory. Parameters:
  • key (str): Configuration key name
  • value (Any): New value to set
Behavior:
  • Changes are immediate and affect all subsequent operations
  • Changes are in-memory only until save() is called
  • Type conversion is automatic for known keys
  • Unknown keys are stored as-is
Example:

save(path: Path | str | None = None) -> None

Save current configuration to a JSON file. Parameters:
  • path (Path | str | None, optional): File path to save to. If None, saves to ~/.tif1rc. Defaults to None.
Behavior:
  • Creates parent directories if they do not exist
  • Uses atomic write (write to temp file, then rename)
  • Saves all current settings, including defaults
  • File format is human-readable JSON
Example:
Configuration File Format:

Available Configuration Keys

Reference of all configuration options:

Environment Variable Configuration

All configuration keys can be set via environment variables using the TIF1_ prefix:
Type Conversion:
  • Booleans: "true", "1", "yes"True; "false", "0", "no"False
  • Integers: Parsed automatically
  • Lists: Comma-separated values (for example, "url1,url2,url3")

Configuration Examples

Development Configuration

Production Configuration

Minimal Latency Configuration

Custom Cache Location

Configuration Best Practices

  1. Use environment variables for deployment: Easier to configure across environments without code changes
  2. Save configuration after tuning: Persist the optimized settings with config.save()
  3. Use polars for production: Better performance and memory efficiency
  4. Disable validation in production: Saves ~10-15% processing time if data quality is trusted
  5. Increase timeout for slow networks: Default 30s may not be enough on mobile/satellite connections
  6. Use ultra-cold start for single lap queries: Faster when only one lap’s telemetry is needed
  7. Configure the circuit breaker for each use case: Lower threshold for fail-fast, higher for resilience
  8. Use multiple CDNs for reliability: Configure fallback CDNs in production

Cache Management

The tif1 caching system is a performance component that reduces data loading times and network bandwidth usage. It implements a multi-layer caching strategy with both in-memory (LRU) and persistent (SQLite) storage.

Cache Architecture

The cache system consists of three layers:
  1. In-memory LRU cache: Fast access to recently used DataFrames (separate caches for pandas and polars)
  2. SQLite persistent cache: Disk-based storage for JSON responses and telemetry data
  3. Automatic cache warming: Prefetches commonly accessed data

get_cache

Returns the persistent SQLite cache singleton instance.
The Cache object manages the SQLite database that stores fetched JSON data and telemetry. Like Config, it is a singleton ensuring consistent cache state across the application. Returns:
  • Cache: Singleton cache instance
Thread Safety:
  • The Cache singleton is thread-safe
  • Uses connection pooling for concurrent access
  • Automatic transaction management with periodic commits
  • Safe for use in multi-threaded applications
Basic Example:

Cache Methods

get(key: str) -> Any | None

Retrieve raw cached data by key. Parameters:
  • key (str): Cache key (typically in format year/gp/session/file.json)
Returns:
  • Cached data (usually dict or list), or None if not found
Behavior:
  • Checks in-memory cache first (O(1) lookup)
  • Falls back to SQLite if not in memory (O(log n) lookup)
  • Automatically deserializes JSON data
  • Updates in-memory cache on SQLite hit
Example:

set(key: str, data: Any) -> None

Store data in cache. Parameters:
  • key (str): Cache key
  • data (Any): Data to cache (must be JSON-serializable)
Behavior:
  • Stores in both memory and SQLite
  • Automatically serializes to JSON
  • Commits to disk periodically (every 10 writes) or on close
  • Overwrites existing data for the same key
Example:

get_telemetry(year: int, gp: str, session: str, driver: str, lap: int) -> Any | None

Get cached telemetry data for a specific lap. Parameters:
  • year (int): Season year
  • gp (str): Grand Prix name (underscored format)
  • session (str): Session type (for example, “Race”, “Qualifying”)
  • driver (str): Driver code (for example, “VER”, “HAM”)
  • lap (int): Lap number
Returns:
  • Telemetry data dict, or None if not cached
Behavior:
  • Uses optimized key format: {year}/{gp}/{session}/telemetry/{driver}/{lap}
  • Checks memory cache first for ultra-fast access
  • Falls back to SQLite for persistent storage
  • Returns raw telemetry dict (not DataFrame)
Example:

set_telemetry(year: int, gp: str, session: str, driver: str, lap: int, data: Any) -> None

Store telemetry data in cache. Parameters:
  • year (int): Season year
  • gp (str): Grand Prix name (underscored format)
  • session (str): Session type
  • driver (str): Driver code
  • lap (int): Lap number
  • data (Any): Telemetry data to cache
Behavior:
  • Stores in both memory and SQLite
  • Uses optimized storage format
  • Automatically commits periodically
  • Enables fast subsequent access
Example:

get_telemetry_batch(year: int, gp: str, session: str, driver: str, laps: list[int]) -> dict[int, Any]

Get multiple laps of telemetry in a single operation. Parameters:
  • year (int): Season year
  • gp (str): Grand Prix name
  • session (str): Session type
  • driver (str): Driver code
  • laps (list[int]): List of lap numbers
Returns:
  • Dict mapping lap number to telemetry data (only includes cached laps)
Performance:
  • Single SQLite query for all laps (much faster than individual queries)
  • Optimized for bulk operations
  • Returns only cached laps (missing laps are omitted)
Example:

has_session_data(year: int, gp: str, session: str) -> bool

Check if cache contains data for a specific session. Parameters:
  • year (int): Season year
  • gp (str): Grand Prix name (underscored format)
  • session (str): Session type
Returns:
  • True if session data exists in cache, False otherwise
Behavior:
  • Checks for presence of key session files (drivers.json, laps.json)
  • Fast operation (index lookup only)
  • Useful for pre-flight checks before loading
Example:

clear() -> None

Remove all entries from both memory and SQLite cache. Behavior:
  • Clears in-memory LRU cache
  • Deletes all rows from SQLite tables
  • Vacuums database to reclaim disk space
  • Resets all cache statistics
  • Thread-safe operation
Example:
Use Cases:
  • Free disk space
  • Force re-fetch of all data
  • Clear corrupted cache entries
  • Reset after configuration changes

close() -> None

Close the SQLite database connection and flush pending writes. Behavior:
  • Commits any pending transactions
  • Closes database connection
  • Releases file locks
  • Should be called before application exit
Example:

Cache Performance Characteristics

Cache Size Management

The cache grows over time as more sessions are loaded. Here is how to manage cache size: Check Cache Size:
Typical Cache Sizes:
  • Single session: ~5-15 MB
  • Full season (all sessions): ~500-800 MB
  • Multiple seasons: ~2-5 GB
Cache Cleanup Strategies:

Async Cache Operations

The cache also supports async operations for use in async applications:

Cache Best Practices

  1. Let the cache warm up: First load is slow, subsequent loads are fast
  2. Do not clear cache unnecessarily: The cache improves performance on repeated loads
  3. Monitor cache size: Set up periodic cleanup if disk space is limited
  4. Use batch operations: get_telemetry_batch() is much faster than individual calls
  5. Close cache on exit: Use atexit.register(cache.close) to ensure clean shutdown
  6. Check cache before loading: Use has_session_data() to avoid unnecessary network calls
  7. Use SSD for cache: Faster than HDD for SQLite operations
  8. Consider cache location: Put cache on fast local storage, not network drives

Data Utilities

The data utilities provide convenient functions for converting between different time formats and safely navigating nested data structures. These are particularly useful when working with lap times, timestamps, and JSON responses.

to_timedelta

Convert various time formats to pandas Timedelta objects.
This function provides a unified interface for converting lap times, sector times, and other duration values into pandas Timedelta objects. The converted values support consistent time arithmetic and comparison. Parameters:
  • x (str | float | int | timedelta): Time value to convert
Supported Input Formats: Behavior:
  • Automatically adds leading "00:" to "M:SS.mmm" format
  • Handles negative times (for example, for deltas)
  • Preserves microsecond precision
  • Returns pandas Timedelta for consistent arithmetic
Basic Examples:
Time Arithmetic:
Working with DataFrames:
Edge Cases:

to_datetime

Convert string timestamps to pandas Timestamp objects.
This function wraps pandas’ to_datetime for consistent datetime parsing across the library. Parameters:
  • x (str): Datetime string to convert
Supported Formats:
  • ISO 8601: "2021-08-29T14:00:00"
  • Standard: "2021-08-29 14:00:00"
  • Date only: "2021-08-29"
  • With timezone: "2021-08-29T14:00:00+02:00"
Examples:
Working with Session Times:

recursive_dict_get

Safely navigate nested dictionaries without raising KeyError.
This function provides safe access to deeply nested dictionary values. It returns a default value instead of raising exceptions when keys do not exist. Parameters:
  • d (dict): Dictionary to navigate
  • *keys (str): Keys to traverse in order
  • default_none (bool, optional): If True, return None for missing keys; if False, return {}. Defaults to False.
Returns:
  • The value at the nested key path, or default value if any key is missing
Basic Examples:
Practical Use Cases:
Comparison with Standard Access:

delta_time

Calculate delta time between two laps (FastF1 compatibility function).
This function provides compatibility with FastF1’s delta_time utility for comparing lap performance. Parameters:
  • reference_lap (Lap): Reference lap object
  • compare_lap (Lap): Lap object to compare against reference
Returns:
  • Tuple of (delta_series, ref_telemetry, comp_telemetry)
    • delta_series: Time delta at each telemetry point
    • ref_telemetry: Reference lap telemetry
    • comp_telemetry: Comparison lap telemetry
Example:
Note: This is a compatibility function. For more advanced delta analysis, consider using the telemetry comparison features in the plotting module.

Reliability & Networking

The tif1 library implements reliability patterns to handle network failures, CDN outages, and transient errors gracefully. The circuit breaker pattern prevents cascading failures, while the CDN manager provides automatic fallback to alternative data sources.

Circuit Breaker Pattern

The circuit breaker is a critical reliability component. It stops the application from repeatedly trying operations that are likely to fail. It acts like an electrical circuit breaker, “opening” after too many failures to prevent cascading issues. Circuit Breaker States:
  1. Closed (Normal): All requests are allowed through
  2. Open (Failing): Requests are blocked immediately without attempting
  3. Half-Open (Testing): Limited requests allowed to test if service recovered
State Transitions:

get_circuit_breaker

Returns the global circuit breaker instance used for all network requests.
The circuit breaker is a singleton that tracks failures across all network operations in the application. Returns:
  • CircuitBreaker: Global circuit breaker instance
Thread Safety:
  • Fully thread-safe with atomic state transitions
  • Uses reentrant locks for nested calls
  • Safe for concurrent use across multiple threads
Basic Example:

CircuitBreaker Properties

state: str

Current circuit breaker state (read-only). Possible Values:
  • "closed": Normal operation, all requests allowed
  • "open": Too many failures, requests blocked
  • "half-open": Testing recovery, limited requests allowed
Example:

failures: int

Current failure count (read-only). Example:

threshold: int

Number of failures before circuit opens. Default: 5 (configurable via circuit_breaker_threshold config)

timeout: int

Seconds to wait before transitioning from open to half-open. Default: 60 (configurable via circuit_breaker_timeout config)

last_failure_time: datetime | None

Timestamp of the most recent failure. Example:

CircuitBreaker Methods

record_success() -> None

Record a successful operation (resets failure count). Behavior:
  • Resets failure counter to 0
  • Transitions from half-open to closed
  • Thread-safe atomic operation
Example:

record_failure() -> None

Record a failed operation (increments failure count). Behavior:
  • Atomically increments failure counter
  • Updates last failure timestamp
  • Opens circuit if threshold reached
  • Thread-safe operation
Example:

check_and_update_state() -> tuple[bool, str]

Check if request should proceed and update state if needed. Returns:
  • Tuple of (should_proceed, current_state)
Example:

reset_circuit_breaker

Reset the circuit breaker to initial closed state.
This function creates a new circuit breaker instance with configuration from the current config, effectively resetting all state. Behavior:
  • Creates new circuit breaker instance
  • Resets failure count to 0
  • Sets state to closed
  • Clears last failure time
  • Reloads configuration from config
Use Cases:
  • After resolving network issues
  • When switching networks
  • After configuration changes
  • For testing purposes
Example:

CDN Management

The CDN manager handles multiple CDN sources with automatic fallback, health tracking, and failure recovery.

get_cdn_manager

Returns the CDN manager responsible for CDN fallback and source health.
The CDN manager maintains a list of CDN sources and automatically falls back to alternatives when the primary CDN fails. Returns:
  • CDNManager: Global CDN manager instance
Basic Example:

CDNManager Methods

get_sources() -> list[CDNSource]

Get list of enabled CDN sources sorted by priority. Returns:
  • List of enabled CDN sources that have not exceeded failure threshold
Example:

mark_failure(source_name: str) -> None

Mark a CDN source as failed (increments failure count). Parameters:
  • source_name (str): Name of the CDN source
Behavior:
  • Increments failure counter for the source
  • Disables source if it reaches max failures (3)
  • Logs warning when source is disabled
Example:

mark_success(source_name: str) -> None

Mark a CDN source as successful (resets failure count). Parameters:
  • source_name (str): Name of the CDN source
Behavior:
  • Resets failure counter to 0
  • Re-enables source if it was disabled
Example:

reset() -> None

Reset all CDN failure counts. Behavior:
  • Resets all failure counters to 0
  • Re-enables all CDN sources
  • Useful after network issues are resolved
Example:

Reliability Examples

Handle Network Failures Gracefully

Monitor Circuit Breaker Health

CDN Fallback Testing

Production Error Handling

Reliability Best Practices

  1. Monitor circuit breaker state: Check state before critical operations
  2. Respect circuit breaker: Do not bypass it when open
  3. Reset after network changes: Call reset_circuit_breaker() when switching networks
  4. Configure thresholds appropriately: Lower for fail-fast, higher for resilience
  5. Use multiple CDNs: Configure fallback CDNs for production
  6. Log failures: Track patterns to identify systemic issues
  7. Implement exponential backoff: Do not retry immediately after failures
  8. Handle DataNotFoundError separately: Do not retry when data genuinely does not exist

Memory Management

The tif1 library uses in-memory LRU (Least Recently Used) caches to speed up repeated access to lap DataFrames. While this improves performance, it can consume significant memory when working with many sessions. These utilities help manage memory usage.

clear_lap_cache

Clear the in-memory LRU cache for lap DataFrames.
This function clears both the pandas and polars lap DataFrame caches, freeing memory without affecting the persistent SQLite cache. Behavior:
  • Clears pandas lap cache
  • Clears polars lap cache
  • Does NOT affect SQLite cache (persistent storage)
  • Immediate effect on memory usage
  • Thread-safe operation
When to Use:
  • After processing large sessions
  • When switching between different analyses
  • To free memory in long-running applications
  • Before loading a different season’s data
Example:
Memory Impact Example:
Typical Memory Usage:
  • Single session lap DataFrame: ~5-15 MB
  • Full season in cache: ~200-500 MB
  • Clearing cache can free 50-80% of lap data memory
Best Practices:
  1. Clear cache between unrelated analyses
  2. Clear cache in batch processing loops
  3. Do not clear too frequently (defeats caching purpose)
  4. Monitor memory usage in long-running applications
  5. Consider clearing cache before loading different seasons
Advanced: Selective Cache Management:

Complete Examples

This section provides examples demonstrating how to combine multiple utilities for real-world use cases.

Example 1: Configure and Save Settings

Set up the configuration for the specific use case and persist it.

Example 2: Debug Session Loading

Enable detailed logging to diagnose data loading issues.

Example 3: Cache Management Workflow

Complete cache inspection and management.

Example 4: Handle Network Issues

Reliable error handling with circuit breaker awareness.

Example 5: Custom Configuration File

Create project-specific configuration.

Example 6: Performance Optimization

Optimize configuration for maximum performance.

Example 7: Memory-Efficient Batch Processing

Process multiple sessions while managing memory usage.

Best Practices

Guidelines for using tif1 utilities effectively in different scenarios.

Development Best Practices

  1. Enable Debug Logging During Development
    • Helps understand data flow and identify issues
    • Shows cache hits/misses for optimization
    • Reveals network request patterns
    • Displays parsing and validation steps
  2. Use Pandas for Easier Debugging
    • More familiar API for most developers
    • Better error messages
    • Easier to inspect intermediate results
    • Switch to polars for production
  3. Keep Validation Enabled
    • Catches data quality issues early
    • Provides clear error messages
    • Helps identify API changes
    • ~10-15% overhead is acceptable in development
  4. Use Shorter Timeouts
    • Fail fast during development
    • Do not wait for slow connections
    • Iterate quickly on code changes
  5. Clear Cache When Testing
    • Ensures that tests run with fresh data
    • Validates cache warming logic
    • Catches cache-related bugs

Production Best Practices

  1. Use Polars for Maximum Performance
    • 2-5x faster DataFrame operations
    • Better memory efficiency
    • Lower CPU usage
    • Handles larger datasets
  2. Disable Validation in Production
    • Saves ~10-15% processing time
    • Reduces CPU usage
    • Only disable if data quality is trusted
    • Keep enabled if data source is unreliable
  3. Increase Timeout for Reliability
    • Handle slow networks gracefully
    • Accommodate mobile/satellite connections
    • Reduce failure rate
    • Balance with user experience
  4. Configure Circuit Breaker Appropriately
    • Higher threshold for production resilience
    • Longer timeout for recovery
    • Prevents cascading failures
    • Monitor circuit breaker state
  5. Use Multiple CDNs
    • Automatic fallback on CDN failure
    • Improved reliability
    • Geographic redundancy
    • Load distribution
  6. Enable CDN Minification
    • Reduces bandwidth by 20-40%
    • Faster downloads
    • Lower data costs
    • Minimal CPU overhead
  7. Set Up Proper Logging
    • Monitor operational health
    • Track performance metrics
    • Debug production issues
    • Audit data access
  8. Close Cache on Exit
    • Ensures data is flushed to disk
    • Prevents corruption
    • Releases file locks
    • Clean shutdown

Performance Optimization

  1. Use Ultra Cold Start for Single Lap Queries
    • Faster for single lap telemetry
    • Skips loading full session data
    • On-demand fetching
    • Not suitable for full session analysis
  2. Clear Lap Cache Periodically
    • Prevents memory bloat
    • Important for long-running applications
    • Clear between unrelated analyses
    • Do not clear too frequently
  3. Use Batch Telemetry Operations
    • Single database query vs multiple
    • Much faster for multiple laps
    • Reduces I/O overhead
  4. Monitor Cache Size
    • Prevent disk space issues
    • Set up monitoring alerts
    • Implement automatic cleanup
    • Archive old seasons
  5. Use SSD for Cache
    • Faster than HDD
    • Reduces SQLite query time
    • Better for concurrent access
    • Avoid network drives

Error Handling

  1. Distinguish Between Error Types
    • Do not retry DataNotFoundError
    • Retry NetworkError with backoff
    • Handle each error type appropriately
  2. Monitor Circuit Breaker State
    • Track circuit breaker state
    • Alert on open state
    • Monitor failure patterns
    • Investigate root causes
  3. Implement Exponential Backoff
    • Do not retry immediately
    • Give service time to recover
    • Reduce load on failing service
    • Increase wait time exponentially
  4. Log Errors with Context
    • Include relevant context
    • Track failure patterns
    • Enable debugging
    • Support monitoring

Memory Management

  1. Clear Cache in Batch Processing
    • Prevent memory bloat
    • Clear every N iterations
    • Balance performance vs memory
    • Monitor memory usage
  2. Use Context Managers for Cache
    • Ensures proper cleanup
    • Prevents resource leaks
    • Clean shutdown
  3. Monitor Memory Usage
    • Set memory thresholds
    • Automatic cleanup
    • Prevent OOM errors
    • Log memory metrics

Configuration Management

  1. Use Environment Variables for Deployment
    • Easy to configure across environments
    • No code changes needed
    • Supports containerization
    • Environment-specific settings
  2. Save Configuration After Tuning
    • Persist optimized settings
    • Share configuration across team
    • Version control config files
    • Document configuration choices
  3. Use Project-Specific Configuration
    • Isolate project caches
    • Different settings per project
    • Easier to manage
    • Portable configuration

Testing

  1. Clear Cache Before Tests
    • Ensures test isolation
    • Prevents cache pollution
    • Reproducible tests
    • Catches cache bugs
  2. Mock Network Calls
    • Faster tests
    • No network dependency
    • Predictable results
    • Test error handling
  3. Test Circuit Breaker Behavior
    • Verify reliability patterns
    • Test failure scenarios
    • Ensure proper recovery
    • Validate configuration

Monitoring and Observability

  1. Track Performance Metrics
    • Monitor load times
    • Track cache hit rates
    • Identify performance regressions
    • Optimize based on data
  2. Set Up Health Checks
    • Regular health checks
    • Alert on issues
    • Track system state
    • Support debugging
  3. Implement Structured Logging
    • Machine-readable logs
    • Easy to parse and analyze
    • Better for log aggregation
    • Supports monitoring tools

Summary

Follow these best practices to:
  • Build reliable applications with proper error handling
  • Optimize performance for the specific use case
  • Manage resources efficiently
  • Debug issues quickly
  • Deploy confidently to production
  • Monitor system health effectively
Remember: The right configuration depends on the specific use case. Development, production, and testing environments should have different configurations optimized for their respective goals.
Last modified on September 3, 2026