tif1 provides utility functions. They manage internal state, performance optimization, reliability layers, and system configuration. These utilities give fine-grained control over caching behavior, network resilience, logging verbosity, data processing pipelines, and more.
This guide is an exhaustive reference for all utility functions. It covers parameters, return values, edge cases, performance implications, thread safety, and real-world usage patterns. Use these utilities when debugging data loading issues, optimizing cache performance, or building production applications.
Overview
The utilities intif1 are organized into several categories:
- Logging: Control diagnostic output verbosity
- Configuration: Manage global settings and preferences
- Cache Management: Control persistent and in-memory caching
- Data Utilities: Convert and manipulate time and data structures
- Reliability & Networking: Manage circuit breakers and CDN fallback
- Memory Management: Clear caches to free resources
Logging
Logging is essential for debugging data loading issues, understanding cache behavior, and diagnosing network problems. Thetif1 library uses Python’s standard logging module and provides a convenient setup function to control verbosity across all internal components.
setup_logging
Configure the logging level for all tif1 components.
tif1 logger hierarchy. It also suppresses noisy connection pool warnings from the underlying HTTP library.
Parameters:
level(int): Standard logging level from Python’sloggingmodule. Defaults tologging.WARNING.
Behavior:
- Configures logging format:
%(asctime)s - %(name)s - %(levelname)s - %(message)s - Sets date format:
%Y-%m-%d %H:%M:%S - Applies to all
tif1.*loggers (core, cache, http, retry, cdn, etc.) - Suppresses
urllib3_future.connectionpoolwarnings (set to ERROR level) - Thread-safe: Can be called from any thread
- DEBUG logging adds ~5-10% overhead due to string formatting
- INFO logging adds ~1-2% overhead
- WARNING and above have negligible performance impact
- Logging is buffered and asynchronous in most Python implementations
- Use DEBUG during development to understand data flow
- Use INFO in production for operational visibility
- Use WARNING (default) for minimal overhead
- Disable logging in performance-critical tight loops
- Consider using a logging handler that writes to files for production
Configuration
Thetif1 configuration system provides a flexible way to customize library behavior through environment variables, configuration files, or programmatic API calls. Configuration is managed through a singleton Config object that loads settings from multiple sources with a defined precedence order.
Configuration Loading Order
Settings are loaded in the following order (later sources override earlier ones):- Built-in defaults: Hardcoded sensible defaults
- Configuration file (
~/.tif1rc): JSON file in user’s home directory - Environment variables: Prefixed with
TIF1_(for example,TIF1_LIB=polars) - Programmatic calls: Using
config.set()at runtime
get_config
Returns the global configuration singleton instance.
Config object is a singleton, meaning all calls to get_config() return the same instance. This ensures configuration consistency across the entire application.
Returns:
Config: Singleton configuration instance
- The Config singleton is thread-safe
- Multiple threads can safely call
get()andset() - File operations (
save()) use atomic writes
Config Methods
get(key: str, default: Any = None) -> Any
Retrieve a configuration value by key.
Parameters:
key(str): Configuration key namedefault(Any, optional): Value to return if key does not exist. Defaults toNone.
- The configuration value, or
defaultif key not found
set(key: str, value: Any) -> None
Update a configuration value in memory.
Parameters:
key(str): Configuration key namevalue(Any): New value to set
- Changes are immediate and affect all subsequent operations
- Changes are in-memory only until
save()is called - Type conversion is automatic for known keys
- Unknown keys are stored as-is
save(path: Path | str | None = None) -> None
Save current configuration to a JSON file.
Parameters:
path(Path | str | None, optional): File path to save to. IfNone, saves to~/.tif1rc. Defaults toNone.
- Creates parent directories if they do not exist
- Uses atomic write (write to temp file, then rename)
- Saves all current settings, including defaults
- File format is human-readable JSON
Available Configuration Keys
Reference of all configuration options:Environment Variable Configuration
All configuration keys can be set via environment variables using theTIF1_ prefix:
- Booleans:
"true","1","yes"→True;"false","0","no"→False - Integers: Parsed automatically
- Lists: Comma-separated values (for example,
"url1,url2,url3")
Configuration Examples
Development Configuration
Production Configuration
Minimal Latency Configuration
Custom Cache Location
Configuration Best Practices
- Use environment variables for deployment: Easier to configure across environments without code changes
- Save configuration after tuning: Persist the optimized settings with
config.save() - Use polars for production: Better performance and memory efficiency
- Disable validation in production: Saves ~10-15% processing time if data quality is trusted
- Increase timeout for slow networks: Default 30s may not be enough on mobile/satellite connections
- Use ultra-cold start for single lap queries: Faster when only one lap’s telemetry is needed
- Configure the circuit breaker for each use case: Lower threshold for fail-fast, higher for resilience
- Use multiple CDNs for reliability: Configure fallback CDNs in production
Cache Management
Thetif1 caching system is a performance component that reduces data loading times and network bandwidth usage. It implements a multi-layer caching strategy with both in-memory (LRU) and persistent (SQLite) storage.
Cache Architecture
The cache system consists of three layers:- In-memory LRU cache: Fast access to recently used DataFrames (separate caches for pandas and polars)
- SQLite persistent cache: Disk-based storage for JSON responses and telemetry data
- Automatic cache warming: Prefetches commonly accessed data
get_cache
Returns the persistent SQLite cache singleton instance.
Cache object manages the SQLite database that stores fetched JSON data and telemetry. Like Config, it is a singleton ensuring consistent cache state across the application.
Returns:
Cache: Singleton cache instance
- The Cache singleton is thread-safe
- Uses connection pooling for concurrent access
- Automatic transaction management with periodic commits
- Safe for use in multi-threaded applications
Cache Methods
get(key: str) -> Any | None
Retrieve raw cached data by key.
Parameters:
key(str): Cache key (typically in formatyear/gp/session/file.json)
- Cached data (usually dict or list), or
Noneif not found
- Checks in-memory cache first (O(1) lookup)
- Falls back to SQLite if not in memory (O(log n) lookup)
- Automatically deserializes JSON data
- Updates in-memory cache on SQLite hit
set(key: str, data: Any) -> None
Store data in cache.
Parameters:
key(str): Cache keydata(Any): Data to cache (must be JSON-serializable)
- Stores in both memory and SQLite
- Automatically serializes to JSON
- Commits to disk periodically (every 10 writes) or on close
- Overwrites existing data for the same key
get_telemetry(year: int, gp: str, session: str, driver: str, lap: int) -> Any | None
Get cached telemetry data for a specific lap.
Parameters:
year(int): Season yeargp(str): Grand Prix name (underscored format)session(str): Session type (for example, “Race”, “Qualifying”)driver(str): Driver code (for example, “VER”, “HAM”)lap(int): Lap number
- Telemetry data dict, or
Noneif not cached
- Uses optimized key format:
{year}/{gp}/{session}/telemetry/{driver}/{lap} - Checks memory cache first for ultra-fast access
- Falls back to SQLite for persistent storage
- Returns raw telemetry dict (not DataFrame)
set_telemetry(year: int, gp: str, session: str, driver: str, lap: int, data: Any) -> None
Store telemetry data in cache.
Parameters:
year(int): Season yeargp(str): Grand Prix name (underscored format)session(str): Session typedriver(str): Driver codelap(int): Lap numberdata(Any): Telemetry data to cache
- Stores in both memory and SQLite
- Uses optimized storage format
- Automatically commits periodically
- Enables fast subsequent access
get_telemetry_batch(year: int, gp: str, session: str, driver: str, laps: list[int]) -> dict[int, Any]
Get multiple laps of telemetry in a single operation.
Parameters:
year(int): Season yeargp(str): Grand Prix namesession(str): Session typedriver(str): Driver codelaps(list[int]): List of lap numbers
- Dict mapping lap number to telemetry data (only includes cached laps)
- Single SQLite query for all laps (much faster than individual queries)
- Optimized for bulk operations
- Returns only cached laps (missing laps are omitted)
has_session_data(year: int, gp: str, session: str) -> bool
Check if cache contains data for a specific session.
Parameters:
year(int): Season yeargp(str): Grand Prix name (underscored format)session(str): Session type
Trueif session data exists in cache,Falseotherwise
- Checks for presence of key session files (drivers.json, laps.json)
- Fast operation (index lookup only)
- Useful for pre-flight checks before loading
clear() -> None
Remove all entries from both memory and SQLite cache.
Behavior:
- Clears in-memory LRU cache
- Deletes all rows from SQLite tables
- Vacuums database to reclaim disk space
- Resets all cache statistics
- Thread-safe operation
- Free disk space
- Force re-fetch of all data
- Clear corrupted cache entries
- Reset after configuration changes
close() -> None
Close the SQLite database connection and flush pending writes.
Behavior:
- Commits any pending transactions
- Closes database connection
- Releases file locks
- Should be called before application exit
Cache Performance Characteristics
Cache Size Management
The cache grows over time as more sessions are loaded. Here is how to manage cache size: Check Cache Size:- Single session: ~5-15 MB
- Full season (all sessions): ~500-800 MB
- Multiple seasons: ~2-5 GB
Async Cache Operations
The cache also supports async operations for use in async applications:Cache Best Practices
- Let the cache warm up: First load is slow, subsequent loads are fast
- Do not clear cache unnecessarily: The cache improves performance on repeated loads
- Monitor cache size: Set up periodic cleanup if disk space is limited
- Use batch operations:
get_telemetry_batch()is much faster than individual calls - Close cache on exit: Use
atexit.register(cache.close)to ensure clean shutdown - Check cache before loading: Use
has_session_data()to avoid unnecessary network calls - Use SSD for cache: Faster than HDD for SQLite operations
- Consider cache location: Put cache on fast local storage, not network drives
Data Utilities
The data utilities provide convenient functions for converting between different time formats and safely navigating nested data structures. These are particularly useful when working with lap times, timestamps, and JSON responses.to_timedelta
Convert various time formats to pandas Timedelta objects.
Timedelta objects. The converted values support consistent time arithmetic and comparison.
Parameters:
x(str | float | int | timedelta): Time value to convert
Behavior:
- Automatically adds leading
"00:"to"M:SS.mmm"format - Handles negative times (for example, for deltas)
- Preserves microsecond precision
- Returns pandas
Timedeltafor consistent arithmetic
to_datetime
Convert string timestamps to pandas Timestamp objects.
to_datetime for consistent datetime parsing across the library.
Parameters:
x(str): Datetime string to convert
- ISO 8601:
"2021-08-29T14:00:00" - Standard:
"2021-08-29 14:00:00" - Date only:
"2021-08-29" - With timezone:
"2021-08-29T14:00:00+02:00"
recursive_dict_get
Safely navigate nested dictionaries without raising KeyError.
d(dict): Dictionary to navigate*keys(str): Keys to traverse in orderdefault_none(bool, optional): IfTrue, returnNonefor missing keys; ifFalse, return{}. Defaults toFalse.
- The value at the nested key path, or default value if any key is missing
delta_time
Calculate delta time between two laps (FastF1 compatibility function).
delta_time utility for comparing lap performance.
Parameters:
reference_lap(Lap): Reference lap objectcompare_lap(Lap): Lap object to compare against reference
- Tuple of
(delta_series, ref_telemetry, comp_telemetry)delta_series: Time delta at each telemetry pointref_telemetry: Reference lap telemetrycomp_telemetry: Comparison lap telemetry
Reliability & Networking
Thetif1 library implements reliability patterns to handle network failures, CDN outages, and transient errors gracefully. The circuit breaker pattern prevents cascading failures, while the CDN manager provides automatic fallback to alternative data sources.
Circuit Breaker Pattern
The circuit breaker is a critical reliability component. It stops the application from repeatedly trying operations that are likely to fail. It acts like an electrical circuit breaker, “opening” after too many failures to prevent cascading issues. Circuit Breaker States:- Closed (Normal): All requests are allowed through
- Open (Failing): Requests are blocked immediately without attempting
- Half-Open (Testing): Limited requests allowed to test if service recovered
get_circuit_breaker
Returns the global circuit breaker instance used for all network requests.
CircuitBreaker: Global circuit breaker instance
- Fully thread-safe with atomic state transitions
- Uses reentrant locks for nested calls
- Safe for concurrent use across multiple threads
CircuitBreaker Properties
state: str
Current circuit breaker state (read-only).
Possible Values:
"closed": Normal operation, all requests allowed"open": Too many failures, requests blocked"half-open": Testing recovery, limited requests allowed
failures: int
Current failure count (read-only).
Example:
threshold: int
Number of failures before circuit opens.
Default: 5 (configurable via circuit_breaker_threshold config)
timeout: int
Seconds to wait before transitioning from open to half-open.
Default: 60 (configurable via circuit_breaker_timeout config)
last_failure_time: datetime | None
Timestamp of the most recent failure.
Example:
CircuitBreaker Methods
record_success() -> None
Record a successful operation (resets failure count).
Behavior:
- Resets failure counter to 0
- Transitions from half-open to closed
- Thread-safe atomic operation
record_failure() -> None
Record a failed operation (increments failure count).
Behavior:
- Atomically increments failure counter
- Updates last failure timestamp
- Opens circuit if threshold reached
- Thread-safe operation
check_and_update_state() -> tuple[bool, str]
Check if request should proceed and update state if needed.
Returns:
- Tuple of
(should_proceed, current_state)
reset_circuit_breaker
Reset the circuit breaker to initial closed state.
- Creates new circuit breaker instance
- Resets failure count to 0
- Sets state to closed
- Clears last failure time
- Reloads configuration from config
- After resolving network issues
- When switching networks
- After configuration changes
- For testing purposes
CDN Management
The CDN manager handles multiple CDN sources with automatic fallback, health tracking, and failure recovery.get_cdn_manager
Returns the CDN manager responsible for CDN fallback and source health.
CDNManager: Global CDN manager instance
CDNManager Methods
get_sources() -> list[CDNSource]
Get list of enabled CDN sources sorted by priority.
Returns:
- List of enabled CDN sources that have not exceeded failure threshold
mark_failure(source_name: str) -> None
Mark a CDN source as failed (increments failure count).
Parameters:
source_name(str): Name of the CDN source
- Increments failure counter for the source
- Disables source if it reaches max failures (3)
- Logs warning when source is disabled
mark_success(source_name: str) -> None
Mark a CDN source as successful (resets failure count).
Parameters:
source_name(str): Name of the CDN source
- Resets failure counter to 0
- Re-enables source if it was disabled
reset() -> None
Reset all CDN failure counts.
Behavior:
- Resets all failure counters to 0
- Re-enables all CDN sources
- Useful after network issues are resolved
Reliability Examples
Handle Network Failures Gracefully
Monitor Circuit Breaker Health
CDN Fallback Testing
Production Error Handling
Reliability Best Practices
- Monitor circuit breaker state: Check state before critical operations
- Respect circuit breaker: Do not bypass it when open
- Reset after network changes: Call
reset_circuit_breaker()when switching networks - Configure thresholds appropriately: Lower for fail-fast, higher for resilience
- Use multiple CDNs: Configure fallback CDNs for production
- Log failures: Track patterns to identify systemic issues
- Implement exponential backoff: Do not retry immediately after failures
- Handle DataNotFoundError separately: Do not retry when data genuinely does not exist
Memory Management
Thetif1 library uses in-memory LRU (Least Recently Used) caches to speed up repeated access to lap DataFrames. While this improves performance, it can consume significant memory when working with many sessions. These utilities help manage memory usage.
clear_lap_cache
Clear the in-memory LRU cache for lap DataFrames.
- Clears pandas lap cache
- Clears polars lap cache
- Does NOT affect SQLite cache (persistent storage)
- Immediate effect on memory usage
- Thread-safe operation
- After processing large sessions
- When switching between different analyses
- To free memory in long-running applications
- Before loading a different season’s data
- Single session lap DataFrame: ~5-15 MB
- Full season in cache: ~200-500 MB
- Clearing cache can free 50-80% of lap data memory
- Clear cache between unrelated analyses
- Clear cache in batch processing loops
- Do not clear too frequently (defeats caching purpose)
- Monitor memory usage in long-running applications
- Consider clearing cache before loading different seasons
Complete Examples
This section provides examples demonstrating how to combine multiple utilities for real-world use cases.Example 1: Configure and Save Settings
Set up the configuration for the specific use case and persist it.Example 2: Debug Session Loading
Enable detailed logging to diagnose data loading issues.Example 3: Cache Management Workflow
Complete cache inspection and management.Example 4: Handle Network Issues
Reliable error handling with circuit breaker awareness.Example 5: Custom Configuration File
Create project-specific configuration.Example 6: Performance Optimization
Optimize configuration for maximum performance.Example 7: Memory-Efficient Batch Processing
Process multiple sessions while managing memory usage.Best Practices
Guidelines for usingtif1 utilities effectively in different scenarios.
Development Best Practices
-
Enable Debug Logging During Development
- Helps understand data flow and identify issues
- Shows cache hits/misses for optimization
- Reveals network request patterns
- Displays parsing and validation steps
-
Use Pandas for Easier Debugging
- More familiar API for most developers
- Better error messages
- Easier to inspect intermediate results
- Switch to polars for production
-
Keep Validation Enabled
- Catches data quality issues early
- Provides clear error messages
- Helps identify API changes
- ~10-15% overhead is acceptable in development
-
Use Shorter Timeouts
- Fail fast during development
- Do not wait for slow connections
- Iterate quickly on code changes
-
Clear Cache When Testing
- Ensures that tests run with fresh data
- Validates cache warming logic
- Catches cache-related bugs
Production Best Practices
-
Use Polars for Maximum Performance
- 2-5x faster DataFrame operations
- Better memory efficiency
- Lower CPU usage
- Handles larger datasets
-
Disable Validation in Production
- Saves ~10-15% processing time
- Reduces CPU usage
- Only disable if data quality is trusted
- Keep enabled if data source is unreliable
-
Increase Timeout for Reliability
- Handle slow networks gracefully
- Accommodate mobile/satellite connections
- Reduce failure rate
- Balance with user experience
-
Configure Circuit Breaker Appropriately
- Higher threshold for production resilience
- Longer timeout for recovery
- Prevents cascading failures
- Monitor circuit breaker state
-
Use Multiple CDNs
- Automatic fallback on CDN failure
- Improved reliability
- Geographic redundancy
- Load distribution
-
Enable CDN Minification
- Reduces bandwidth by 20-40%
- Faster downloads
- Lower data costs
- Minimal CPU overhead
-
Set Up Proper Logging
- Monitor operational health
- Track performance metrics
- Debug production issues
- Audit data access
-
Close Cache on Exit
- Ensures data is flushed to disk
- Prevents corruption
- Releases file locks
- Clean shutdown
Performance Optimization
-
Use Ultra Cold Start for Single Lap Queries
- Faster for single lap telemetry
- Skips loading full session data
- On-demand fetching
- Not suitable for full session analysis
-
Clear Lap Cache Periodically
- Prevents memory bloat
- Important for long-running applications
- Clear between unrelated analyses
- Do not clear too frequently
-
Use Batch Telemetry Operations
- Single database query vs multiple
- Much faster for multiple laps
- Reduces I/O overhead
-
Monitor Cache Size
- Prevent disk space issues
- Set up monitoring alerts
- Implement automatic cleanup
- Archive old seasons
-
Use SSD for Cache
- Faster than HDD
- Reduces SQLite query time
- Better for concurrent access
- Avoid network drives
Error Handling
-
Distinguish Between Error Types
- Do not retry DataNotFoundError
- Retry NetworkError with backoff
- Handle each error type appropriately
-
Monitor Circuit Breaker State
- Track circuit breaker state
- Alert on open state
- Monitor failure patterns
- Investigate root causes
-
Implement Exponential Backoff
- Do not retry immediately
- Give service time to recover
- Reduce load on failing service
- Increase wait time exponentially
-
Log Errors with Context
- Include relevant context
- Track failure patterns
- Enable debugging
- Support monitoring
Memory Management
-
Clear Cache in Batch Processing
- Prevent memory bloat
- Clear every N iterations
- Balance performance vs memory
- Monitor memory usage
-
Use Context Managers for Cache
- Ensures proper cleanup
- Prevents resource leaks
- Clean shutdown
-
Monitor Memory Usage
- Set memory thresholds
- Automatic cleanup
- Prevent OOM errors
- Log memory metrics
Configuration Management
-
Use Environment Variables for Deployment
- Easy to configure across environments
- No code changes needed
- Supports containerization
- Environment-specific settings
-
Save Configuration After Tuning
- Persist optimized settings
- Share configuration across team
- Version control config files
- Document configuration choices
-
Use Project-Specific Configuration
- Isolate project caches
- Different settings per project
- Easier to manage
- Portable configuration
Testing
-
Clear Cache Before Tests
- Ensures test isolation
- Prevents cache pollution
- Reproducible tests
- Catches cache bugs
-
Mock Network Calls
- Faster tests
- No network dependency
- Predictable results
- Test error handling
-
Test Circuit Breaker Behavior
- Verify reliability patterns
- Test failure scenarios
- Ensure proper recovery
- Validate configuration
Monitoring and Observability
-
Track Performance Metrics
- Monitor load times
- Track cache hit rates
- Identify performance regressions
- Optimize based on data
-
Set Up Health Checks
- Regular health checks
- Alert on issues
- Track system state
- Support debugging
-
Implement Structured Logging
- Machine-readable logs
- Easy to parse and analyze
- Better for log aggregation
- Supports monitoring tools
Summary
Follow these best practices to:- Build reliable applications with proper error handling
- Optimize performance for the specific use case
- Manage resources efficiently
- Debug issues quickly
- Deploy confidently to production
- Monitor system health effectively