tif1 provides utility functions. They manage internal state, performance optimization, reliability layers, and system configuration. These utilities give fine-grained control over caching behavior, network resilience, logging verbosity, data processing pipelines, and more.
This guide is an exhaustive reference for all utility functions. It covers parameters, return values, edge cases, performance implications, thread safety, and real-world usage patterns. Use these utilities when debugging data loading issues, optimizing cache performance, or building production applications.
Overview
The utilities intif1 are organized into several categories:
- Logging: Control diagnostic output verbosity
- Configuration: Manage global settings and preferences
- Cache Management: Control persistent and in-memory caching
- Data Utilities: Convert and manipulate time and data structures
- Reliability & Networking: Manage circuit breakers and CDN fallback
- Memory Management: Clear caches to free resources
Logging
Logging is essential for debugging data loading issues, understanding cache behavior, and diagnosing network problems. Thetif1 library uses Python’s standard logging module and provides a convenient setup function to control verbosity across all internal components.
setup_logging
Configure the logging level for all tif1 components.
tif1 logger hierarchy. It also suppresses noisy connection pool warnings from the underlying HTTP library.
Parameters:
level(int): Standard logging level from Python’sloggingmodule. Defaults tologging.WARNING.
Behavior:
- Configures logging format:
%(asctime)s - %(name)s - %(levelname)s - %(message)s - Sets date format:
%Y-%m-%d %H:%M:%S - Applies to all
tif1.*loggers (tif1.core,tif1.cache,tif1.async_fetch,tif1.http_session,tif1.cdn,tif1.retry, and others) - Suppresses
urllib3_future.connectionpoolwarnings (set to ERROR level) - Thread-safe: Can be called from any thread
- DEBUG and INFO logging format and write a record for each cache lookup and fetch step
- WARNING and above log only problems, so they add almost no work
- Python logging is synchronous: each record is written in the thread that logs it
- Use DEBUG during development to understand data flow
- Use INFO in production for operational visibility
- Use WARNING (default) for minimal overhead
- Disable logging in performance-critical tight loops
- Consider using a logging handler that writes to files for production
Configuration
Thetif1 configuration system provides a flexible way to customize library behavior through environment variables, configuration files, or programmatic API calls. Configuration is managed through a singleton Config object that loads settings from multiple sources with a defined precedence order.
Configuration Loading Order
Settings are loaded in the following order (later sources override earlier ones):- Built-in defaults: Hardcoded sensible defaults
- Configuration file (
~/.tif1rc): JSON file in user’s home directory - Environment variables: Prefixed with
TIF1_(for example,TIF1_LIB=polars) - Programmatic calls: Using
config.set()at runtime
get_config
Returns the global configuration singleton instance.
Config object is a singleton, meaning all calls to get_config() return the same instance. This ensures configuration consistency across the entire application.
Returns:
Config: Singleton configuration instance
get()andset()are plain dictionary operations without a lock- Set the configuration once at start-up, before worker threads start
save()writes the file in place; it is not an atomic write
Config Methods
get(key: str, default: Any = None) -> Any
Retrieve a configuration value by key.
Parameters:
key(str): Configuration key namedefault(Any, optional): Value to return if key does not exist. Defaults toNone.
- The configuration value, or
defaultif key not found
set(key: str, value: Any) -> None
Update a configuration value in memory.
Parameters:
key(str): Configuration key namevalue(Any): New value to set
- Most settings take effect on the next operation that reads them
- Some components read their settings once, when they are created. The CDN manager reads
cdnsandcdn_use_minificationwhentif1.cdnis first imported;get_cdn_manager().reload()applies a change. The circuit breaker applies new values afterreset_circuit_breaker() - Changes are in-memory only until
save()is called set()does no type conversion: pass the type that the key expects (for example, anintfortimeout)- Unknown keys are stored as-is
save(path: Path | str | None = None) -> None
Save current configuration to a JSON file.
Parameters:
path(Path | str | None, optional): File path to save to. IfNone, saves to~/.tif1rc. Defaults toNone.
- Does not create parent directories: the target directory must exist
- Writes the file in place (not an atomic write)
- Logs an error instead of raising an exception when the write fails
- Saves all current settings, including defaults
- File format is human-readable JSON
Available Configuration Keys
Reference of all configuration options:Environment Variable Configuration
Most configuration keys can be set via environment variables using theTIF1_ prefix:
- Booleans:
"true","1","yes","on"→True;"false","0","no","off"→False - Integers: Parsed automatically
- Lists: Comma-separated values (for example,
"url1,url2,url3")
Configuration Examples
Development Configuration
Production Configuration
Minimal Latency Configuration
Custom Cache Location
Configuration Best Practices
- Use environment variables for deployment: Easier to configure across environments without code changes
- Save configuration after tuning: Persist the optimized settings with
config.save() - Use polars when the analysis code uses polars: Laps and telemetry then arrive as polars frames
- Keep validation off in production:
validate_datais off by default; turn it on to check untrusted data - Increase timeout for slow networks: Default 30s may not be enough on mobile/satellite connections
- Keep ultra-cold start on: It makes the first load of an uncached session faster
- Configure the circuit breaker for each use case: Lower threshold for fail-fast, higher for resilience
- Use multiple CDNs for reliability: Configure fallback CDNs in production
Cache Management
Thetif1 caching system is a performance component that reduces data loading times and network bandwidth usage. It implements a multi-layer caching strategy with both in-memory (LRU) and persistent (SQLite) storage.
Cache Architecture
The cache system consists of three layers:- In-memory LRU tiers of the
Cacheobject: Raw and parsed JSON and telemetry payloads - SQLite persistent cache: Disk-based storage for JSON responses, telemetry payloads and built DataFrames
- Process-global lap caches: One LRU of lap DataFrames for pandas and one for polars (see
clear_lap_cache())
get_cache
Returns the persistent SQLite cache singleton instance.
Cache object manages the SQLite database that stores fetched JSON data and telemetry. Like Config, it is a singleton ensuring consistent cache state across the application.
Returns:
Cache: Singleton cache instance
- The Cache singleton is thread-safe
- Uses one SQLite connection (WAL mode) guarded by a lock
- Automatic transaction management with periodic commits
- Safe for use in multi-threaded applications
Cache Methods
get(key: str) -> Any | None
Retrieve raw cached data by key.
Parameters:
key(str): Cache key. Session files use{year}/{gp}/{session}/{file}.jsonwith URL-encoded names, for example2025/Monaco%20Grand%20Prix/Race/drivers.json
- Cached data (usually dict or list), or
Noneif not found
- Checks in-memory cache first (O(1) lookup)
- Falls back to SQLite if not in memory (O(log n) lookup)
- Automatically deserializes JSON data
- Updates in-memory cache on SQLite hit
set(key: str, data: Any) -> None
Store data in cache.
Parameters:
key(str): Cache keydata(Any): Data to cache (must be JSON-serializable)
- Stores in both memory and SQLite
- Automatically serializes to JSON
- Commits to disk periodically (every
cache_commit_intervalwrites, default 100) or on close - Overwrites existing data for the same key
get_telemetry(year: int, gp: str, session: str, driver: str, lap: int) -> Any | None
Get cached telemetry data for a specific lap.
Parameters:
year(int): Season yeargp(str): Grand Prix name, URL-encoded as insession.gp(for example,"Belgian%20Grand%20Prix")session(str): Session name, URL-encoded as insession.session(for example,"Race","Practice%201")driver(str): Driver code (for example, “VER”, “HAM”)lap(int): Lap number
- Raw telemetry payload (a dict of channel lists such as
time,speed,rpm), orNoneif not cached
- Keys entries on the tuple
(year, gp, session, driver, lap) - Checks memory cache first for ultra-fast access
- Falls back to SQLite for persistent storage
- Returns the raw payload dict (not a DataFrame)
set_telemetry(year: int, gp: str, session: str, driver: str, lap: int, data: Any) -> None
Store telemetry data in cache.
Parameters:
year(int): Season yeargp(str): Grand Prix name, URL-encoded as insession.gpsession(str): Session name, URL-encoded as insession.sessiondriver(str): Driver codelap(int): Lap numberdata(Any): Telemetry data to cache
- Stores in both memory and SQLite
- Compresses payloads of 4 KB or more in SQLite (zstd, or zlib without zstd)
- Automatically commits periodically
- Enables fast subsequent access
get_telemetry_batch(year: int, gp: str, session: str, driver_laps: list[tuple[str, int]]) -> dict[tuple[str, int], Any]
Get multiple laps of telemetry in a single operation.
Parameters:
year(int): Season yeargp(str): Grand Prix name, URL-encoded as insession.gpsession(str): Session name, URL-encoded as insession.sessiondriver_laps(list[tuple[str, int]]):(driver_code, lap_number)pairs; one batch can mix drivers
- Dict mapping
(driver_code, lap_number)to telemetry data (only includes cached laps)
- Single SQLite query for all laps (much faster than individual queries)
- Optimized for bulk operations
- Returns only cached laps (missing laps are omitted)
has_session_data(year: int, gp: str, session: str) -> bool
Check if cache contains data for a specific session.
Parameters:
year(int): Season yeargp(str): Grand Prix name, URL-encoded as insession.gp(the plain name also works)session(str): Session name, URL-encoded as insession.session(the plain name also works)
Trueif session data exists in cache,Falseotherwise
- Returns
Truewhen any cached JSON file or telemetry entry belongs to the session - Fast operation (a memory scan, then at most two SQLite queries)
- Returns
Truewhen the probe itself fails, so that loads still try the cache - Useful for pre-flight checks before loading
clear() -> None
Remove all entries from both memory and SQLite cache.
Behavior:
- Clears in-memory LRU cache
- Deletes all rows from SQLite tables
- Does not vacuum the database: the file keeps its size until SQLite reuses the free pages
- Thread-safe operation
- Free disk space
- Force re-fetch of all data
- Clear corrupted cache entries
- Reset after configuration changes
close() -> None
Close the SQLite database connection and flush pending writes.
Behavior:
- Commits any pending transactions
- Closes database connection
- Releases file locks
tif1registers anatexithook that closes the global cache, so an explicit call is not necessary
Closing the global cache also releases it: the next
get_cache() call opens a new, working instance, and later loads use it. The closed object itself stays closed: its reads miss and its writes are dropped. Call get_cache() again instead of keeping the old reference.Cache Size Management
The cache grows over time as more sessions are loaded. Here is how to manage cache size: Check Cache Size:tif1. Then delete cache.sqlite, cache.sqlite-wal and cache.sqlite-shm in the cache directory. A process that has the database open continues to write to the deleted file.
Async Cache Operations
The cache also supports async operations for use in async applications:Cache Best Practices
- Let the cache warm up: First load is slow, subsequent loads are fast
- Do not clear cache unnecessarily: The cache improves performance on repeated loads
- Monitor cache size: Set up periodic cleanup if disk space is limited
- Use batch operations:
get_telemetry_batch()is much faster than individual calls - Do not keep a closed cache object:
tif1closes the cache at exit. Afterclose(), callget_cache()again - Check what is cached:
has_session_data()tells whether any file of a session is in the cache - Use SSD for cache: Faster than HDD for SQLite operations
- Consider cache location: Put cache on fast local storage, not network drives
Data Utilities
The data utilities provide convenient functions for converting between different time formats and safely navigating nested data structures. These are particularly useful when working with lap times, timestamps, and JSON responses.to_timedelta
Convert various time formats to pandas Timedelta objects.
Timedelta objects. The converted values support consistent time arithmetic and comparison.
Parameters:
x(str | float | int | timedelta): Time value to convert
Behavior:
- Automatically adds leading
"00:"to"M:SS.mmm"format - Handles negative numbers (for example,
-1.234for a delta). A negative string needs the full"-00:00:01.234"form - Keeps nanosecond precision
- Returns pandas
Timedeltafor consistent arithmetic
to_datetime
Convert string timestamps to pandas Timestamp objects.
to_datetime for consistent datetime parsing across the library.
Parameters:
x(str): Datetime string to convert
- ISO 8601:
"2021-08-29T14:00:00" - Standard:
"2021-08-29 14:00:00" - Date only:
"2021-08-29" - With timezone:
"2021-08-29T14:00:00+02:00"
recursive_dict_get
Safely navigate nested dictionaries without raising KeyError.
d(dict): Dictionary to navigate*keys(str): Keys to traverse in orderdefault_none(bool, optional): IfTrue, returnNonefor missing keys; ifFalse, return{}. Defaults toFalse.
- The value at the nested key path, or default value if any key is missing
delta_time
Calculate delta time between two laps (FastF1 compatibility function).
delta_time, without the distance scaling of recent FastF1 versions. It interpolates the comparison lap’s time onto the reference lap’s distance axis.
Parameters:
reference_lap(Lap): Reference lap (for example, fromlaps.pick_fastest()orDriver.get_lap()), or a telemetry frame withTimeandDistancecompare_lap(Lap): Lap to compare against the reference (same kinds asreference_lap)
- Tuple of
(delta_series, ref_telemetry, comp_telemetry)delta_series: Float seconds at each reference telemetry point. A positive value means that the comparison lap is slower at that point. The series is empty when either lap has fewer than two samplesref_telemetry: Reference lap telemetry (pandas, also on the polars backend)comp_telemetry: Comparison lap telemetry
Reliability & Networking
Thetif1 library implements reliability patterns to handle network failures, CDN outages, and transient errors gracefully. The circuit breaker pattern prevents cascading failures, while the CDN manager provides automatic fallback to alternative data sources.
Circuit Breaker Pattern
The circuit breaker is a critical reliability component. It stops the application from repeatedly trying operations that are likely to fail. It acts like an electrical circuit breaker, “opening” after too many failures to prevent cascading issues. Circuit Breaker States:- Closed (Normal): All requests are allowed through
- Open (Failing): Requests are blocked immediately without attempting
- Half-Open (Testing): Requests are allowed again. A success closes the breaker and a failure opens it again
get_circuit_breaker
Returns the global circuit breaker instance used for CDN payload requests.
CircuitBreaker: Global circuit breaker instance
- Fully thread-safe with atomic state transitions
- Uses reentrant locks for nested calls
- Safe for concurrent use across multiple threads
CircuitBreaker Properties
state: str
Current circuit breaker state (read-only).
Possible Values:
"closed": Normal operation, all requests allowed"open": Too many failures, requests blocked"half-open": Testing recovery, limited requests allowed
failures: int
Current failure count.
Example:
threshold: int
Number of failures before circuit opens.
Default: 5 (configurable via circuit_breaker_threshold config)
timeout: int
Seconds to wait before transitioning from open to half-open.
Default: 60 (configurable via circuit_breaker_timeout config)
last_failure_time: datetime | None
Timestamp of the most recent failure.
Example:
CircuitBreaker Methods
record_success() -> None
Record a successful operation (resets failure count).
Behavior:
- Resets failure counter to 0, unless the circuit is open
- Transitions from half-open to closed
- Thread-safe atomic operation
fetch_data() stands for the application’s own network call):
record_failure() -> None
Record a failed operation (increments failure count).
Behavior:
- Atomically increments failure counter
- Updates last failure timestamp
- Opens circuit if threshold reached
- Thread-safe operation
check_and_update_state() -> tuple[bool, str]
Check if request should proceed and update state if needed.
Returns:
- Tuple of
(should_proceed, current_state)
reset_circuit_breaker
Reset the circuit breaker to initial closed state.
- Creates new circuit breaker instance
- Resets failure count to 0
- Sets state to closed
- Clears last failure time
- Reloads configuration from config
- After resolving network issues
- When switching networks
- After configuration changes
- For testing purposes
CDN Management
The CDN manager handles multiple CDN sources with automatic fallback, health tracking, and failure recovery.get_cdn_manager
Returns the CDN manager responsible for CDN fallback and source health.
CDNManager: Global CDN manager instance
CDNManager Methods
get_sources() -> list[CDNSource]
Get list of enabled CDN sources sorted by priority.
Returns:
- List of enabled CDN sources that have not exceeded failure threshold
mark_failure(source_name: str) -> None
Mark a CDN source as failed (increments failure count).
Parameters:
source_name(str): Name of the CDN source
- Increments failure counter for the source
- Disables source if it reaches max failures (3)
- Logs warning when source is disabled
request_from_cdn() stands for the application’s own request):
mark_success(source_name: str) -> None
Mark a CDN source as successful (resets failure count).
Parameters:
source_name(str): Name of the CDN source
- Resets failure counter to 0
- Re-enables source if it was disabled
reset() -> None
Reset all CDN failure counts.
Behavior:
- Resets all failure counters to 0
- Re-enables all CDN sources
- Useful after network issues are resolved
Reliability Examples
Handle Network Failures Gracefully
Monitor Circuit Breaker Health
CDN Fallback Testing
Production Error Handling
Reliability Best Practices
- Monitor circuit breaker state: Check state before critical operations
- Respect circuit breaker: Do not bypass it when open
- Reset after network changes: Call
reset_circuit_breaker()when switching networks - Configure thresholds appropriately: Lower for fail-fast, higher for resilience
- Use multiple CDNs: Configure fallback CDNs for production
- Log failures: Track patterns to identify systemic issues
- Implement exponential backoff: Do not retry immediately after failures
- Handle DataNotFoundError separately: Do not retry when data genuinely does not exist
Memory Management
Thetif1 library uses in-memory LRU (Least Recently Used) caches to speed up repeated access to lap DataFrames. While this improves performance, it can consume significant memory when working with many sessions. These utilities help manage memory usage.
clear_lap_cache
Clear the in-memory LRU cache for lap DataFrames.
- Clears pandas lap cache
- Clears polars lap cache
- Does NOT affect SQLite cache (persistent storage)
- Frees the cached lap DataFrames that no
Sessionobject still holds - Thread-safe operation
- After processing large sessions
- When switching between different analyses
- To free memory in long-running applications
- Before loading a different season’s data
psutil: pip install psutil):
- Clear cache between unrelated analyses
- Clear cache in batch processing loops
- Do not clear too frequently (defeats caching purpose)
- Monitor memory usage in long-running applications
- Consider clearing cache before loading different seasons
Complete Examples
This section provides examples demonstrating how to combine multiple utilities for real-world use cases.Example 1: Configure and Save Settings
Set up the configuration for the specific use case and persist it.Example 2: Debug Session Loading
Enable detailed logging to diagnose data loading issues.Example 3: Cache Management Workflow
Complete cache inspection and management.Example 4: Handle Network Issues
Reliable error handling with circuit breaker awareness.Example 5: Custom Configuration File
Create project-specific configuration.Example 6: Performance Optimization
Optimize configuration for maximum performance.Example 7: Memory-Efficient Batch Processing
Process multiple sessions while managing memory usage.Best Practices
Guidelines for usingtif1 utilities effectively in different scenarios.
Development Best Practices
-
Enable Debug Logging During Development
- Helps understand data flow and identify issues
- Shows cache hits/misses for optimization
- Reveals network request patterns
- Displays parsing and validation steps
-
Use Pandas for Easier Debugging
- More familiar API for most developers
Lapsmethods such aspick_driver()andpick_fastest()are available- Easier to inspect intermediate results
-
Keep Validation Enabled
- Catches data quality issues early
- Provides clear error messages
- Helps identify API changes
- The validation overhead is acceptable in development
-
Use Shorter Timeouts
- Fail fast during development
- Do not wait for slow connections
- Iterate quickly on code changes
-
Clear Cache When Testing
- Ensures that tests run with fresh data
- Forces new downloads from the CDN
- Catches cache-related bugs
Production Best Practices
-
Use Polars When the Analysis Code Uses Polars
- Laps and telemetry arrive as polars frames
- No conversion step in polars pipelines
- The
Lapshelper methods (pick_*) need the pandas backend
-
Keep Validation Off in Production
- Skips the validation work on each payload
- Reduces CPU usage
- Only disable if data quality is trusted
- Keep enabled if data source is unreliable
-
Increase Timeout for Reliability
- Handle slow networks gracefully
- Accommodate mobile/satellite connections
- Reduce failure rate
- Balance with user experience
-
Configure Circuit Breaker Appropriately
- Higher threshold for production resilience
- Longer timeout for recovery
- Prevents cascading failures
- Monitor circuit breaker state
-
Use Multiple CDNs
- Automatic fallback on CDN failure
- Improved reliability
- Geographic redundancy
- Load distribution
-
Set Up Proper Logging
- Monitor operational health
- Track performance metrics
- Debug production issues
- Audit data access
Performance Optimization
-
Keep Ultra Cold Start On
- Faster first load of an uncached session
- Skips the cache read and validation on that first load
- Still writes the fetched payloads to the cache
- A cached session loads from the cache as usual
-
Clear Lap Cache Periodically
- Prevents memory bloat
- Important for long-running applications
- Clear between unrelated analyses
- Do not clear too frequently
-
Use Batch Telemetry Operations
- Single database query vs multiple
- Much faster for multiple laps
- Reduces I/O overhead
-
Monitor Cache Size
- Prevent disk space issues
- Set up monitoring alerts
- Implement automatic cleanup
- Archive old seasons
-
Use SSD for Cache
- Faster than HDD
- Reduces SQLite query time
- Better for concurrent access
- Avoid network drives
Error Handling
-
Distinguish Between Error Types
- Do not retry
ValueError(unknown names) or DataNotFoundError - Retry NetworkError with backoff
- Handle each error type appropriately
- Do not retry
-
Monitor Circuit Breaker State
- Track circuit breaker state
- Alert on open state
- Monitor failure patterns
- Investigate root causes
-
Implement Exponential Backoff
- Do not retry immediately
- Give service time to recover
- Reduce load on failing service
- Increase wait time exponentially
-
Log Errors with Context
- Include relevant context
- Track failure patterns
- Enable debugging
- Support monitoring
Memory Management
-
Clear Cache in Batch Processing
- Prevent memory bloat
- Clear every N iterations
- Balance performance vs memory
- Monitor memory usage
-
Monitor Memory Usage
- Set memory thresholds
- Automatic cleanup
- Prevent OOM errors
- Log memory metrics
Configuration Management
-
Use Environment Variables for Deployment
- Easy to configure across environments
- No code changes needed
- Supports containerization
- Environment-specific settings
-
Save Configuration After Tuning
- Persist optimized settings
- Share configuration across team
- Version control config files
- Document configuration choices
-
Use Project-Specific Configuration
- Isolate project caches
- Different settings per project
- Easier to manage
- Portable configuration
Testing
-
Clear Cache Before Tests
- Ensures test isolation
- Prevents cache pollution
- Reproducible tests
- Catches cache bugs
-
Block the Network in Unit Tests
- Faster tests
- No network dependency
- Predictable results
- Test error handling
-
Test Circuit Breaker Behavior
- Verify reliability patterns
- Test failure scenarios
- Ensure proper recovery
- Validate configuration
Monitoring and Observability
-
Track Performance Metrics
- Monitor load times
- Track cache hit rates
- Identify performance regressions
- Optimize based on data
-
Set Up Health Checks
- Regular health checks
- Alert on issues
- Track system state
- Support debugging
-
Implement Structured Logging
- Machine-readable logs
- Easy to parse and analyze
- Better for log aggregation
- Supports monitoring tools
Summary
Follow these best practices to:- Build reliable applications with proper error handling
- Optimize performance for the specific use case
- Manage resources efficiently
- Debug issues quickly
- Deploy confidently to production
- Monitor system health effectively