Overview
The HTTP and networking layer is the foundation oftif1’s data fetching infrastructure. It is engineered for performance, reliability, and resilience. It loads Formula 1 telemetry and session data from remote CDN sources. Cold starts can require fetching 100+ telemetry files in parallel.
The networking layer is optimized for the patterns of F1 data access. These patterns include high-concurrency parallel fetching, connection reuse, retry strategies, and graceful degradation under failure conditions. A full session (20 drivers × 50+ laps) yields 1000+ telemetry files.
Architecture Philosophy
The networking layer is built on four core principles that guide every design decision:-
Performance First: Every design decision prioritizes speed and throughput. Connection pooling removes the TCP and TLS handshakes from reused connections. HTTP/2 multiplexing allows multiple requests over a single connection. JSON parsing uses
orjson, which is faster than the standard library parser. - Resilience by Design: Network failures are expected, not exceptional. The circuit breaker pattern prevents cascading failures by blocking requests during sustained outages. Exponential backoff with jitter reduces load on failing services while allowing recovery. Multi-CDN fallback ensures graceful degradation when primary sources fail. The system can tolerate complete CDN failures. It recovers automatically when services come back online.
- Resource Efficiency: The thread pool is sized from the concurrency settings, open connections are reused, and the cache avoids repeat downloads. The connection pool (at least 256 connections) is larger than the default concurrency caps.
- Observability: Metrics for connection reuse, circuit breaker state, and CDN health support monitoring and debugging. Periodic logging of the connection counters needs no manual instrumentation.
Key Components
The networking stack provides a complete set of features:1. High-Performance HTTP Session Management
The session layer usesniquests (a modern fork of requests with HTTP/2 and HTTP/3 support) with aggressive connection pooling and reuse strategies:
- Dynamic Connection Pooling: Pool size automatically scales based on configured concurrency. The minimum is 256 connections, up to 512+ for burst handling
- Keep-Alive Headers: Every request sends
Keep-Alivevalues fromkeepalive_timeout(default: 120s) andkeepalive_max_requests(default: 10000) - HTTP/2 and HTTP/3 Support: HTTP/2 with multiplexing by default; HTTP/3 only when
http_disable_http3isFalse - DNS-over-HTTPS (DoH) Resolvers: The
http_resolverslist can name Cloudflare and Google DoH resolvers. tif1 moves to the next entry only when building the session with an entry fails - Thread-Safe Creation: Session creation and closing run under a lock
- Connection Reuse Metrics: Request counters with a reuse rate
2. Async Parallel Fetching
Built on Python’sasyncio with thread pool executors for maximum concurrency:
- Concurrent Request Handling: Fetch 20-128+ files simultaneously (configurable based on workload)
- Thread Pool Optimization: Shared executor with worker count matching concurrency requirements
- Process Pool JSON Parsing: Optional multi-process JSON parsing for non-telemetry payloads (off by default;
json_parse_workers) - Semaphore-Based Rate Limiting: Automatic concurrency control to prevent CDN throttling
- Graceful Error Handling: Failed requests return
Noneinstead of crashing, enabling partial data loading
3. Retry Logic
The retry mechanism uses exponential backoff, jitter, and multi-CDN fallback:- Configurable Retry Count: Default 3 tries per request, the first one included. Zero-retry mode (one try) gives the fastest cold starts
- Exponential Backoff: Delay increases exponentially (2^attempt seconds) with configurable factor
- Jitter Support: On the async path, random jitter of up to
retry_jitter_maxseconds prevents thundering herd problems (the default0.0adds no jitter). The sync retry path scales each delay by a random factor from 0.5 to 1.5 whenretry_jitteris on - Max Delay Cap: Configurable maximum delay (default: 60s) prevents excessive wait times
- CDN Fallback: Automatically tries all configured CDN sources before failing
- Smart Error Classification: Distinguishes between retryable errors (5xx, timeouts) and fatal errors (404, validation failures)
- Pool Exhaustion Handling: Automatic backoff when connection pool is exhausted (0.01-0.5s with jitter by default)
4. Circuit Breaker Pattern
Prevents cascading failures and protects against sustained network issues:- Failure Threshold: Opens after 5 consecutive failures (configurable)
- Timeout Period: Blocks requests for 60 seconds when open (configurable)
- Half-Open State: After the timeout, requests go through again; a success closes the breaker and a failure opens it again
- Thread-Safe State Transitions: Atomic state changes using reentrant locks
- Monotonic Time Tracking: Uses
time.monotonic()for accurate timeout tracking - Protected Paths: Async fetches and the sync retry path check the breaker. The sync zero-retry path and
PayloadLoader.get_url()do not
5. Multi-CDN Fallback System
Reliable CDN management with automatic failover, health tracking, and optional minification:- Multiple CDN Support: Configure multiple CDN sources with priority ordering via
cdnsconfig array - Automatic Failover: Switches to backup CDNs on failure without user intervention
- Health Tracking: Tracks failure counts per CDN, automatically disables after 3 consecutive failures
- jsDelivr Primary: jsDelivr is the primary CDN. Hugging Face buckets are the fallback, and StaticDelivr is the last-resort backup
- URL Formatting: Automatic URL construction for TracingInsights data repositories (StaticDelivr:
{cdn_base}/{year}/main/{gp}/{session}/{path}, jsDelivr:{cdn_base}/{year}@main/{gp}/{session}/{path}, Hugging Face:{cdn_base}/{year}/resolve/{gp}/{session}/{path}) - Dynamic Enable/Disable: CDNs can be dynamically disabled based on error patterns (for example, HTTP/2 multiplexing incompatibility after redirects)
- Minification for custom mirrors:
cdn_use_minificationrequestsfile.min.jsonfrom custom mirrors that serve minified copies. The default CDNs ignore it, because none of them serves.min.jsonfiles - Failure Recovery: A successful request resets the failure count of that CDN. A CDN with 3 failures is no longer tried, so only
reset()orreload()brings it back - Priority Ordering: CDNs are tried in priority order (lower priority number = higher precedence)
cdn_use_minification only affects custom mirrors listed in cdns. For such a mirror, JSON files are requested in minified form:
- Original:
https://mirror.example.com/tif1/2024@main/Bahrain%20Grand%20Prix/Race/VER/laptimes.json - Minified:
https://mirror.example.com/tif1/2024@main/Bahrain%20Grand%20Prix/Race/VER/laptimes.min.json
.min.json request answers 404 on all three, so tif1 never sends one there. The option is off by default.
CDN Health Management:
The CDN manager tracks failures per source and automatically disables CDNs that fail repeatedly:
raw.githubusercontent.com- GitHub’s raw content CDN has aggressive rate limiting (60 requests/hour for unauthenticated users). The one request tif1 makes there on purpose is the live season’sschedule_{year}.jsonintif1.events, outside the CDN chain and cached forschedule_ttl_hours
6. Rate Limiting and Concurrency Control
Prevents CDN throttling and manages resource consumption:- Configurable Concurrency: Default 22 concurrent requests, and 32 for telemetry prefetch (configurable to 128+)
- Semaphore-Based Control: Async semaphores limit concurrent operations
- Per-Operation Limits: Different limits for different workloads (lap data vs telemetry)
- Backoff on Exhaustion: Automatic short backoff when the connection pool reports exhaustion
7. Performance Monitoring
Real-time metrics for optimization and debugging:- Connection Reuse Tracking: Monitor connection pool efficiency (reuse rate, total requests, pools created)
- Periodic Logging: Automatic statistics logging at configurable intervals (default: 60s)
- Circuit Breaker State: Inspect current state (closed, open, half-open) and failure counts
- CDN Health Metrics: Track per-CDN failure counts and enabled status
Use Cases and Optimizations
The networking layer is specifically optimized for F1 data loading scenarios:Cold Start Optimization (100+ Telemetry Files)
When loading a session for the first time,tif1 may need to fetch 100+ telemetry files in parallel:
- High Concurrency: 32 concurrent requests during telemetry prefetch by default (configurable to 128+)
- Connection Reuse: Persistent connections skip the TCP and TLS handshakes
- HTTP/2 Multiplexing: Multiple requests over single connection reduce latency
- Zero-Retry Mode: Tries each CDN once without delays.
ultra_cold_skip_retries(default:True) turns it on for the cold-start path - Process Pool Parsing: Optional multi-process JSON parsing for non-telemetry payloads. Telemetry always parses in threads because of IPC overhead
Large JSON Payloads (Lap Data, Telemetry Arrays)
Lap and telemetry data can contain thousands of data points:- Fast JSON Parsing: Uses
orjsonon the raw response bytes - Raw Cache Writes: An unchanged payload is stored as the original bytes, without a new serialization
- Validation Pipeline: Optional payload validation (disabled by default for performance)
- Cache Integration: Parsed data automatically cached to avoid re-fetching
Network Failure Scenarios
Graceful degradation when CDN sources fail or network is unstable:- Multi-CDN Fallback: Automatically tries backup CDNs (jsDelivr → Hugging Face buckets → StaticDelivr)
- Circuit Breaker Protection: Prevents cascading failures by blocking requests during sustained outages
- Exponential Backoff: Reduces load on failing services while allowing recovery
- Partial Data Loading: Failed requests return
None, allowing successful data to be used
Resource-Constrained Environments
Optimizations for limited bandwidth or CPU:- Configurable Concurrency: Reduce concurrent requests (5-10) for slow connections
- Connection Pool Sizing: Adjust pool size based on available resources
- Timeout Configuration: Increase timeouts (60s+) for slow networks
- Minification for custom mirrors:
cdn_use_minificationapplies only to mirrors that serve.min.jsoncopies
Thread Safety and Resource Management
All networking components are designed for safe concurrent use:- Thread-Safe Sessions: HTTP session creation and closing run under a lock
- Reentrant Locks: Circuit breaker uses reentrant locks for nested call support
- Atomic State Transitions: Circuit breaker state changes run under its lock. CDN failure counters use no lock (see the CDN API)
- Automatic Cleanup: Resources automatically released via
atexithandlers - Graceful Shutdown: Executors and sessions properly closed on program exit
Configuration
The networking layer is configurable via environment variables or.tif1rc file. All networking-related settings can be tuned for each specific use case, from ultra-fast cold starts to resource-constrained environments.
Configuration File Location:
The .tif1rc file is a JSON configuration file that can be placed in:
- Path specified by
TIF1_CONFIG_FILEenvironment variable (highest priority) - Current working directory (if
TIF1_TRUST_CWD_CONFIG=true) - Home directory (
~/.tif1rc) (default location)
timeout(int, default: 30): Request timeout in seconds. Increase for slow connections or large files.max_retries(int, default: 3): Total tries for each request, including the first. Each try walks every CDN. Set to 1 (or 0) for zero-retry mode (one try).max_workers(int, default: 20): Thread pool size for async operations. The executor is sized as the max ofmax_workersand the concurrency caps (max_concurrent_requests, telemetry prefetch).
retry_backoff_factor(float, default: 2.0): Exponential backoff multiplier. Delay = factor^attempt seconds.retry_jitter(bool, default: true): Add random jitter to backoff delays to prevent thundering herd.retry_jitter_max(float, default: 0.0): Maximum jitter in seconds on the async path (random value between 0 and this). The default adds no jitter there.max_retry_delay(float, default: 60.0): Maximum backoff delay in seconds (caps exponential growth).
circuit_breaker_threshold(int, default: 5): Number of consecutive failures before opening circuit.circuit_breaker_timeout(int, default: 60): Seconds to wait before testing recovery (half-open state).
pool_connections(int, default: not set): Number of connection pools to maintain. If not set, tif1 uses the largest concurrency setting, with a minimum of 256.pool_maxsize(int, default: not set): Maximum connections per pool. If not set, tif1 uses 4xpool_connections(1024 with the defaults). The minimum is 512.keepalive_timeout(int, default: 120): Timeout in seconds sent in theKeep-Aliverequest header.keepalive_max_requests(int, default: 10000): Request limit sent in theKeep-Aliverequest header.
max_concurrent_requests(int, default: 22): Maximum concurrent HTTP requests for general operations.telemetry_prefetch_max_concurrent_requests(int, default: 32): Maximum concurrent requests for telemetry prefetch.
http_multiplexed(bool, default: true): Enable HTTP/2 multiplexing for reduced latency.http_disable_http3(bool, default: true): HTTP/2 over TCP is the default. Set it tofalseto negotiate HTTP/3.http_resolvers(array, default: [“standard”, “doh+cloudflare://”, “doh+google://”]): Resolvers to try in order when tif1 builds the session.
cdns(array, default: jsDelivr, Hugging Face buckets, StaticDelivr): CDN sources in priority order.cdn_use_minification(bool, default: false): Request.min.jsonfiles from custom mirrors. The default CDNs ignore it.
pool_exhaustion_backoff_base(float, default: 0.01): Base backoff delay in seconds when pool is exhausted.pool_exhaustion_backoff_max(float, default: 0.5): Maximum backoff delay in seconds for pool exhaustion.pool_exhaustion_backoff_jitter(float, default: 0.01): Maximum random jitter in seconds for pool exhaustion backoff.
json_parse_workers(int, default: 0): Number of worker processes for JSON parsing. 0 = disabled (recommended for telemetry).connection_stats_log_interval(float, default: 60.0): Seconds between automatic connection stats logging.
TIF1_ prefix:
HTTP Session Management
The HTTP session layer provides a sharedniquests.Session instance with optimized configuration for F1 data fetching. The session is lazily initialized on first use and automatically cleaned up on program exit.
get_session
Get or create the global HTTP session instance with connection pooling and a configurable resolver list.
- Dynamic pool sizing based on configured concurrency levels
- Minimum 256 connections for optimal performance
- Maximum pool size of 512+ for burst handling
- Non-blocking pool behavior (pool_block=False) to prevent deadlocks
Connection: keep-aliveandKeep-Alive: timeout=120, max=10000on every request- Configurable via
keepalive_timeoutandkeepalive_max_requests - The server decides how long it keeps a connection open
- Default list: system DNS (
standard), then Cloudflare DoH (doh+cloudflare://), then Google DoH (doh+google://) - Configurable via
http_resolversconfig option (array of resolver strings) - tif1 builds the session with the first entry. It moves to the next entry only when building the session with an entry raises an error
- A failed DNS lookup during a request does not switch the resolver. With the default list, tif1 uses system DNS
- tif1 hands the chosen resolver to the
HTTPAdapterthat it mounts onhttps://, so HTTPS requests use it
.tif1rc file:
"standard","default","none", or""- Use system DNS"doh+cloudflare://"- Cloudflare DNS-over-HTTPS (1.1.1.1)"doh+google://"- Google DNS-over-HTTPS (8.8.8.8)
NetworkError is raised. The older spelling doh://cloudflare is translated to doh+cloudflare://.
Protocol Support:
- HTTP/2 with multiplexing (configurable via
http_multiplexed, default:true)- Multiple requests over a single TCP connection
- Reduces latency by eliminating TCP handshake overhead
- Header compression reduces bandwidth usage
- Automatic protocol negotiation via ALPN
- HTTP/3 support (off by default; set
http_disable_http3tofalseto negotiate it)- QUIC-based protocol (UDP)
- Automatic fallback to HTTP/2 if HTTP/3 is unavailable
- Automatic protocol negotiation
- Client and server negotiate the best available protocol
- Transparent fallback from HTTP/3 → HTTP/2 → HTTP/1.1
- Fewer Connections: Many requests to a CDN share one TCP connection
- No Head-of-Line Blocking at the HTTP level: Slow responses do not block fast ones
- Reduced Latency: Fewer TCP and TLS handshakes
- Lower Resource Usage: Fewer file descriptors and socket buffers
"non-multiplexed response after a redirect" disables the CDN automatically. Multiplexing can also be disabled manually:
trust_env=Falseto skip environment lookups on every request. The session therefore ignores proxy variables (such asHTTPS_PROXY) and.netrcfilesConnection: keep-aliveandKeep-Aliveheaders on every request- Connection reuse tracking for monitoring
- Shared
niquests.Sessioninstance (lazily initialized)
NetworkError: If session creation fails with all configured DNS resolvers
- Uses double-checked locking pattern for safe concurrent initialization
- Multiple threads can safely call
get_session()simultaneously - tif1 sends requests on the shared session from its worker threads
The HTTP session is automatically managed. Direct interaction is rarely needed. The async fetch layer handles all session management internally.
close_session
Close the global HTTP session and release all connections.
atexit for cleanup on program exit, so manual calls are rarely necessary.
Behavior:
- Closes all connections in the connection pool
- Releases socket file descriptors
- Clears internal session state
- Thread-safe with lock protection
- Idempotent (safe to call multiple times)
- Testing scenarios that need a reset of session state
- Long-running applications that need to free resources
- Before forking processes (to avoid sharing file descriptors)
Session cleanup is automatically registered with
atexit, so manual cleanup is usually unnecessary. The session closes properly when the program exits.get_connection_stats
Get current connection pool statistics for monitoring, debugging, and optimization.
total_requests(int): Number of payload requests that tif1 sent through the shared session since process start (or the last reset)connections_reused(int): Number of those requests that used an open pooled connectionconnections_created(int): Number of shared sessions created (typically 1)reuse_rate(float): Percentage of requests that reused connections (0-100)
- 90-100%: Excellent - optimal connection reuse
- 70-90%: Good - most connections are reused
- 50-70%: Fair - some optimization possible
- <50%: Poor - check pool configuration or request patterns
tif1 counts a request as reused when niquests reports no connection or TLS handshake time for it (
response.conn_info). connections_created counts session creations, not TCP connections. Requests sent directly with get_session().get() are not counted.tif1 logs the statistics at INFO level when it counts a request and at least
connection_stats_log_interval seconds (default: 60) have passed since the last log. Use this function for on-demand monitoring or custom logging.reset_connection_stats
Reset connection pool statistics to zero (primarily for testing).
- Resets
total_requeststo 0 - Resets
connections_reusedto 0 - Resets
connections_createdto 0 - Resets last log time to 0
- Thread-safe with lock protection
Async Parallel Fetching
The async fetch module provides high-performance parallel data loading. It uses Python’sasyncio with thread pool executors and optional process pool JSON parsing. This layer is optimized for F1 data loading patterns. Cold starts may require fetching 100+ telemetry files at once.
Architecture
The async fetch system uses a multi-layered architecture:- Async Coordination Layer:
asynciomanages concurrent operations and task scheduling - Thread Pool Executor: Handles blocking I/O operations (HTTP requests, file I/O, JSON parsing)
- Process Pool Executor: Optional multi-process JSON parsing for large payloads (configurable via
json_parse_workers) - HTTP Session Layer: Shared
niquests.Sessionwith connection pooling and reuse - Cache Integration: Automatic cache reads/writes with memory and SQLite tiers
fetch_json_async
Asynchronously fetch and parse JSON data with caching, retry logic, CDN fallback, and validation.
tif1. Single session tables also load through the sync PayloadLoader path. It implements a multi-stage pipeline optimized for performance and reliability.
Pipeline Stages:
- Cache Lookup: Checks the in-memory cache first, then the SQLite cache. A recent “missing on every CDN” verdict raises
DataNotFoundErrorhere - Network Fetch: Tries primary CDN (jsDelivr), falls back to Hugging Face buckets, then StaticDelivr on failure
- JSON Parsing: Uses
orjsonfor fast parsing, optional process pool for non-telemetry payloads - Validation: Optional payload validation (non-strict mode allows partial data), off unless a
validate_*config flag is on - Cache Write: Persists the payload to the SQLite cache for fast subsequent access
max_retries is 1 (or 0), the function tries each CDN once without delays. This mode suits fast startup when occasional transient failures are acceptable.
Pool Exhaustion Handling: On pool exhaustion, tif1 waits a short backoff (0.01-0.5s with jitter by default). The fetch then moves on to the next CDN. This prevents cascading failures.
Parameters:
year(int): Season year (for example, 2024)gp(str): Grand Prix name (for example, “Bahrain Grand Prix”)session(str): Exact session name (for example, “Race”, “Qualifying”, “Practice 1”, “Sprint”)path(str): Path to JSON file relative to session directory- Driver list:
"drivers.json" - Lap data:
"session_laptimes.json"(all drivers),"VER/laptimes.json","HAM/laptimes.json", etc. - Telemetry:
"VER/1_tel.json"({driver}/{lap}_tel.jsonformat) - Weather:
"weather.json" - Race control:
"rcm.json"
- Driver list:
max_retries(int | None): Total tries for each request, including the first- Default: Uses
max_retriesconfig value (typically 3) - Set to 1 for zero-retry mode (ultra-fast cold start, tries each CDN once); 0 behaves the same here
- Set to 5+ for unreliable networks
- The wait between tries uses exponential backoff, plus optional jitter
- Default: Uses
timeout(int | None): Request timeout in seconds- Default: Uses
timeoutconfig value (typically 30) - Increase to 60+ for slow connections or large files
- Applies per-request, not total operation time
- Timeout triggers retry with next CDN
- Default: Uses
use_cache(bool): If True, read from cache before network fetch- Default: True
- Set to False to force fresh fetch (bypasses cache read entirely)
- Automatically False in CI mode (
ci_mode=True) - Cache lookup checks memory first, then SQLite
write_cache(bool): If True, persist successful network responses to cache- Default: True
- Set to False for ephemeral data (testing, one-time queries)
- Automatically False in CI mode (
ci_mode=True) - Writes to the SQLite cache (persistent across processes)
validate_payload(bool): If True, run payload validation before returning data- Default: True
- Validation runs only when the
validate_data,validate_lap_times, orvalidate_telemetryconfig flag for that payload isTrue; all three default toFalse - Validation is non-strict (allows missing optional fields)
- Validates race control, weather, lap times, and telemetry based on file path (
drivers.jsonis not validated) - Sanitizes telemetry payloads (removes validator-only defaults that break DataFrame construction)
- Parsed JSON data as dictionary (never None, raises on error)
-
Dictionary structure depends on file type:
drivers.json:{"drivers": [{"driver": "VER", "dn": "1", "fn": "Max", "ln": "Verstappen", "team": "Red Bull Racing", ...}, ...]}*/laptimes.json,session_laptimes.json:{"lap": [...], "time": [...], "s1": [...], "compound": [...], ...}*_tel.json:{"tel": {"time": [...], "speed": [...], "rpm": [...], "distance": [...], ...}}weather.json:{"wT": [...], "wAT": [...], "wTT": [...], ...}rcm.json:{"time": [...], "cat": [...], "msg": [...], "flag": [...], ...}
Sessionrenames them to the DataFrame columns (LapNumber,AirTemp, …).
NetworkError: If network request fails after all retries and CDN fallbacks- Includes URL and status code (if available) for debugging
- Raised when all CDNs fail or circuit breaker is open
- Check circuit breaker state and CDN health if this occurs frequently
DataNotFoundError: If data does not exist (every CDN refused it with a 4xx status other than 408 or 429, such as 404)- Includes year, event, and session for context
- Not retried (404 is a fatal error indicating missing data)
- Common for sessions that have not been run yet or invalid driver codes
- With
use_cache=True, tif1 remembers the verdict formissing_payloads_ttl_days(default: 7) and raises again without a network request
InvalidDataError: If JSON parsing or validation fails- Includes a reason for debugging
- Not retried (data corruption is a fatal error)
- May indicate CDN issues or incompatible data format
This function is the foundation of the async data loading in
tif1. The high-level Session API uses it for its async and parallel fetches. Direct calls are rarely needed, except when building custom data loading logic.fetch_multiple_async
Fetch multiple JSON files in parallel with optimized batch size, graceful error handling, and automatic rate limiting.
-
Small Batches (requests ≤ max_concurrent_requests):
- Uses
asyncio.gather()for maximum parallelism - All requests execute simultaneously without queuing
- Uses
-
Large Batches (requests > max_concurrent_requests):
- Uses semaphore-based rate limiting to prevent CDN throttling
- Limits concurrent requests (cache reads included), queues excess until slots become available
fetch_json_async which raises exceptions, this function implements graceful degradation for partial data loading:
- Failed requests return
Noneinstead of raising exceptions - Successful requests return parsed data dictionaries
DataNotFoundError(404) is silently converted toNone(common for missing telemetry files)- Other exceptions are logged with warning level and converted to
None - Allows partial data loading (some files succeed, others fail)
- Check result list for
Nonevalues to identify failed requests
requests(list[tuple[int, str, str, str]]): List of (year, gp, session, path) tuples- Each tuple represents one file to fetch
- Example:
[(2024, "Bahrain Grand Prix", "Race", "VER/laptimes.json"), (2024, "Bahrain Grand Prix", "Race", "HAM/laptimes.json")] - Can contain 100+ tuples for telemetry prefetch (all drivers × all laps)
- Order is preserved in results (result[i] corresponds to requests[i])
use_cache(bool): If True, read from cache before network fetch- Default: True
- Applies to all requests in the batch
write_cache(bool): If True, persist successful network responses to cache- Default: True
- Applies to all requests in the batch
- Failed requests are not cached (an all-CDN 4xx verdict is remembered, see
fetch_json_async)
validate_payload(bool): If True, run payload validation before returning data- Default: True
- Applies to all requests in the batch
- Validation runs only when the
validate_*config flag for that payload isTrue
max_retries(int | None): Total tries for each request, including the first- Default: Uses
max_retriesconfig value (typically 3) - Applies independently to each request
- Failed requests after all retries return
None
- Default: Uses
timeout(int | None): Request timeout in seconds- Default: Uses
timeoutconfig value (typically 30) - Applies per-request, not total batch time
- Increase for slow connections or large files
- Default: Uses
max_concurrent_requests(int | None): Maximum concurrent requests- Default: Uses
max_concurrent_requestsconfig value (typically 22) - Increase to 50-128 for fast connections and telemetry prefetch
- Decrease to 5-10 for slow connections or rate-limited CDNs
- Higher values = faster but more resource intensive
- Default: Uses
defer_telemetry_writes(bool): If True, collect telemetry payloads and write them to the cache in one serial flush after the batch- Default: False
- The cache writes then do not occupy concurrency slots
- List of fetched data dictionaries (same length as
requests) - Each element is either:
dict[str, Any]: Successfully fetched and parsed dataNone: Request failed (logged but not raised)
- Order matches input
requestslist (result[i] corresponds to requests[i]) - Filter out
Nonevalues to get only successful results
This function is used internally by
Session.load() for parallel driver lap data loading and telemetry prefetch. It is the key to tif1’s fast cold start performance.validate_payload: If True, run payload validation before returning datamax_retries: Total tries for each request, including the firsttimeout: Request timeout in secondsmax_concurrent_requests: Maximum concurrent requests (defaults to config value, typically 22)
- List of fetched data dictionaries (None for failed requests). Exceptions are logged and converted to None for graceful degradation. DataNotFoundError (404) is silently converted to None.
fetch_with_rate_limit
Execute async function with rate limiting using semaphore.
coro_func: Async function to execute*args: Positional arguments for coro_funcsemaphore: Optional semaphore for rate limiting. If None, creates one based on max_concurrent_requests config**kwargs: Keyword arguments for coro_func
- Result from coro_func execution
close_session
Close the shared HTTP session that the async fetch layer holds, and drop that reference. It does nothing when no async fetch has run yet; tif1.http_session.close_session() closes the shared session in that case.
cleanup_resources
Clean up all async resources including session and executor.
Rate Limiting
The async fetch module includes automatic rate limiting to prevent CDN throttling:- Maximum 22 concurrent requests by default (configurable via
max_concurrent_requestsconfig) - Configurable via
max_concurrent_requestsparameter infetch_multiple_async - Automatic short backoff on connection pool exhaustion (a 429 response gets no special handling)
- Semaphore-based concurrency control
Connection Pooling
The HTTP session uses aggressive connection pooling for maximum performance:- Dynamic pool sizing based on concurrency (at least 256 connections and a pool size of at least 512)
- Reuses TCP connections across requests
Keep-Aliverequest headers with configurable values (default: 120s, 10000 max requests)- Automatic connection cleanup
- Connection reuse metrics tracking
- No TCP and TLS handshakes on reused connections
- Lower CPU usage
- Reduced network overhead
Retry Logic
Payload requests include automatic retry with exponential backoff and CDN fallback:- Default: 3 tries, the first one included (configurable via
max_retriesconfig) - Backoff: 2^attempt seconds, plus optional jitter (configurable)
- Max delay: 60 seconds (configurable via
max_retry_delay) - Retries on: Connection errors, timeouts, 5xx errors, pool exhaustion
- No retry on: 404 (data not found), 4xx client errors
- CDN fallback: Tries all configured CDN sources before failing
- Zero-retry mode (
max_retries=1): Ultra-fast cold start optimization (tries all CDNs once, no delays)
- Base backoff: 0.01 seconds, doubled on each try (configurable via
pool_exhaustion_backoff_base) - Max backoff: 0.5 seconds (configurable via
pool_exhaustion_backoff_max) - Jitter: up to 0.01 seconds (configurable via
pool_exhaustion_backoff_jitter)
Circuit Breaker Integration
HTTP requests are protected by a circuit breaker to prevent cascading failures:- Opens after 5 consecutive failures (configurable via
circuit_breaker_threshold) - Blocks requests for 60 seconds when open (configurable via
circuit_breaker_timeout) - After the timeout, lets requests through again (half-open state)
- Closes on successful request
- Thread-safe with atomic state transitions
- Uses monotonic time for accurate timeout tracking
closed: Normal operation, requests allowedopen: Too many failures, requests blockedhalf-open: Testing recovery; requests are allowed, and the next result closes or reopens the breaker
Complete Examples
Parallel session loading
Custom timeout handling
Batch processing with rate limiting
Resource cleanup
Zero-retry mode for ultra-fast cold starts
Performance Tips
- Use async methods for multiple requests: They fetch files in parallel instead of one after another.
- Tune max_concurrent_requests to match the network: Higher values (50-128) for fast connections, lower (5-10) for slow.
-
Keep validation off in production: The
validate_*config flags default toFalse.validate_payload=Falseskips validation even when a flag is on.
- Use zero-retry mode for cold starts: Fastest possible startup, tries all CDNs once. A value of 0 also makes one try.
- Monitor connection pool stats: Track reuse rate for optimization.
- Monitor circuit breaker: Check state if experiencing network issues.
- Clean up resources: Automatic via
atexit, but can call manually if needed.
- Use process pool for JSON parsing: For large non-telemetry payloads, enable multi-process parsing. Put the script entry point under
if __name__ == "__main__":, because the worker processes can re-import the main module.
Advanced Topics
Connection Pool Optimization
The connection pool is central totif1’s networking performance. Tune it to match the workload. Correct tuning has a direct effect on performance.
How Connection Pooling Works:
For each HTTP request, the session checks if an existing connection to that host is available in the pool:
- Pool Hit: Reuse existing connection (no TCP handshake and no TLS negotiation)
- Pool Miss: Create new connection (full TCP + TLS overhead)
- Increased latency (requests queued waiting for connections)
- “Pool timeout” or “Max retries” errors
- Automatic backoff delays (0.01-0.5s with jitter by default)
- A
Connection pool exhaustion detectedwarning in the log
JSON Parsing Optimization
JSON parsing is a significant bottleneck for large payloads (lap data, telemetry arrays).tif1 uses several strategies to optimize parsing performance.
Default Strategy (orjson in thread pool):
By default, JSON parsing happens in the thread pool using orjson, which is faster than the standard library parser:
if __name__ == "__main__": (see the benchmark below):
- ✅ Large lap data files
- ✅ CPU-bound workloads with few large files
- ❌ Telemetry payloads: tif1 always parses them in threads, because IPC overhead dominates
- ❌ Resource-constrained environments
Zero-Retry Mode (Ultra-Fast Cold Start)
Zero-retry mode is an advanced optimization that trades reliability for speed. Instead of retrying failed requests with exponential backoff, it tries each CDN once and moves on. The time saved is the backoff delay of each retry, which only failed requests pay. Enabling Zero-Retry Mode:- ✅ Fast, reliable networks (fiber, datacenter)
- ✅ Development/testing environments
- ✅ When speed is critical and occasional failures are acceptable
- ❌ Slow or unreliable networks
- ❌ Production systems requiring high reliability
HTTP/2 Multiplexing Deep Dive
HTTP/2 multiplexing letstif1 send many requests over few connections. Learn how it works to gain the full benefit.
What is Multiplexing?
HTTP/2 allows multiple requests to share a single TCP connection:
- Reduced Latency: No new TCP and TLS handshakes for requests on an open connection
- Lower Resource Usage: Fewer file descriptors, socket buffers, and kernel resources
- Better Throughput: No head-of-line blocking (slow requests do not block fast ones)
- Header Compression: HPACK compression reduces bandwidth usage
Circuit Breaker Patterns
The circuit breaker prevents cascading failures by blocking requests during sustained outages. Learn its state machine to use it effectively. State Machine:tif1 creates the circuit breaker on import. A config.set on a circuit_breaker_* key has no effect until reset_circuit_breaker() builds a new breaker. Call get_circuit_breaker() again after a reset, because the old object is no longer in use.Performance Monitoring and Profiling
Monitoring of these metrics helps identify bottlenecks and optimize performance. Real-Time Performance Dashboard:Troubleshooting
This section provides solutions to common networking issues intif1.
Slow Requests
Symptoms:- Requests taking >5 seconds to complete
- Timeouts on fast connections
- Inconsistent performance
Connection Errors
Symptoms:NetworkErrorexceptions- “Connection refused” or “Connection reset” errors
- Intermittent failures
Rate Limiting
Symptoms:- 429 “Too Many Requests” errors
- Requests being throttled or blocked
- CDN returning rate limit errors
Memory Issues
Symptoms:- Out of memory errors
- Slow performance due to swapping
- High memory usage during telemetry prefetch
Pool Exhaustion
Symptoms:- “Pool timeout” or “Max retries” errors
- Automatic backoff delays (0.01-0.5s by default)
- A
Connection pool exhaustion detectedwarning in the log
DNS Resolution Failures
Symptoms:- “Name or service not known” errors
- DNS lookup timeouts
- Intermittent connection failures
HTTP/2 Multiplexing Issues
Symptoms:- “MultiplexingError: non-multiplexed response after a redirect” errors
- CDNs being automatically disabled
- Lower than expected connection reuse rate
Validation Errors
Symptoms:InvalidDataErrorexceptions- “Validation failed” errors
- Data corruption warnings