Overview
Thehttp_session module provides HTTP session management for tif1. It implements a thread-safe singleton pattern with connection pooling, a configurable DNS resolver list, and request counters. This infrastructure supports high-throughput parallel data fetching with low network overhead.
Why HTTP Session Management Matters
Formula 1 session data is spread across many JSON files on CDN servers. A session load requests one file per payload:- A few requests for the session tables (drivers, laps, weather, race control messages)
- One request per driver lap for telemetry, so a full race with telemetry can need more than 1000 requests
- DNS resolution of the CDN host name
- TCP handshake: one round trip
- TLS handshake: one or two round trips
Key Features
- Thread-safe singleton pattern: Single shared session across all threads and requests
- Aggressive connection pooling: Dynamically sized pools based on concurrency settings
- HTTP/2 multiplexing: Multiple requests over single connections (on by default,
http_multiplexed) - HTTP/3 support: QUIC, off by default (
http_disable_http3) - Resolver list: Configurable DNS resolvers, including DNS-over-HTTPS (DoH) entries (see DoH support)
- Request counters: Counts of the HTTP requests that tif1 sends through the shared session
- Keep-alive headers: Configurable
Keep-Alivetimeout and request limit values - Automatic resource cleanup: Graceful shutdown on process exit
- No environment settings:
trust_envis off, so the session does not read proxy variables (such asHTTPS_PROXY) or.netrcfiles
Core API
get_session
_session_lock) to ensure thread-safe initialization. Once the session is created, subsequent calls return the cached instance without acquiring the lock. This keeps the common case free of lock contention.
Session Lifecycle:
- First call creates the session with optimized settings
- Subsequent calls return the cached instance
- Session persists until process exit or explicit
close_session()call - Automatic cleanup registered via
atexithandler
niquests.Session: Shared session instance configured with:- Dynamic connection pooling based on concurrency settings
- The first resolver from
http_resolversthat builds a session - HTTP/2 multiplexing (if enabled)
- HTTP/3 only when
http_disable_http3isFalse(the default isTrue) - Keep-alive headers with configurable timeouts
- Disabled
trust_envfor predictable behavior
- HTTPAdapter mounted on
https://with custom pool settings - Connection pooling:
pool_connectionsandpool_maxsizedynamically calculated - Keep-alive headers:
Connection: keep-alivewith timeout and max request limits - Resolver list: Tries each entry of
http_resolversin order. It moves to the next entry only when session creation raises an error - Multiplexing: HTTP/2 multiplexing enabled by default
- First call: Builds the session and mounts its adapter. DNS lookups and connections happen at the first request, not here
- Subsequent calls: Return the cached instance without a lock
Configuration changes only take effect when tif1 builds a session. To apply new settings, call
tif1.async_fetch.close_session() and then close_session() from this module. The first call drops the reference that the async fetch layer keeps to the old session. The next get_session() call builds a new session.close_session
Connection Statistics
get_connection_stats
- Dictionary with connection metrics:
total_requests: Number of payload requests that tif1 sent through the shared sessionconnections_reused: Number of those requests that used an open pooled connectionconnections_created: Number of shared sessions createdreuse_rate:connections_reusedas a percentage oftotal_requests(0-100)
tif1 counts a request as reused when niquests reports no connection or TLS handshake time for it. The first request on a new connection is not reused. Data served from the cache makes no request, so a session that loads from the cache leaves the request counters at 0. During
Session.load(), tif1 fetches the session tables through a separate short-lived niquests session, and these counters do not include those requests.reset_connection_stats
Configuration
The HTTP session is configured via the global config object. Key settings:
Example:
DNS-over-HTTPS (DoH) Support
Thehttp_resolvers list sets the DNS resolvers that tif1 passes to niquests. tif1 tries the entries in order only when it creates the shared session. It moves to the next entry only when session creation raises an error. A failed DNS lookup during a request does not switch the resolver.
Default resolver order:
- Standard DNS (
"standard") doh+cloudflare://doh+google://
The resolver strings use niquests’
<scheme>+<provider>:// form. The older spelling doh://cloudflare is still accepted in .tif1rc and is translated to doh+cloudflare://. tif1 passes the chosen resolver to the pooled HTTPAdapter it mounts on https://, so HTTPS requests use it.Connection Pooling
The HTTP session uses aggressive connection pooling for maximum performance:- Connection reuse: Keeps connections alive for multiple requests
- Pool sizing: Dynamically sized based on concurrency settings
- Keep-alive: Configurable
Keep-Aliveheader values - Thread-safe: Single shared session across all threads
- Eliminates TCP handshake overhead
- Reduces TLS negotiation time
- Improves throughput for parallel requests
- Lowers latency for sequential requests
Advanced Configuration
HTTP/2 Multiplexing
HTTP/2 multiplexing is enabled by default to send multiple requests over a single connection. Disable it when needed:HTTP/2 multiplexing is enabled by default. Disable it only when a specific CDN configuration causes issues.
Custom pool sizing
Pool sizes are calculated from the concurrency settings. Override them only for specific requirements:The library automatically sizes connection pools based on
max_workers, max_concurrent_requests, and telemetry_prefetch_max_concurrent_requests settings. Manual override is rarely needed.Keep-Alive Tuning
These values only set theKeep-Alive request header. The server decides how long it keeps a connection open.
Troubleshooting
Connection pool exhaustion
Increase the pool size when connection pool warnings appear:DNS Resolution Failures
tif1 does not switch to DoH when a DNS lookup fails during a request. To use DoH, put a DoH entry first inhttp_resolvers and rebuild the shared session:
Connection reuse issues
The counters show how many payload requests went to the network. They also show how many of them used an open connection (see Connection Statistics):Best Practices
- Do not create multiple sessions: Use the shared session for all requests
- Monitor connection stats: Track how many requests reach the network
- Tune pool size: Match the pool size to the concurrency needs
- Use DoH when system DNS is unreliable: Put a DoH entry first in
http_resolvers - Keep HTTP/2 multiplexing on: It is the default. Disable it only when a CDN causes errors
- Let the library manage cleanup: Session closes automatically on exit
Summary
The http_session module provides:- Shared HTTP session with connection pooling
- A configurable resolver list
- Request counters
- Configurable pool sizing and keep-alive
- Thread-safe singleton pattern
- Automatic resource cleanup