cdn module provides multi-source Content Delivery Network (CDN) management. It includes automatic fallback, health tracking, circuit breaker patterns, and source selection. Data fetching continues when individual CDN sources fail, experience outages, or degrade in performance.
Overview
The CDN system is a core component of tif1’s data fetching infrastructure. It is designed for availability, reliability, and performance when it retrieves Formula 1 data from remote sources. The system implements circuit breaker logic, automatic failover, health monitoring, and priority-based routing. These patterns keep the application operational under adverse network conditions.Why CDN Management Matters
Formula 1 data fetching presents unique challenges:- Data availability: Historical and live session data must be reliably accessible across multiple seasons (2018-2026+)
- Geographic distribution: Users worldwide need fast access regardless of location
- Network resilience: CDN providers can experience outages, rate limiting, or degraded performance
- Bandwidth optimization: Session data can be large (telemetry, lap times, weather data)
- Cost efficiency: Proper CDN selection and fallback minimize redundant requests and bandwidth waste
Core Capabilities
1. Multi-Source Fallback with Automatic Failover
The CDN manager maintains a prioritized list of CDN sources and automatically tries alternative sources when the primary source fails. This ensures high availability even when individual CDN providers experience issues. How it works:- Sources are tried in priority order (lowest priority number first)
- If a source fails, the next available source is tried immediately
- Successful requests reset the failure counter for that source
- Failed sources are temporarily disabled after reaching the failure threshold
- A 404 or 403 is a per-file refusal, not a failure. The next source is tried (raced, on the async path) and the counter is untouched. jsDelivr returns 403 for uncached files of a repository over its 50 MB package limit. Counting those would disable the fastest mirror for the rest of the process
try_sources()reads the status from theresponseattribute of the exception, as niquestsHTTPErrorcarries it. For aNetworkErrorwithout aresponse, such as the one the syncPayloadLoadertransport raises, it reads thestatus_codecontext key. A 403 is a refusal on both paths
- Zero-downtime failover: Automatic switching to backup sources without user intervention
- Counter reset on success: A successful request sets the failure count of that source back to 0
- Configurable priorities: Define preferred CDN providers based on performance, cost, or geographic location
2. Health Tracking and Circuit Breaker Pattern
Each CDN source is continuously monitored for reliability. The circuit breaker pattern prevents wasting time on consistently failing sources by temporarily disabling them after repeated failures. Circuit breaker states:- Closed (healthy): 0-2 failures—source is available and used normally
- Open (disabled): 3+ failures—source is excluded from
get_sources()until reset - Recovery: There is no automatic half-open state.
try_sources()never tries an open source, so it stays open until a call toreset(),reload(), ormark_success(name)
- Default: 3 consecutive failures before disabling. Transport errors, timeouts, and 5xx responses count. A 404 or a 403 never counts
- Configurable via
_max_failuresattribute - Failures are tracked per-source in
_failure_countsdictionary
- Reduced latency: Skip known-bad sources instead of waiting for timeouts
- Prevent cascading failures: Isolate failing sources to protect overall system health
- Manual recovery: Call
reset()after a network outage to re-enable every source
3. Priority-Based Routing
Sources are assigned priority levels (integer values, lower = higher priority) that determine the order in which they are tried. Priority levels support these strategies:- Optimize for performance: Set fastest CDN as priority 1
- Optimize for cost: Set free/unlimited CDN as priority 1, paid CDN as backup
- Optimize for geography: Set regional CDN as priority 1 for users in that region
- Implement tiered fallback: Primary (priority 1) → Regional backup (priority 2) → Global backup (priority 3)
- Sources are sorted by priority (ascending order: 1, 2, 3, …)
- Sources with the same priority maintain their insertion order
- Priority can be set when creating
CDNSourceobjects - Sources are automatically re-sorted when added via
add_source()
record_latency(), latency_estimate()). A source can measure more than twice as slow as a lower-priority source, with at least eight samples on each. Then get_sources() moves it behind that source for the rest of the process. Sources without samples keep their configured position. reset() clears the estimates.
Stalled requests are hedged. In the async pipeline the first source’s request waits hedge_delay() seconds. That delay is two times the source’s moving average, clamped between 1 second and cdn_hedge_max_delay. A request that has not answered by then is re-issued on the next source, and the first usable answer wins. The abandoned request counts as a latency sample at the hedge delay, so a slow mirror’s estimate stays honest. A healthy source sends one request per file. cdn_hedge=false restores the one-after-the-other loop.
4. JSON Minification Support
Thecdn_use_minification option makes tif1 request file.min.json in place of file.json from custom mirrors. It is off by default.
The default CDNs ignore the option. The TracingInsights data repositories have no
.min.json files, and jsDelivr minifies only JavaScript and CSS. jsDelivr, Hugging Face and StaticDelivr all return 404 for them (checked 2026-10-14). tif1 never sends a minified request to those three sources.- When
use_minification=Trueon aCDNSource, URLs are transformed:file.json→file.min.json - The source must serve the
.min.jsonfile. Otherwise the request returns 404 and tif1 moves to the next source - The global
cdn_use_minificationoption applies the flag to custom mirrors only. jsDelivr, Hugging Face and StaticDelivr sources always keep it off
- Only for a custom mirror that serves a
.min.jsoncopy of each payload. Setuse_minification=Trueon thatCDNSourceonly
5. Configurable Custom Sources
Add custom CDN endpoints to support:- Private mirrors: Host a private copy of F1 data for guaranteed availability
- Regional CDNs: Optimize performance for specific geographic regions
- Corporate proxies: Route requests through internal infrastructure
- Development/testing: Point to local servers or staging environments
- Backup sources: Add redundant sources for applications that require high availability
- Programmatic: Use
add_source()method to add sources at runtime - Config file: Define sources in
~/.tif1rcJSON configuration - Environment variable: Set
TIF1_CDNScomma-separated list - Runtime config: Call
config.set("cdns", [...]), thenget_cdn_manager().reload()
6. Singleton Pattern for Global State
The CDN manager uses a singleton pattern to ensure consistent state across the entire application:- Single source of truth: All data fetching operations use the same CDN manager instance
- Shared health tracking: Failure counts and circuit breaker state are global
- Consistent configuration: Configuration changes affect all subsequent requests
- Thread-safe reads: Multiple threads can safely read CDN sources
- Memory efficient: Only one CDN manager instance exists per process
7. Transparent Integration
The CDN system operates in the background. Applications typically do not interact with it directly:- Automatic initialization: The CDN manager is created when
tif1.cdnis first imported (the first session load imports it) - Zero-configuration default: Works out-of-the-box with the jsDelivr → Hugging Face → StaticDelivr chain
- Session API integration:
Session.load()automatically uses CDN system - Error handling: Network errors are caught and trigger automatic fallback
- Logging: Debug-level logs show which CDN source is being used
Architecture
The CDN system consists of three main components working together:1. CDNSource (Data Class)
Represents a single CDN endpoint with configuration:- Store CDN configuration (URL, priority, settings)
- Format complete URLs for specific resources via
format_url() - Support minification by transforming file paths
2. CDNManager (Orchestrator)
Manages multiple CDN sources and implements fallback logic:- Initialize sources from configuration (file, env vars, defaults)
- Validate source URLs (HTTPS only, block raw.githubusercontent.com)
- Track health via failure counters (circuit breaker pattern)
- Provide available sources via
get_sources()(filters disabled/failed sources) - Implement fallback logic via
try_sources()(tries sources in priority order). Async fetchers use the async counterparttry_sources_async(year, gp, session, path, fetch_func). It walks the same source chain with awaitable fetch functions - Manage source lifecycle (
add_source(),reset(),reload())
3. Global Singleton
A singleCDNManager instance shared across the application:
- Ensure consistent CDN state across all data fetching operations
- Provide global access point via
get_cdn_manager()function - Initialize once on first import, reuse for all subsequent calls
Data Flow
Here’s how a typical data fetch flows through the CDN system:When to Use This API
Most users never need to interact with the CDN API directly. It works transparently in the background. Use this API in these situations:Direct Interaction Scenarios
-
Adding custom CDN sources: Add a private mirror, regional CDN, or backup source
-
Monitoring CDN health: Track which sources are failing, or build health dashboards
-
Debugging network issues: Understand which CDN is used or why requests fail
-
Checking the minification flag: Keep it off, because the default CDNs return 404 for
.min.jsonfiles -
Testing fallback behavior: Verify that the application handles CDN failures gracefully
-
Implementing custom retry logic: Build a custom data fetching layer
-
Recovering from widespread failures: The network was down, and all sources now need a reset
Configuration Scenarios
- Setting up regional CDNs: Optimize performance for specific geographic regions
- Implementing cost optimization: Use free CDN as primary, paid CDN as backup
- Corporate environments: Route through internal proxies or mirrors
- Development/testing: Point to local servers or staging environments
- High-availability requirements: Add multiple backup sources for production applications
The default configuration uses three sources in priority order. The sources are jsDelivr (primary), Hugging Face buckets (
https://huggingface.co/buckets/tracinginsights, fallback), and StaticDelivr (last resort).jsDelivr and StaticDelivr serve the TracingInsights GitHub data repositories. The Hugging Face buckets mirror the same repositories, so they stay available when the GitHub-backed CDNs fail.For most users, the default configuration needs no changes.Quick Start
For most use cases, the CDN API needs no direct interaction. It works transparently in the background. These common operations serve advanced use cases:Basic Usage (Transparent)
Inspecting CDN Status
Adding Custom CDN Sources
Enabling Minification
Keep the globalcdn_use_minification option off: the default CDNs do not serve .min.json files (see JSON Minification Support). Enable minification only on a custom source that serves minified copies:
Monitoring and Recovery
Configuration via File
Instead of programmatic configuration, define CDN sources in~/.tif1rc:
Configuration via Environment Variables
API Reference
Module-Level Functions
get_cdn_manager()
CDNManager object, ensuring consistent CDN state across the entire application.
Why Singleton Pattern?
The singleton pattern is used because:
- Shared health tracking: CDN health monitoring must be consistent across all data fetching operations
- Global configuration: Configuration changes should affect all subsequent requests
- Consistent failure counts: Circuit breaker state must be global to prevent redundant retries
- Memory efficiency: Only one CDN manager instance exists per process
- Thread safety: Single instance simplifies concurrent access patterns
CDNManager: The global singleton instance
- Thread-safe operations:
get_sources(), readingsourceslist, reading_failure_counts - Not thread-safe:
add_source(),mark_failure(),mark_success(),reset()
- Configure all sources during initialization (single-threaded)
- Only read sources during concurrent execution
- Use locks to modify sources from multiple threads
CDNManager Class
TheCDNManager class is the core orchestrator of the multi-source CDN system. It maintains a list of CDN sources and tracks their health. It implements fallback logic with circuit breaker patterns. It also provides methods for managing sources.
Initialization
The CDN manager initializes once, whentif1.cdn is first imported. The first session load imports it. Configuration changes made after that point do not change the existing manager until reload() runs. During initialization (and in reload()), the following steps occur:
Initialization Sequence:
- Load configuration: Reads the
cdnslist from the tif1 config. The config merges~/.tif1rcand theTIF1_CDNSenvironment variable - Parse source URLs: The config splits a comma-separated
TIF1_CDNSvalue into individual URLs - Validate sources: Drops URLs that are not HTTPS and URLs on the blocked host
raw.githubusercontent.com - Create CDNSource objects: Wraps each URL in a
CDNSource. The name isjsDelivr,HuggingFace,StaticDelivr, orCDN {n}for other hosts - Assign priorities: Each source gets its 1-based position in the
cdnslist as its priority, so the list order is the try order - Initialize health tracking: Sets up
_failure_countsdictionary with all sources at 0 failures and sets_max_failuresto 3 - Fallback to defaults: If no valid sources found, uses jsDelivr, Hugging Face buckets, and StaticDelivr as defaults
- Environment variable:
TIF1_CDNS(comma-separated list of HTTPS URLs) - Config file: The file named by
TIF1_CONFIG_FILE,./.tif1rc(ifTIF1_TRUST_CWD_CONFIG=true), or~/.tif1rc; tif1 reads the first one that exists - Default:
https://cdn.jsdelivr.net/gh/TracingInsights,https://huggingface.co/buckets/tracinginsights, andhttps://cdn.staticdelivr.com/gh/TracingInsights(in priority order)
- URLs must start with
https://(HTTP is rejected for security) - URLs containing
raw.githubusercontent.comare rejected (rate limits) - A
raw.githubusercontent.comURL is logged as a warning and skipped. Non-HTTPS URLs are dropped without a warning - If all URLs are invalid, falls back to the default jsDelivr → Hugging Face → StaticDelivr chain
~/.tif1rc:
Attributes
sources
- Always sorted: Maintained in priority order (1, 2, 3, …)
- Includes all sources: Both
enabled=Trueandenabled=Falsesources - Includes failed sources: Sources that have exceeded failure threshold
- Mutable: Can be modified directly, but prefer using
add_source()for automatic sorting
- Iterate over all sources (including disabled) for health reporting
- Count total configured sources
- Inspect source configuration (URLs, priorities, settings)
- Debug CDN setup
_failure_counts
_max_failures (default: 3), it is automatically excluded from get_sources() until reset.
Structure:
- Initialized to 0: All sources start with 0 failures (healthy state)
- Incremented on failure:
mark_failure()increments the count - Reset on success:
mark_success()resets to 0 - Circuit breaker threshold: Sources with count >=
_max_failuresare excluded - Persistent across requests: Counts persist for the lifetime of the CDN manager instance
mark_failure(source_name)to incrementmark_success(source_name)to resetreset()to reset all sources
_max_failures
- Closed (healthy):
failure_count < _max_failures→ Source is available - Open (disabled):
failure_count >= _max_failures→ Source is excluded fromget_sources() - Recovery: A manual
mark_success(),reset(), orreload()sets the count to 0 → Source becomes available again
- Not too sensitive: Tolerates transient network issues (1-2 temporary failures)
- Not too lenient: Quickly disables consistently failing sources (3 failures = clear pattern)
- Fast recovery: Single successful request resets the counter
Methods
get_sources()
- Disabled sources: Sources with
enabled=False - Failed sources: Sources where
_failure_counts[name] >= _max_failures(circuit breaker open)
list[CDNSource]: Available sources sorted by priority (ascending)- Empty list if all sources are disabled or have exceeded failure threshold
- Sources are sorted by priority: 1, 2, 3, … (lower number = higher priority)
- Sources with the same priority maintain their insertion order (stable sort)
- The first source in the list is tried first by
try_sources()
- Data fetching: Primary method used by
try_sources()to determine which sources to try - Health monitoring: Check how many sources are now available
- Debugging: Understand which sources the next request uses
- Load balancing: Implement custom logic based on available sources
add_source()
sources list in priority order, and its failure count is initialized to 0.
This method is useful for:
- Adding private CDN mirrors
- Adding regional CDN endpoints for better performance
- Adding backup sources for increased reliability
- Testing custom CDN configurations
source(CDNSource): The CDN source object to add
- Sources are automatically sorted by priority after insertion
- If a source with the same name already exists, both are kept. They share one failure counter, and the call resets it to 0. Use unique names
- The new source is immediately available for use via
get_sources() - Failure count is initialized to 0 (healthy state)
mark_failure()
get_sources() until reset.
This implements a circuit breaker pattern to prevent wasting time on consistently failing sources. The circuit breaker helps:
- Reduce latency by skipping known-bad sources
- Prevent cascading failures
- Allow sources to recover (via
reset()ormark_success())
source_name(str): Name of the CDN source that failed
- Increments failure count by 1
- If count reaches
_max_failures(3), logs a warning - Source is automatically excluded from
get_sources()after threshold - Does not disable the source permanently—can be recovered via
reset()ormark_success()
try_sources() calls this method automatically when a CDN request fails. Call it manually only for custom fetching logic.
Example:
try_sources() Classifies Errors
The fetch function only raises. try_sources() decides from the exception whether to call mark_failure():
mark_success()
try_sources() calls this method. The count then goes back to 0. A source with an open circuit is never tried, so only a manual call or reset() brings it back.
Parameters:
source_name(str): Name of the CDN source that succeeded
- Sets failure count to 0 for the specified source
- Source becomes immediately available via
get_sources()if its circuit was open (theenabledflag does not change) - Called automatically by
try_sources()andtry_sources_async()on successful fetch
try_sources() calls this method automatically when a CDN request succeeds. Applications rarely need to call it manually.
Example:
reset()
- After network outage: When the network connection was down and all sources failed
- After CDN maintenance: When CDN providers have resolved their issues
- Testing: To test fallback behavior from a clean state
- Manual recovery: To force retry of all sources
- Resets failure count to 0 for every source in
cdn.sources - All sources become immediately available via
get_sources()(ifenabled=True) - Does not modify source configuration (priority, URLs, minification settings)
reload()
cdns and cdn_use_minification config values, then resets all failure counts. Call it after config.set("cdns", ...) to apply the change to the running manager.
Behavior:
- Runs the same steps as Initialization
- Drops sources that were added with
add_source() - Sets
_max_failuresback to 3
try_sources()
- Get available sources: Calls
get_sources()to get enabled, healthy sources sorted by priority - Check availability: If no sources available, raises
NetworkErrorimmediately - Try each source (in priority order):
- Format URL via
source.format_url(year, gp, session, path) - Log debug message:
"Trying CDN: {source.name} - {url}" - Call
fetch_func(url)to try the HTTP request - On success: Call
mark_success(source.name)and return data (done!) - On
DataNotFoundError(404): Fall through to the next source. The failure count is NOT incremented (a 404 is not a CDN health problem). - On an exception whose status is 403 or 404 (
response.status_code, or thestatus_codecontext key of aNetworkError): Same as a 404 - On
InvalidDataError: Raise it at once - On other exceptions: Log warning, call
mark_failure(source.name), try next source
- Format URL via
- All sources failed: Raise
DataNotFoundErrorif every source refused with a 4xx status other than 408 or 429; otherwiseNetworkErrorwith details from last exception
year(int): Season year (for example, 2021, 2022, 2023)gp(str): Grand Prix name, URL-encoded (for example, “Belgian%20Grand%20Prix”, “Monaco%20Grand%20Prix”)- Important: Must be URL-encoded (spaces as
%20, not+) - Use
urllib.parse.quote()if encoding manually
- Important: Must be URL-encoded (spaces as
session(str): Session name, URL-encoded likegp(for example, “Race”, “Qualifying”, “Practice%201”, “Sprint”)path(str): Resource path relative to session directory (for example, “drivers.json”, “session_laptimes.json”, “VER/laptimes.json”, “VER/1_tel.json”)fetch_func(Callable[[str], Any]): Function that takes a URL string and returns fetched data- Should raise
DataNotFoundErrorfor 404 responses - Should raise other exceptions for network/server errors
- Pass exception details as context keywords, for example
DataNotFoundError(url=url). These classes build their own message, so amessage=keyword raisesTypeError - Return type can be any (dict, list, bytes, etc.)
- Should raise
Any: Fetched data from the first successful CDN source (return type depends onfetch_func)
DataNotFoundError: If every CDN source refused the resource with a 4xx status (404 included). A 408 or 429 means “try again later”, so it givesNetworkErrorinstead- Raised only after all sources were tried
- Indicates the requested data does not exist on any mirror
NetworkError: If any source failed for another reason, or no source is available- Includes URL path and status code from last exception
- Indicates all sources are unavailable or experiencing issues
InvalidDataError: Raised at once whenfetch_funcraises it. Another CDN cannot fix an invalid payload, so no other source is tried
-
Data not found (404 or 403): Possibly a stale or divergent mirror
- Falls through to the next CDN source (the payload may exist elsewhere)
- Does NOT increment failure counter (not a CDN health problem)
- If every source refuses with a 4xx status other than 408 or 429, raises
DataNotFoundError
-
Network/server errors: Temporary error—CDN source is failing
- Logged as warning
- Failure counter incremented via
mark_failure() - Next source is tried
- If all sources fail, raises
NetworkError
try_sources_async(). Its fetch_func is a coroutine function that receives the CDNSource and the formatted URL:
try_sources(), with one difference. After a refusal (404 or 403), it fetches all remaining sources at the same time and returns the first payload that arrives.
- Latency: Each failed source adds latency (timeout + retry time). Use reasonable timeouts in
fetch_func. - Circuit breaker: After 3 failures, sources are automatically disabled, reducing latency for subsequent requests.
- Minification: Leave
use_minificationoff for the default CDNs. They return 404 for.min.jsonfiles. - Connection pooling: Reuse HTTP sessions in
fetch_funcfor better performance. - Parallel fetching:
try_sources()is sequential. For parallel fetching of multiple resources, call it multiple times concurrently.
try_sources()uses no locks. Concurrent calls are safe for reads, but two threads can update the same failure counter at the same time- A lost counter update only delays or advances the circuit breaker by one failure
- In high-concurrency scenarios, consider external synchronization for
mark_failure()/mark_success()calls
CDNSource
Dataclass representing a CDN source configuration.Constructor
name: Human-readable source namebase_url: Base URL for the CDN (thecdnsconfig accepts only HTTPS URLs)priority: Priority level (lower number = higher priority, default: 0)enabled: Whether source is now enabled (default: True)use_minification: Enable JSON minification (appends.minbefore.json, default: False)
Methods
format_url()
year: Season year (for example, 2021)gp: Grand Prix name, URL-encoded (for example, “Belgian%20Grand%20Prix”)session: Session name, URL-encoded (for example, “Race”, “Practice%201”)path: Resource path (for example, “drivers.json”)
- Complete CDN URL string
base_url:
use_minification=True and path ends with .json, the path is transformed from file.json to file.min.json.
Example:
Configuration
CDN behavior can be configured via the global config. The CDN manager reads these keys whentif1.cdn is first imported and on each reload() call. Set them before the first session load, or call reload() after the change:
Practical Example: 2021 Belgian Grand Prix
Here’s a complete example showing how the CDN system works when fetching data for the 2021 Belgian Grand Prix Race:Best Practices
Configuration
- Use HTTPS only: HTTP CDN sources are rejected for security
- Leave minification off: The default CDNs return 404 for
.min.jsonfiles - Configure via file: Use
~/.tif1rcfor persistent configuration - Set priorities wisely: Lower number = higher priority (1 before 2)
Error Handling
- Distinguish error types: Handle
DataNotFoundError(404) vsNetworkError(all sources failed) - Let CDN manager handle fallback: Do not implement custom retry logic
- Log failures: Enable debug logging to see which CDN is being used
Performance
- Leave minification off: A
.min.jsonrequest costs one 404 per payload on the default CDNs - Use connection pooling: Reuse HTTP sessions in fetch functions
- Set reasonable timeouts: Balance between patience and responsiveness
- Monitor circuit breaker: Check failure counts to identify problematic sources
Reliability
- Add backup sources: At least 2-3 sources for production applications
- Use regional CDNs: Optimize for the user’s geographic location
- Implement auto-recovery: Periodically reset sources after network issues
- Monitor health: Track availability and failure rates
Security
- Never use raw.githubusercontent.com: Rate limited and blocked by tif1
- Validate custom CDN URLs: Ensure they are trusted sources
- Use HTTPS everywhere: HTTP sources are automatically rejected
- Audit CDN sources: Regularly review configured sources
Troubleshooting
All CDN Sources Failing
Symptoms:NetworkError: Network request failed: url='2021/Belgian%20Grand%20Prix/Race/drivers.json'
Solutions:
- Check network connectivity:
ping cdn.jsdelivr.net - Verify CDN URLs are accessible in browser
- Reset all sources:
cdn.reset() - Check firewall/proxy settings
- Enable debug logging to see detailed errors
Data Not Found (404)
Symptoms:DataNotFoundError: Data not found
Cause: Requested data does not exist (not a CDN failure)
Solutions:
- Verify session exists: Check year, GP name, session name
- Check data availability: Some sessions may not have all data types
- Use correct path: Ensure
pathparameter is correct (for example, “drivers.json”)
Slow Performance
Symptoms: Data fetching is slow Solutions:- Check that
cdn_use_minificationis off:config.get("cdn_use_minification") - Add regional CDN: Closer to the user’s geographic location
- Check network speed: Run speed test
- Use connection pooling: Reuse HTTP sessions
- Increase timeout: May be timing out prematurely
Circuit Breaker Stuck Open
Symptoms: Sources remain disabled after network recovery Solution: Reset failure countsRelated APIs
- HTTP Session API: Underlying HTTP client used by CDN system
- Retry API: Retry logic and circuit breaker patterns
- Config API: CDN configuration management
- Exceptions API: NetworkError and DataNotFoundError details