Overview
Thetif1 configuration system controls every part of the library’s behavior. It covers network settings, caching strategies, data validation, and performance settings. The configuration architecture is flexible. It allows tuning for a specific use case: low-latency data access, high-throughput batch processing, or development debugging.
This guide covers all configuration options, their interactions, performance implications, and best practices for different deployment scenarios. Interactive Jupyter notebooks, production data pipelines, and CI/CD environments have different requirements. This guide helps with the configuration of tif1 for each one.
Configuration Philosophy
tif1 is built for performance-first operation. The default configuration values give good out-of-the-box performance for most use cases. The library also exposes dozens of tuning parameters for advanced use. These parameters minimize latency or maximize throughput for a specific workload.
The configuration system follows a layered override model. Settings can be specified at multiple levels with clear precedence rules:
- Default values — Hardcoded defaults optimized for general use
- Configuration file (
.tif1rc) — Persistent settings stored in the user’s home directory - Environment variables (
TIF1_*) — Deployment-specific overrides - Programmatic API (
config.set()) — Runtime adjustments based on workload
Configuration Architecture
The configuration system is implemented as a singleton pattern. This pattern ensures that all parts of an application share the same configuration state. This design provides several benefits:- Consistency: All modules and components use the same settings
- Efficiency: No redundant configuration loading or memory overhead
- Simplicity: Single source of truth for all configuration values
- Thread-safe reads: Multiple threads can safely read configuration values
- Dynamic updates: Most keys are read on each operation, so a change applies to the next call (see
set()for the exceptions)
When to Configure
Consider customizing the configuration when:- Performance is critical — The workload needs minimal latency or maximum throughput for time-sensitive applications
- Network conditions vary — A slow connection, a corporate proxy, DNS issues, or specific network requirements apply
- Resource constraints exist — Memory usage, connection pools, or concurrent requests need limits due to system limitations
- Data validation is needed — Data issues need debugging, quality checks need implementing, or compliance needs strict validation
- Caching behavior matters — Fresh data, shared storage, or access patterns need different cache settings
- Backend preferences differ — A workflow needs the polars backend or specific DataFrame behavior
- Deployment environment differs — Production, staging, development, or CI/CD environments have different performance, reliability, and debugging requirements
- Workload characteristics change — The workload switches between interactive analysis (low concurrency) and batch processing (high concurrency)
- Compliance requirements exist — Data storage locations need control, certain features need disabling, or regulatory requirements apply
- Debugging is required — Verbose logging, validation, or specific diagnostic information is needed
Configuration Use Cases
Different use cases benefit from different configuration strategies: Interactive Analysis (Jupyter/IPython)- Lower concurrency (5-20 workers) to avoid overwhelming the system
- Enable ultra cold start for fast initial loads
- Moderate cache settings for iterative exploration
- Optional validation for data quality checks
- High concurrency (50-200 workers) for maximum throughput
- Aggressive connection pooling to handle burst traffic
- Large cache sizes to minimize redundant fetches
- Disable validation for maximum performance
- Balanced concurrency based on expected load
- Reliable retry and circuit breaker settings
- Persistent cache with appropriate sizing
- Complete logging for monitoring
- CI mode enabled, so payload fetches skip the persistent cache
- Shorter timeouts to fail fast
- Minimal caching to ensure fresh data
- Validation enabled to catch data issues early
- Cache disabled for fresh data on every run
- Validation enabled to catch issues early
- Verbose logging (DEBUG level)
- Lower concurrency for easier debugging
Configuration Loading Process
Whentif1 initializes, it loads configuration in the following sequence:
- Initialize defaults — All configuration keys start with sensible default values hardcoded in the library
- Load config file — If a
.tif1rcfile exists, its values override defaults. The library searches for config files in this order:- Path specified in
TIF1_CONFIG_FILEenvironment variable (if set) .tif1rcin current working directory (only ifTIF1_TRUST_CWD_CONFIG=true)~/.tif1rcin user’s home directory (default location)
- Path specified in
- Apply environment variables — Any
TIF1_*environment variables override file settings - Accept runtime changes — Programmatic
config.set()calls override everything
- Explicit path (
TIF1_CONFIG_FILEenv var) — Highest priority, useful for testing or custom deployments - Current directory (
./.tif1rc) — Only checked ifTIF1_TRUST_CWD_CONFIG=truefor security - Home directory (
~/.tif1rc) — Default location, most common for user-specific settings
.tif1rc from the current working directory. This prevents malicious config files in untrusted directories. Set TIF1_TRUST_CWD_CONFIG=true only if the current directory is trusted.
Example Configuration Loading:
get()), not when they are set. This design allows any value to be set programmatically. Invalid values are rejected when accessed, falling back to the provided default. Validation includes:
- Type checking of numeric keys and lists
- Range validation (positive numbers, valid enums)
- Format validation (HTTPS URLs for CDNs)
- Path expansion (~ to home directory)
get() then returns its default argument (None when no default is given).
Configuration File Management
File Location and Discovery
The.tif1rc configuration file is a JSON file that stores persistent configuration settings. The library searches for this file in multiple locations with a specific precedence order.
Default Location:
TIF1_CONFIG_FILE environment variable:
.tif1rc file in the project directory. For security reasons, this file is only loaded if it is explicitly enabled:
File Format
The configuration file must be valid JSON with a single object containing key-value pairs:- Must be valid JSON (not JSON5, JSONC, or other variants)
- Root element must be an object
{} - Keys must be strings matching configuration key names
- Values must match expected types (string, number, boolean, array)
- Comments are not supported (JSON does not allow comments)
- Trailing commas are not allowed
get() rejects it when it is read.
Creating Configuration Files
Method 1: Programmatic Creation The recommended way to create a configuration file is using thesave() method:
Managing Multiple Configurations
For different environments or use cases, maintain multiple configuration files. Switch between them using environment variables. Example: Development vs ProductionConfiguration File Best Practices
- Version control: Commit project-specific
.tif1rcfiles to version control - Documentation: Add comments in a separate README explaining configuration choices
- Validation: Test configuration files before deploying to production
- Backup: Keep backups of working configurations before making changes
- Minimal: Only include settings that differ from defaults to keep files small
- Security: Never commit sensitive information (API keys, passwords) to config files
Environment Variables
Every configuration key exceptcdn_use_minification can be overridden using environment variables with the TIF1_ prefix. Environment variables take precedence over configuration files but are overridden by programmatic config.set() calls.
Environment Variable Naming
Configuration keys are converted to environment variables by:- Adding the
TIF1_prefix - Converting to uppercase
lib→TIF1_LIBenable_cache→TIF1_ENABLE_CACHEmax_workers→TIF1_MAX_WORKERScache_dir→TIF1_CACHE_DIR
Type Conversion
Environment variables are strings, so the library automatically converts them to the appropriate type: Boolean Values:- True:
1,true,yes,on(case-insensitive) - False:
0,false,no,off(case-insensitive)
- Integers:
42,100,0 - Floats:
3.14,2.0,0.5
- Used as-is
- Comma-separated strings
Complete Environment Variable Reference
Environment Variable Use Cases
Docker Containers:Configuration API Reference
get_config()
Config object is a singleton. There is only one instance per Python process. All calls to get_config() return the same object. This ensures configuration consistency across the entire application.
Returns:
Config— The global configuration singleton instance
config.set()) during concurrent operations may lead to race conditions. It is recommended to configure the library once at startup before spawning threads or processes.
Example:
Configuration Methods
TheConfig object provides three primary methods for configuration values: get() for reading, set() for modifying, and save() for persisting changes to disk.
get(key, default=None)
Retrieve the value for a specific configuration key. This method includes built-in validation for many configuration keys. It ensures values are within acceptable ranges and of the correct type.
key(str) — The configuration key name (case-sensitive)default(Any, optional) — Default value to return if the key does not exist or validation fails. Defaults toNone.
Any— The configuration value, ordefaultif the key does not exist or validation fails
get() method performs automatic validation for many configuration keys:
- Numeric values — Must be positive integers or floats for keys like
timeout,max_workers,pool_connections, etc. - Retry count —
max_retriescan be 0 or positive (0 means no retries) - Backoff factor —
retry_backoff_factormust be >= 1.0 - Library selection —
libmust be either"pandas"or"polars" - CDN URLs —
cdnsmust be a list of HTTPS URLs - Path expansion —
cache_dirautomatically expands~to the user’s home directory
default value instead of the invalid value.
Example:
set(key, value)
Update a configuration value in memory for the current Python session. Changes made with set() are not persisted to disk unless save() is explicitly called.
key(str) — The configuration key name (case-sensitive)value(Any) — The new value to set. Type should match the expected type for the key.
None
- Most keys apply to the next operation that reads them
- Cache keys (
cache_dir,memory_*_max_items,cache_commit_interval,sqlite_timeout,missing_payloads_ttl_days) apply when the cache is created. Set them before the first data load - HTTP session keys (pool sizes, keep-alive, resolvers, HTTP/2 and HTTP/3 switches) and the thread-pool size also apply only at creation
cdnsandcdn_use_minificationapply aftertif1.get_cdn_manager().reload()- The circuit breaker keys apply after
tif1.reset_circuit_breaker() - Changes are session-only unless
save()is called - No validation is performed during
set()— validation happens inget() - Custom keys that are not part of the default configuration can be set
- Runtime optimization — Adjust settings based on workload characteristics
- A/B testing — Compare performance with different configurations
- Dynamic tuning — Increase concurrency for large batches, decrease for small queries
- Temporary overrides — Disable caching for a specific operation, then re-enable
Changes made with
set() only affect the current Python session. To make changes permanent, call save() after setting the desired values.save(path=None)
Persist the current in-memory configuration to a JSON file on disk. This makes configuration changes permanent across Python sessions.
path(Path | None, optional) — Path where the configuration file should be saved. IfNone, saves to the default location (~/.tif1rc). Defaults toNone.
None
- Writes the entire current configuration (including defaults) to the specified file
- Creates the file if it does not exist (the parent directory must already exist)
- Overwrites the file if it already exists
- Uses JSON format with 2-space indentation for readability
- Never raises: a failed write (missing directory, no permission) is logged as an error and the method returns
None
The
save() method writes the entire configuration, not just the modified keys. This ensures the saved file is a complete, self-contained configuration.Configuration Keys
Core Settings
Path where the SQLite cache database is stored. By default, tif1 follows FastF1’s OS-specific layout: Windows
%LOCALAPPDATA%/Temp/tif1, macOS ~/Library/Caches/tif1, and Linux/other POSIX ~/.cache/tif1 when ~/.cache exists (otherwise ~/.tif1).Enable Pydantic validation of the race-control (
rcm.json) and weather payloads. Lap times and telemetry have their own keys (validate_lap_times, validate_telemetry).Validation adds overhead. Disabled by default for performance.
Cold-start fast path. When the session is not yet in the persistent cache, payloads skip the cache read and Pydantic validation, and (with
ultra_cold_skip_retries) the per-source retry delays. Fetched payloads are still written to the cache. A session that is already cached loads from the cache as usual.Network Settings
Total attempts for each payload fetch, the first included:
3 means up to two retries. Set it to 0 or 1 for a single attempt without retries.Lower bound for the size of the fetch thread pool. The pool size is the largest of
max_workers, max_concurrent_requests and telemetry_prefetch_max_concurrent_requests.Maximum number of concurrent requests in one batch fetch. The bulk telemetry prefetch uses
telemetry_prefetch_max_concurrent_requests instead.HTTP Session Settings
Disable HTTP/3 (QUIC). The default is HTTP/2 over TCP. In interleaved live A/Bs, HTTP/2 was at parity or faster for the bulk telemetry fetch, and every stalled fetch observed ran over HTTP/3. Set it to
False to negotiate HTTP/3 again.Check the CDN certificate for revocation (OCSP) on the first connection of each process. The certificate chain and host name are verified either way. The first check of a certificate makes a request to the certificate authority before the first data request. With
http_revocation_cache, later processes use the kept answer instead. Set it to False to skip the check.Keep the GOOD answers of the revocation check in the SQLite cache and use them again in later processes. A kept answer is used only for the same certificate, only until its expiry time (
nextUpdate), and never for more than 7 days. REVOKED and UNKNOWN answers, and answers with a signature that tif1 cannot verify, are not kept. A process that uses a kept answer finds a revocation only after that answer expires. Set it to False to check again in each process.Warm the shared CDN connections in daemon threads during
get_session(), so the first requests of a cold load skip the DNS and TLS handshake. The warm-up starts before pandas or polars is imported, and only for a session that is not cached yet. Skipped in offline and CI mode.Fetch the first files of a cold load (
drivers.json, the session lap times) with the Python standard library while pandas or polars loads. niquests loads only when a telemetry request needs it, and the first telemetry requests use the standard-library connections until niquests has an open connection. These requests verify the certificate chain and the host name, but they do not do the revocation check (http_revocation_check). Has no effect when cdn_preconnect is off. Set it to False to send every request through niquests.Freeze the heap in an exit hook so the interpreter’s final garbage collections skip the objects pandas, numpy and pyarrow (or polars) built at import. Saves 40-80 ms at the end of every script. Memory is still released; cyclic garbage alive at exit is not finalized, which CPython does not guarantee at shutdown anyway.
TIF1_EXIT_GC_FREEZE=0 turns it off.Re-issue a stalled request on the next CDN source and take the first answer. Each source keeps a moving average of its request latency. A request waits two times that average before the second request starts, clamped between 1 second and
cdn_hedge_max_delay. A source measured at more than twice the latency of a lower-priority source is re-ranked behind it for the process. A healthy source still sends one request per file.Upper bound, in seconds, for the hedge wait. A source without latency samples uses this value, so a warm edge never hedges and a cold edge hedges from its first wave.
Number of connection pools to maintain. If not set, automatically calculated as An explicit value below
max(256, max_workers, max_concurrent_requests, telemetry_prefetch_max_concurrent_requests).Most users should rely on automatic sizing. Only set explicitly for specific performance tuning.
10 is raised to 10.Maximum connections per pool. If not set, automatically calculated as
max(512, pool_connections * 4) to handle burst traffic.Automatically sized to 4x pool_connections with a minimum of 512. An explicit value is also floored at
max(512, pool_connections), so it cannot shrink the pool below 512. Only override for specific use cases.Maximum requests per keep-alive connection, sent in the
Keep-Alive request header.DNS resolver list, with DNS-over-HTTPS (DoH) support.
tif1 creates the HTTP session with the first resolver in the list. It moves to the next entry only when the session cannot be created with the current one. A DNS failure during a request does not switch the resolver.Default: ["standard", "doh+cloudflare://", "doh+google://"]Resolver Types:standard— System DNS resolver (fastest, but may be blocked or censored)doh+cloudflare://— Cloudflare DNS-over-HTTPS (1.1.1.1)doh+google://— Google DNS-over-HTTPS (8.8.8.8)
<protocol>+<provider>:// form. tif1 translates the older spelling (doh://cloudflare, doh://google, doh://quad9, doh://adguard, doh://nextdns) to this form.Benefits of DoH:- Bypass DNS blocking or censorship
- Improved privacy (encrypted DNS queries)
- Reliability when system DNS is misconfigured
- Consistent resolution across different networks
- System DNS (
standard) is fastest when working correctly - DoH adds latency due to HTTPS overhead
- Keep
standardfirst unless system DNS is blocked or misconfigured
Telemetry Settings
When
True, session.load(telemetry=True) has the effect of session.load(telemetry="lazy"). The load does not fetch telemetry. The first access of a lap of a driver fetches all the laps of that driver.The number of drivers that a cold lazy load fetches in the background. This starts when the code reads the drivers in the order of the laps table. The background requests use the free request slots of the current fetch.
-1 fetches all the drivers that are not loaded. 0 fetches each driver only when the code reads one of its laps.The maximum number of background requests in flight for
lazy_telemetry_lookahead. A background request starts only when no lap that the code waits for is in the queue.Logging Settings
Interval in seconds for logging connection pool statistics. The statistics are logged at
INFO level, so they show only with tif1.setup_logging(logging.INFO) or lower.Advanced Configuration
These settings are for advanced users and performance tuning. Most users should use the defaults.
Cache Internals
Cache Internals
Lifetime in days of an all-CDN-missing verdict (
missing_payloads table). Within the window, tif1 skips the CDN probe for that payload. Environment variable: TIF1_MISSING_PAYLOADS_TTL_DAYS.Age in hours after which the live event schedule of the current or a future season is re-fetched from the
theOehrly/f1schedule repository. 0 re-fetches on every lookup; negative values are rejected. Past seasons come from the vendored files and never refresh. Environment variable: TIF1_SCHEDULE_TTL_HOURS.Merge the official Ergast classification into
Session.results and order Session.drivers by it. Off, every session keeps the lap-derived classification and nothing extra is fetched. Environment variable: TIF1_ERGAST_RESULTS.Ask the Jolpica API for what the TracingInsights Stats repository cannot answer (seasons list, driver lists, status counts, rounds not published yet). Off, those queries raise
ErgastInvalidRequestError or DataNotFoundError. Environment variable: TIF1_ERGAST_FALLBACK.Base URL of the Jolpica API; a self-hosted instance works. Environment variable:
TIF1_JOLPICA_BASE_URL.Lifetime in hours of current-season Stats files and of every Jolpica answer in the cache; past-season Stats files never expire.
0 re-fetches on every lookup. Environment variable: TIF1_ERGAST_TTL_HOURS.Jolpica requests the client allows itself in any one second (the API’s unauthenticated burst limit). Environment variable:
TIF1_ERGAST_RATE_LIMIT_BURST.Jolpica requests the client allows itself in any one hour. The default is fastf1’s Jolpica limit, which is below the API’s unauthenticated limit of 500. Environment variable:
TIF1_ERGAST_RATE_LIMIT_HOURLY.GitHub repository (
owner/repo) holding the Ergast Stats files, served through the configured CDNs. Environment variable: TIF1_STATS_REPO.CDN Configuration
CDN Configuration
List of CDN URLs to use for data fetching, in priority order (jsDelivr primary, Hugging Face buckets fallback, StaticDelivr backup). Must be HTTPS URLs;
raw.githubusercontent.com URLs are skipped. The CDN manager reads this key when it is created: call tif1.get_cdn_manager().reload() after a change.Request
.min.json payloads instead of .json. The three default CDNs do not serve .min.json. jsDelivr and StaticDelivr answer 404, and Hugging Face mirrors the files as they are. So tif1 ignores this key for them. It has an effect only for a custom mirror in cdns that serves minified files. No environment variable maps to this key.Retry & Circuit Breaker
Retry & Circuit Breaker
Upper bound in seconds of the random delay added to each retry wait in the batch fetch path when
retry_jitter is on; 0.0 adds nothing. The retry_with_backoff decorator uses multiplicative jitter (50-150% of the delay) instead and ignores this key.Maximum delay between retries in seconds, in the batch fetch path. The
retry_with_backoff decorator does not cap its delays.Number of consecutive failures before circuit breaker opens. Read when the global circuit breaker is created (at import) and by
reset_circuit_breaker(); call the latter after changing it at runtime.Connection Pool Exhaustion
Connection Pool Exhaustion
Data Validation
Data Validation
Prefetch Strategies
Prefetch Strategies
Start the all-laps telemetry prefetch in a background thread once laps are loaded. Takes effect only when
prefetch_all_telemetry_on_first_lap_request is also True.Other Advanced Settings
Other Advanced Settings
Offline mode: payloads come from the persistent cache only. A payload that is not cached fails with
NetworkError, and no request is sent. The live schedule is not downloaded.CI mode: payload fetches neither read nor write the persistent cache (the per-session memo still applies), and
get_session() skips the CDN preconnect.Cast the
Driver, Team, Compound and TrackStatus lap columns to pl.Categorical in polars laps frames. Pandas laps frames always use the category dtype for these columns.Configuration file format
The.tif1rc file is a JSON file with key-value pairs:
- Default:
~/.tif1rc - Custom: Specify path with
TIF1_CONFIG_FILEenvironment variable
Environment Variables
Every configuration key exceptcdn_use_minification can be set via environment variables with the TIF1_ prefix:
Configuration Precedence
Configuration is loaded in this order (later overrides earlier):- Default values (hardcoded in library)
.tif1rcfile (~/.tif1rcorTIF1_CONFIG_FILE)- Environment variables (
TIF1_*) - Programmatic calls (
config.set())
Common configuration patterns
High-Performance Setup
Low-Latency Setup
Development Setup
Production Setup
Setting Log Level
tif1 has no log-level config key. Set the log level with setup_logging:
Configuration Patterns and Recipes
This section provides configuration recipes for common use cases, with detailed explanations of why each setting is chosen.Maximum Performance Configuration
For absolute maximum throughput in batch processing scenarios:- High worker counts maximize parallel fetching
- Large connection pools prevent pool exhaustion
- Long keep-alive reduces connection overhead
- Disabled validation eliminates CPU overhead
- Aggressive prefetching reduces sequential fetches
- Large caches reduce redundant network requests
- High memory usage
- May overwhelm slower systems
- Not suitable for resource-constrained environments
Minimum Latency Configuration
For interactive analysis where first-byte latency matters most:- Ultra cold start skips cache reads, validation and retry delays on uncached sessions
- DoH can be faster than misconfigured system DNS
- Moderate concurrency balances speed and resource usage
- Short timeout fails fast on slow connections
- HTTP/2 multiplexing reduces connection overhead
- Selective prefetching reduces wait time for common operations
- May skip retries on transient failures
- Short timeout may fail on slow networks
- Moderate concurrency limits maximum throughput
Development and Debugging Configuration
For development environments where debugging and data quality matter more than performance:- Disabled cache ensures fresh data on every run
- Validation catches data quality issues early
- Low concurrency makes logs easier to follow
- Long timeout accommodates debugging pauses
- The validated load path runs on every session
- Verbose logging provides detailed diagnostic information
- Much slower than production configuration
- High CPU overhead from validation
- Verbose logs can be overwhelming
Production Service Configuration
For production services that need reliability, performance, and observability:- Balanced concurrency handles typical load
- Reliable retry settings handle transient failures
- Circuit breaker prevents cascading failures
- Reasonable timeout balances reliability and speed
- Production logging reduces noise
- Optimized pooling handles burst traffic
- Not maximum performance (prioritizes reliability)
- Higher memory usage than minimal configuration
- May be overkill for low-traffic services
CI/CD Pipeline Configuration
For continuous integration and testing environments:- CI mode skips the persistent cache and the CDN preconnect
- Disabled cache ensures tests use fresh data
- Validation catches data quality regressions
- Low concurrency respects CI runner limits
- Short timeout and few retries fail fast
- Only the system resolver is configured
- Slower than production configuration
- May fail on transient network issues
- Not suitable for performance testing
Resource-Constrained Configuration
For systems with limited CPU, memory, or network bandwidth:- Minimal concurrency reduces CPU and memory usage
- Small caches limit memory usage
- Disabled prefetching reduces unnecessary fetches
- Ultra cold start skips cache reads and validation on uncached sessions
- Much slower than high-performance configuration
- Sequential operations dominate execution time
- Not suitable for large-scale analysis
Offline/Cache-Only Configuration
For working with previously cached data without network access:- Offline mode prevents network requests
- Cache must be enabled to serve data
- No retries since network is unavailable
- Cache misses fail at once, without a request
- Only works with previously cached data
- Cache misses result in immediate failures
- No way to fetch new data
High-Reliability Configuration
For scenarios where reliability matters more than performance:- Conservative concurrency reduces load on CDN
- Aggressive retries handle transient failures
- Long timeout accommodates slow networks
- Conservative circuit breaker tolerates more failures
- Later resolvers apply only when the HTTP session cannot be created with earlier ones
- Cache provides redundancy
- Slower than performance-optimized configuration
- May retry excessively on persistent failures
- Higher latency due to conservative settings
Memory-Optimized Configuration
For minimizing memory usage while maintaining reasonable performance:- Moderate concurrency balances speed and memory
- Small caches reduce memory footprint
- Frequent commits keep fewer writes pending in an open transaction
- Disabled prefetching reduces memory usage
- Slower than high-performance configuration
- More frequent disk I/O from cache commits
- May not be suitable for large-scale analysis
Configuration Troubleshooting
Common Configuration Issues
Issue: Configuration changes not taking effect Symptoms:- Changes made with
config.set()do not seem to work - Environment variables are ignored
- Config file changes do not apply
- Check configuration precedence (programmatic > env > file > defaults)
- Verify environment variable names (must be
TIF1_prefix, uppercase) - Ensure config file is valid JSON
- Check config file location (use
TIF1_CONFIG_FILEto specify) - Restart Python process after changing config file
- Check for validation failures in logs
- For keys that apply only when an object is created, see
set()
Issue: Poor performance despite high concurrency settings Symptoms:
- High
max_workersbut slow execution - Connection pool exhaustion warnings
- Low CPU utilization
- Check if connection pool is too small
- Verify network bandwidth is not saturated
- Check if CDN is rate limiting
- Ensure cache is enabled
- Monitor connection reuse rate
Issue: High memory usage Symptoms:
- Python process using excessive memory
- Out of memory errors
- System slowdown
- Reduce cache sizes
- Lower concurrency
- Disable prefetching
Issue: Frequent timeout errors Symptoms:
- Many timeout errors in logs
- Slow data loading
- Inconsistent performance
- Increase timeout value
- Check network connectivity
- Try different DNS resolvers
- Reduce concurrency
- Increase retries
Issue: Cache not working Symptoms:
- Every request hits the network
- No performance improvement on repeated queries
- Cache directory empty
- Verify cache is enabled
- Check cache directory permissions
- Ensure cache directory exists
- Check disk space
- Verify SQLite timeout is not too short
Best Practices
Configuration Strategy
- Start with defaults — The default configuration is optimized for most use cases. Only change settings for a specific need.
-
Use config files for persistent settings — Store common settings in
~/.tif1rcfor user-specific configuration that persists across all projects. - Use environment variables for deployment — Configure per-environment settings (dev, staging, production) using environment variables without modifying code.
- Use programmatic API for runtime changes — Adjust settings dynamically based on workload characteristics or user preferences.
- Document the configuration — Keep a README or comments explaining why specific settings were chosen.
- Test configuration changes — Verify performance impact before deploying to production. Use benchmarks to measure improvements.
- Monitor in production — Track connection stats, cache hit rates, and error rates to validate configuration choices.
-
Version control project configs — Commit project-specific
.tif1rcfiles to version control for reproducibility.
Performance Optimization
-
Let pool sizing auto-calculate — Only override
pool_connectionsandpool_maxsizefor specific tuning needs. The automatic sizing works well for most cases. - Monitor connection reuse — Enable connection stats logging to track connection reuse rate. Low reuse means that many requests open new connections.
- Balance concurrency and resources — Higher concurrency is not always better. Find the right level for the system and network.
-
Try the polars backend for large datasets — Set
libto"polars"and benchmark the workload; the better backend depends on the analysis. - Enable caching — Cache improves performance for repeated queries. Only disable for debugging or when fresh data is critical.
- Tune prefetching — Enable prefetching for common access patterns, but disable for memory-constrained environments.
- Use ultra cold start — Keep it enabled for the fastest first load of an uncached session. Disable it to run the validated path on every load.
Reliability
- Keep validation disabled in production — Enable only for debugging or data quality checks. Validation adds significant overhead.
- Configure reliable retry settings — Use exponential backoff with jitter to handle transient failures gracefully.
- Set appropriate timeouts — Balance between failing fast and tolerating slow networks. 30-60 seconds is reasonable for most cases.
- Use circuit breakers — Configure circuit breaker thresholds to prevent cascading failures.
-
DNS resolvers — Keep system DNS (
standard) first. Put a DoH resolver first only when system DNS is blocked or misconfigured. - Monitor error rates — Track timeout, retry, and circuit breaker events to identify configuration issues.
Security and Privacy
-
Trust config files carefully — Only set
TIF1_TRUST_CWD_CONFIG=truein trusted directories. - Use DoH for privacy — DNS-over-HTTPS encrypts DNS queries, improving privacy on untrusted networks.
- Validate config files — Ensure config files are valid JSON and do not contain malicious values.
- Limit cache locations — Store cache in secure locations with appropriate permissions.
Development Workflow
- Separate dev and prod configs — Maintain different configurations for development and production environments.
-
Use CI mode in pipelines — Enable
ci_modeso payload fetches skip the persistent cache. - Enable validation in dev — Catch data quality issues early by enabling validation in development.
- Disable cache in dev — Ensure fresh data during development by disabling cache.
- Verbose logging in dev — Use DEBUG logging to understand library behavior during development.
Memory Management
- Monitor memory usage — Track Python process memory to identify configuration issues.
- Reduce cache sizes — Lower cache sizes if memory usage is too high.
- Disable prefetching — Reduce memory usage by disabling aggressive prefetching.
-
Frequent cache commits — Lower
cache_commit_intervalto commit SQLite writes more often.
Troubleshooting
-
Enable debug logging — Use
tif1.setup_logging(logging.DEBUG)to see detailed diagnostic information. -
Check configuration values — Verify actual configuration values with
config.get(). - Monitor connection stats — Enable connection stats logging to track pool usage.
- Test incrementally — Change one setting at a time to isolate issues.
- Compare with defaults — Reset to defaults to verify custom configuration is the issue.
Configuration Maintenance
- Review periodically — Revisit configuration as workload characteristics change.
- Update with library — Check release notes for new configuration options.
- Benchmark regularly — Measure performance to validate configuration choices.
- Document changes — Keep a changelog of configuration changes and their rationale.
- Backup working configs — Save backups before making experimental changes.
Summary
Thetif1 configuration system controls library behavior through multiple configuration sources with clear precedence rules. Key takeaways:
Configuration Sources (Precedence Order)
- Programmatic API (
config.set()) — Highest precedence, runtime changes - Environment variables (
TIF1_*) — Deployment-specific overrides - Configuration file (
.tif1rc) — Persistent user settings - Default values — Hardcoded defaults optimized for general use
Key Features
- Singleton pattern — Single configuration instance per Python process
- Lazy initialization — Configuration loaded on first access
- Validation on read — Invalid values rejected when accessed, not when set
- Type conversion — Automatic conversion from environment variables
- Path expansion — Automatic
~expansion for paths - Thread-safe reads — Safe to read from multiple threads
Configuration Categories
- Core settings — DataFrame library, caching, validation
- Network settings — Timeouts, retries, concurrency
- HTTP session — Connection pooling, keep-alive, multiplexing
- Cache configuration — Memory limits, commit intervals, SQLite settings
- CDN configuration — CDN URLs, minification
- Retry & circuit breaker — Backoff, jitter, thresholds
- Prefetch strategies — Automatic data prefetching
- Telemetry settings — Telemetry-specific concurrency
- Logging — Connection stats interval
- Advanced settings — Polars options, JSON parsing, offline mode
Common Use Cases
- Maximum performance — High concurrency, large pools, aggressive prefetching
- Minimum latency — Ultra cold start, DoH, moderate concurrency
- Development — Disabled cache, enabled validation, verbose logging
- Production — Balanced settings, reliable retries, monitoring
- CI/CD — CI mode, disabled cache, fast failure
- Resource-constrained — Low concurrency, small caches, minimal prefetching
- Offline — Cache-only mode, no network requests
- High-reliability — Aggressive retries, long timeouts, conservative circuit breaker
Best Practices Summary
- Start with defaults, change only when needed
- Use config files for persistent settings
- Use environment variables for deployment
- Use programmatic API for runtime changes
- Monitor performance and adjust accordingly
- Document configuration choices
- Test changes before production deployment
- Keep validation disabled in production
- Enable caching for performance
- Balance concurrency with resources
Getting Help
If configuration issues occur:- Enable debug logging:
tif1.setup_logging(logging.DEBUG) - Check actual values:
config.get(key) - Verify precedence: Check file, env vars, and programmatic sets
- Review validation: Check logs for validation warnings
- Compare with defaults: Reset to defaults to isolate issues
- Consult documentation: Review this guide for detailed explanations
tif1 for each use case.
Configuration Reference Tables
Quick Reference: Common Settings
Quick Reference: Performance Impact
Legend: ↑ = Increases, ↓ = Decreases
Related Pages
Caching Strategy
Learn about the multi-layer caching system and its configuration options.
Best Practices
Performance optimization techniques and benchmarking.
HTTP Session
HTTP session configuration and connection pooling in detail.
Installation
Setup guide and initial configuration.
CLI
Command-line tools for events, sessions, drivers and the cache.
Backends
Pandas vs Polars backend comparison and configuration.
Additional Resources
Example Configuration Files
Minimal Configuration:cache_dir shows a placeholder: the real default depends on the OS (see cache_dir above).
Configuration Validation Script
Use this script to validate the configuration:Configuration Migration Script
Use this script to migrate from old configuration format:Environment Variable Generator
Use this script to generate environment variables from config file:Frequently Asked Questions
Q: Do I need to configure tif1? A: No, the default configuration works well for most use cases. Only configure for specific performance, reliability, or resource requirements. Q: What is the difference between config file and environment variables? A: Config files are persistent and user-specific. Environment variables are deployment-specific and override config files. Use config files for personal settings, environment variables for deployment settings. Q: Can I use multiple config files? A: Yes, use theTIF1_CONFIG_FILE environment variable to specify which config file to load. Only one config file is loaded at a time (the first valid file in the search order).
Q: How do I reset to default configuration?
A: Delete or rename the .tif1rc file and unset all TIF1_* environment variables. The library then uses hardcoded defaults.
Q: Why does my configuration not take effect?
A: Check configuration precedence (programmatic > env > file > defaults). Verify environment variable names are correct (TIF1_ prefix, uppercase). Ensure config file is valid JSON. Check logs for validation warnings.
Q: What is the performance impact of validation?
A: Validation parses every payload through Pydantic models, which costs CPU time on every load. Keep it disabled in production unless the data quality checks are needed.
Q: Should I use pandas or polars?
A: Pandas is the default and has the widest compatibility with the plotting and analysis ecosystem. Switch lib to "polars" and benchmark the workload when more performance is needed.
Q: How much memory does tif1 use?
A: It depends on the configuration and workload, mostly on how many sessions and telemetry laps are loaded. Reduce the in-memory cache sizes and concurrency, and disable prefetching, to lower memory usage.
Q: What is ultra cold start mode?
A: Ultra cold start is the fast path for sessions that are not yet in the persistent cache. Payloads skip the cache read, Pydantic validation and (with ultra_cold_skip_retries) the retries. They are still written to the cache. Already cached sessions load from the cache as usual.
Q: How do I optimize for my use case?
A: See the “Configuration Patterns and Recipes” section for detailed configurations for different use cases (performance, latency, development, production, etc.).
Q: Can I change configuration at runtime?
A: Yes, use config.set() to change values at runtime. Most keys apply to the next operation; the set() reference lists the keys that apply only when an object is created. Changes are not persisted unless config.save() is called.
Q: What is the recommended production configuration?
A: See the “Production Service Configuration” in the “Configuration Patterns and Recipes” section for a complete production configuration.
Q: How do I debug configuration issues?
A: Enable debug logging (tif1.setup_logging(logging.DEBUG)), check actual values (config.get(key)), verify precedence, and review logs for validation warnings.
Q: What is the difference between pool_connections and pool_maxsize?
A: pool_connections is the number of connection pools (one per host). pool_maxsize is the maximum connections per pool. Both auto-calculate by default based on concurrency settings.
Q: Should I enable prefetching?
A: Enable for common access patterns (for example, accessing driver laps after loading session). Disable for memory-constrained environments or when access patterns are unpredictable.
Q: What is the recommended cache size?
A: Default (1024 items) works for most cases. Increase for large-scale analysis (4096+). Decrease for memory-constrained environments (256-512).
Q: How do I configure for CI/CD?
A: Enable ci_mode, disable cache, enable validation, use lower concurrency, shorter timeout, and fewer retries. See “CI/CD Pipeline Configuration” for details.
Q: Can I use tif1 offline?
A: Yes, enable offline_mode to use only cached data. A payload that is not cached raises NetworkError without a request. See “Offline/Cache-Only Configuration” for details.
Q: What is the impact of HTTP/2 multiplexing?
A: HTTP/2 multiplexing reduces connection overhead by reusing connections for multiple requests. Enabled by default. Disable it only if specific proxies or CDNs cause issues.
Q: How do I configure DNS resolvers?
A: Use http_resolvers to specify the resolver order. The HTTP session uses the first resolver with which it can be created, by default system DNS. Put a DoH resolver first for privacy or when system DNS is blocked.
Q: What is the circuit breaker for?
A: Circuit breaker prevents cascading failures by stopping requests after consecutive failures. Automatically recovers after timeout. Configure threshold and timeout based on reliability requirements.
Q: How do I monitor configuration effectiveness?
A: Enable connection stats logging, track cache hit rates, monitor error rates, and benchmark performance. Adjust configuration based on metrics.