Skip to content

Configuration

Quick Start

Recommended: Use .env file

The easiest way to get started is to copy .env.example to .env and fill in your tokens. The SDK will load it automatically:

cp .env.example .env
# Edit .env and add your tokens

Configuration File

The SDK reads configuration from a JSON file. Start with the included template:

cp config/default.json my_config.json

Or generate a sample programmatically:

from silo_sdk.utils.config import create_sample_config
create_sample_config("my_config.json")

Configuration Keys

Key Required Default Description
API_URL No https://extapi.authentic8.com Base URL for the Authentic8 API
ADMIN_TOKEN For browsing/org — Token for BrowsingAPI and OrgManagementAPI
SYNC_TOKEN For user mgmt — Token for UserManagementAPI
FILE_TOKEN For file storage — Token for FileAPI
LOG_TOKEN For log extraction — Token for LogExtractionAPI
SCRAPE_TOKEN For harvesting — Token for HarvesterAPI
BUCKET_ID For file storage — Default file storage bucket ID
CATCHALL_USER For browsing — Default Silo username for browsing isolation contexts
TOP_ORG For the MCP server — Top-level organization name used as the org argument in API calls. a8_validate_config treats it as required, a8_health_check probes ADMIN_TOKEN against it, and a8_get_api_info reports it.
HARVEST_EGRESS_ORG No (empty) Org whose egress the harvest lookups check (list_harvest_egresses, the live harvest egress check, and the MCP harvest tools). Harvest resolves egress for the org the harvest token belongs to, and a lookup sees only egress provisioned to that org and the orgs above it, so set this when the harvest token belongs to a sub-org of TOP_ORG. When empty, TOP_ORG is used.
ORG_VANITY_URL No — Organization vanity URL
REQUEST_TIMEOUT No 30 HTTP request timeout (seconds) for every API except log extraction
LOG_EXTRACT_TIMEOUT No (unset) HTTP request timeout (seconds) for LogExtractionAPI only. Ships blank in config/default.json; when unset, LogExtractionAPI falls back to 600 seconds — see Log extraction timeout
MAX_RETRIES No 3 Retry count on transient errors
RETRY_DELAY No 5 Stored on the client as retry_delay; not currently read by the SDK — synchronous retries use urllib3 exponential backoff and the async paths compute their own
LOG_LEVEL No INFO SDK log level (DEBUG, INFO, WARNING, ERROR)
ENABLE_ENCRYPTION No false Accepted and stored, but not currently read by the SDK — setting it does not change any behavior. Log decryption is driven by the functions on the Log Decryption page, not by this flag
DEFAULT_DEST_PATH No / Default storage destination path for harvest results
DEBUG_MODE No false Accepted and stored, but not currently read by the SDK — setting it does not change any behavior. Use LOG_LEVEL to get debug logging
DEFAULT_MAX_USES No 1 Default max_uses for research session contexts created via a8_create_research_session MCP tool
ENABLE_STATUS_CACHE No false Cache harvest task status responses in HarvesterAPI, so repeated status polls for the same task ID reuse the last response instead of calling the API
STATUS_CACHE_TTL No 30 Lifetime in seconds of a cached harvest task status entry. Only consulted when ENABLE_STATUS_CACHE is true. 0 disables caching; validate_harvester_config() rejects values above 300.
RATE_LIMIT No {"requests_per_second": 10, "burst": 20} Declared rate-limit budget. validate_harvester_config() checks the shape (both values must be positive), but the SDK does not throttle requests on its own — enforce the limit in your own code or at a proxy.
DEFAULT_EGRESS_INFO No {"name": "New York, NY"} Default egress location, used by the bundled harvest examples for their non-asset tasks. Validated against the known egress locations. Asset and wget harvest tasks draw from the six asset datacenter locations (Singapore, Dubai, Frankfurt, Sao Paulo, Johannesburg, New York, NY): pass one of those or World, since any other name only logs a warning. The demos send the asset default, {"name": "New York, NY"}, to their asset tasks instead of this key. The SDK's own defaults do not read this key: with no egress given, TaskBuilder, create_simple_task and bulk_create_and_monitor_async use "New York, NY" for asset and wget tasks and "World" (any location) for every other task type. The context scripts do not read it either: --research without --egress uses "World".
DEFAULT_POLICY No See config/default.json Default browsing policy — a list of {"type": ..., "params": [...]} entries. Read by the bundled examples and reported by a8_validate_config as an optional key; API methods take the policy as an argument rather than reading this key.
ENABLE_REQUEST_LOGGING No false Log one INFO line per outbound request with the wire commands and their parameters. Credentials are redacted — see Request logging.
LIVE_EGRESS_VALIDATION No false Check a requested egress against the platform's live directory. create_context checks the username's directory and raises EgressNotOfferedError (a ValidationError) if it is not offered. Harvest task creation checks the pool for the task type, for HARVEST_EGRESS_ORG (or TOP_ORG if unset), and only warns, because an org's Launcher Options hide regions from the directory but not from harvest. Costs one extra request per context, and one per pool per minute for harvest. Never blocks on an empty directory or a failed lookup. Override per call with validate_egress=.

TLS verification is not configurable

Certificates are always verified, against the certifi trust store, on both the synchronous and the httpx async code paths. There is no setting that turns this off. Earlier releases shipped a VERIFY_SSL key in config/default.json that no SDK code read; it has been removed rather than wired up, so nothing suggests a switch that does not exist. If you are terminating TLS at a proxy with a private CA, add that CA to the system or certifi trust store rather than disabling verification.

ORG_VANITY_URL is not the API org name

TOP_ORG holds the API org name (e.g. MyCompany) that commands such as org.get and listusers expect. ORG_VANITY_URL is the SSO vanity slug — a different value the API does not accept as an org name.

Log extraction timeout

LogExtractionAPI uses its own request timeout, LOG_EXTRACT_TIMEOUT. The key ships blank in config/default.json, and the client falls back to 600 seconds when it is unset. The server serializes extracts behind a per-extract lock with a 600-second TTL, so a client that gives up sooner abandons a request that is still running while the lock stays held — every retry then fails until the TTL expires. Pinning the client timeout to the lock TTL means anything that does time out has also released its lock.

Two rules govern how it combines with REQUEST_TIMEOUT:

  • The default raises the extraction timeout but never lowers one already set higher. If REQUEST_TIMEOUT is above 600 — as it might be if you raised it to work around the previous 30-second extraction limit — extraction keeps that larger value.
  • An explicit LOG_EXTRACT_TIMEOUT wins outright, including a value below REQUEST_TIMEOUT, because setting it is a deliberate choice rather than an inherited default.

REQUEST_TIMEOUT continues to govern every other API. A non-numeric or non-positive LOG_EXTRACT_TIMEOUT raises ConfigurationError when the configuration is loaded or the client is constructed, rather than failing on the first request.

Request logging

Set ENABLE_REQUEST_LOGGING to true to have the client log one INFO line per outbound request, naming the HTTP method, the URL and the command array being sent:

INFO - API request: POST https://extapi.authentic8.com/api payload=[{"command": "setauth", "data": "***REDACTED***"}, {"command": "listusers", "org": "MyCompany"}]

This is a separate switch from LOG_LEVEL on purpose. It answers "which commands did the SDK send, with which parameters" — the question you have when reconciling SDK behavior against the API — without turning on every DEBUG line in the SDK.

Credentials are removed before anything is written. Fields whose names contain token, auth, password, secret, credential or key are masked, as is the token the setauth command carries, and the token value is then stripped from the rendered line by literal match as a second pass.

The key accepts a real boolean or a string. "1", "true", "yes" and "on" — in any case, with surrounding whitespace ignored — turn it on; anything else, including "false" and "0", leaves it off. The string handling matters because a value supplied through ${VAR} substitution arrives as text, and a plain truth test on the string "false" would switch the flag on for someone who had just switched it off.

Because it is a diagnostic you turn on for one run and off again, it is settable from the environment as A8_ENABLE_REQUEST_LOGGING — no config file edit, and nothing left switched on in a file you might commit:

A8_ENABLE_REQUEST_LOGGING=true python your_script.py

The same variable works for the MCP server on every transport. Its streamable-HTTP mode reads no config file — it builds its configuration from credentials.yaml — so the variable is consulted there as the fallback for a key that file does not mention.

Two limits are worth knowing:

  • Requests only. Responses are not logged. Response bodies carry user records, file contents and log extracts, so logging them is a data-classification decision rather than a debugging one and does not ride along on this flag.
  • Non-credential parameters are logged verbatim — usernames, org names, file paths and search patterns all appear in the line. Treat the resulting log at the same sensitivity as the data your calls touch.

Environment Variables

All configuration values can be set via environment variables. The A8_ prefix is the convention for all SDK variables:

Environment Variable Config Key Description
A8_API_URL API_URL API base URL
A8_ADMIN_TOKEN ADMIN_TOKEN Browsing / org management token
A8_SYNC_TOKEN SYNC_TOKEN User management token
A8_FILE_TOKEN FILE_TOKEN File storage token
A8_LOG_TOKEN LOG_TOKEN Log extraction token
A8_SCRAPE_TOKEN SCRAPE_TOKEN Harvesting token
A8_BUCKET_ID BUCKET_ID Default file storage bucket
A8_CATCHALL_USER CATCHALL_USER Default Silo username for browsing contexts
A8_ORG_VANITY_URL ORG_VANITY_URL Organization vanity URL
A8_LOG_LEVEL LOG_LEVEL SDK log level
A8_REQUEST_TIMEOUT REQUEST_TIMEOUT HTTP request timeout in seconds (fallback: 30)
A8_LOG_EXTRACT_TIMEOUT LOG_EXTRACT_TIMEOUT Log extraction request timeout in seconds (fallback: 600)
A8_MAX_RETRIES MAX_RETRIES Retry count on transient errors (fallback: 3)
A8_RETRY_DELAY RETRY_DELAY Stored on the client; not currently read by the SDK (fallback: 5)
A8_TOP_ORG TOP_ORG Top-level org name, also read by the bundled scripts (export_logs, find_user_orgs, etc.)
A8_HARVEST_EGRESS_ORG HARVEST_EGRESS_ORG Org harvest egress lookups check (fallback: TOP_ORG)
A8_TARGET_ORG (script env var) Destination org for move_users.py
A8_DEFAULT_MAX_USES DEFAULT_MAX_USES Default max_uses for a8_create_research_session MCP tool (fallback: 1)
A8_ENABLE_REQUEST_LOGGING ENABLE_REQUEST_LOGGING Log one line per outbound request, credentials redacted (fallback: false) — see Request logging
A8_LIVE_EGRESS_VALIDATION LIVE_EGRESS_VALIDATION Check egress against the live directory in create_context, and in harvest task creation (warn-only) (fallback: false)
A8_ENABLE_STATUS_CACHE ENABLE_STATUS_CACHE Cache harvest task status responses (fallback: false)
A8_STATUS_CACHE_TTL STATUS_CACHE_TTL Lifetime in seconds of a cached status entry (fallback: 30)

Environment Variable Substitution

Config values support bash-style env var substitution. load_config() substitutes after it merges config/{A8_CONFIG_ENV}.json over config/default.json, so placeholders in either file are resolved, and a default.json placeholder whose key the profile file overrides is never read:

{
  "API_URL": "${A8_API_URL:-https://extapi.authentic8.com}",
  "ADMIN_TOKEN": "${A8_ADMIN_TOKEN}",
  "SYNC_TOKEN": "${A8_SYNC_TOKEN}",
  "FILE_TOKEN": "${A8_FILE_TOKEN}",
  "LOG_TOKEN": "${A8_LOG_TOKEN}",
  "SCRAPE_TOKEN": "${A8_SCRAPE_TOKEN}",
  "CATCHALL_USER": "${A8_CATCHALL_USER}"
}
  • ${VAR_NAME} — the variable's value. An unset variable resolves to an empty string; loading does not fail.
  • ${VAR_NAME:-default} — the variable's value, or default if the variable is unset or set to an empty string. ${VAR_NAME:-} means "empty unless set".
  • ${VAR_NAME:default} — identical to ${VAR_NAME:-default}; the dash is optional.

A placeholder can sit anywhere inside a string value, and one value can hold several ("${VAR_A}-${VAR_B}"). Values nested in objects and arrays are substituted too; keys are not. A default runs to the first }, so it can contain colons, as a URL does, but defaults do not nest. Only the braced form is recognized: $VAR_NAME stays as literal text. Bash's colon-less ${VAR_NAME-default} is not supported either. It is read as a variable literally named VAR_NAME-default, which a shell cannot export, so it resolves to an empty string.

A missing token is not caught when the configuration loads

load_config() succeeds with every A8_*_TOKEN variable unset, and each token key holds an empty string. The error comes later, when you construct a client that needs the token, and names the variable to set: BrowsingAPI(config) raises ConfigurationError: Missing required configuration: ADMIN_TOKEN (set A8_ADMIN_TOKEN). A token for an API you never construct goes unnoticed. HarvesterAPI also checks BUCKET_ID this way, but keys that no client checks at construction, such as TOP_ORG and CATCHALL_USER, stay empty until a call uses them. The MCP server's a8_validate_config tool lists every required key that is empty.

Substitution always produces a string, so the loader casts the keys whose type matters before any consumer sees them. Numeric keys are parsed and range-checked, and a value that is not a number raises ConfigurationError. A numeric key whose placeholder resolves to an empty string is removed instead, so the built-in default or the reading code's own fallback applies; an empty string in any other non-boolean key, such as API_URL, is kept as is. Boolean keys are parsed against the spellings that mean on — 1, true, yes, on, case-insensitively and with surrounding whitespace ignored — and anything else, including false and 0, means off. A switch never raises: there is no unparseable value, only one that is not recognized as on. This matters because a plain truth test on the string "false" is True, which would read a switch as the opposite of what was set.

Environment Files (.env)

The SDK uses python-dotenv to automatically load .env files:

# .env (never commit this file)
A8_ADMIN_TOKEN=your-admin-token
A8_SYNC_TOKEN=your-sync-token
A8_FILE_TOKEN=your-file-token
A8_LOG_TOKEN=your-log-token
A8_SCRAPE_TOKEN=your-scrape-token
A8_BUCKET_ID=your-bucket-id
A8_CATCHALL_USER=your-catchall-username
A8_TOP_ORG=your-org-name

See .env.example in the repository root for a complete template:

cp .env.example .env
# then fill in your values

Never commit .env files

.env is listed in .gitignore. Keep tokens and usernames out of source control. The committed config/default.json uses only ${A8_*} env var references — no real values.

Loading Configuration

from silo_sdk import load_config

# Reads config/default.json from the working directory, and merges
# config/{A8_CONFIG_ENV}.json over it when A8_CONFIG_ENV is set
config = load_config()

export A8_CONFIG_ENV=development
# Call load_config() with no argument to get the overlay:
# loads config/default.json, then overlays config/development.json
config = load_config()

from silo_sdk import create_config
import os

config = (create_config()
    .with_api_url("https://extapi.authentic8.com")
    .with_browsing_token(os.environ["A8_ADMIN_TOKEN"])
    .with_sync_token(os.environ["A8_SYNC_TOKEN"])
    .with_timeout(30)
    .build())

Where load_config() finds its base configuration

Called with no argument, load_config() first loads .env and .env.{A8_CONFIG_ENV} from the current working directory, then uses the first of these that exists as its base configuration, also relative to the current working directory:

  1. config/default.json, with the config/{A8_CONFIG_ENV}.json overlay applied on top
  2. config.json, then sample_config.json — a single file, with no overlay
  3. A copy of config/default.json bundled inside the installed package. The config/{A8_CONFIG_ENV}.json overlay still applies on top of it if that file exists.

The bundled copy holds only ${A8_*} references, the same as the file in the repository, so an install from PyPI needs no config file at all: set the A8_* environment variables and call load_config(). API_URL defaults to https://extapi.authentic8.com when A8_API_URL is unset.

A config file in the working directory wins

The bundled copy is used only when none of the files above exist. A stray config/default.json, config.json or sample_config.json in the directory your program runs from replaces it, and a .env there overrides variables already set in the environment.

Passing a path bypasses the profile

Only load_config() with no argument applies the A8_CONFIG_ENV profile. load_config("my_config.json") loads exactly that one file: config/{A8_CONFIG_ENV}.json is never read, so tokens set there are silently ignored. .env and .env.{A8_CONFIG_ENV} are still loaded either way. The profile is also skipped when load_config() falls back to config.json or sample_config.json, which it loads alone.

Multi-Environment Setup

config/
  default.json      # base values with env var refs — version-controlled
  production.json   # production overrides — version-controlled
  development.json  # dev overrides — gitignored
  qa.json           # QA overrides — gitignored

Select the environment with the A8_CONFIG_ENV variable. The name becomes the overlay filename, so it must contain only letters, digits, hyphens, and underscores:

export A8_CONFIG_ENV=development   # loads default.json then development.json
export A8_CONFIG_ENV=qa            # loads default.json then qa.json
export A8_CONFIG_ENV=production    # loads default.json then production.json

No overlay is applied when A8_CONFIG_ENV is unset

There is no implicit default environment. With the variable unset, load_config() reads config/default.json alone. An overlay file that does not exist is skipped silently, so a typo in the environment name is not an error — it just means no overrides are applied.

The same variable also selects a second environment file for secrets: .env is loaded first, then .env.{A8_CONFIG_ENV} on top of it. You can set A8_CONFIG_ENV in .env itself; a value exported in the shell takes precedence, so a default in .env never overrides a profile you select for one command. An A8_CONFIG_ENV set inside .env.{A8_CONFIG_ENV} is ignored, because that file was chosen by the profile name, so the .env and config/ files always come from the same profile.

.env overrides other shell variables

For every other variable, a value in .env or .env.{A8_CONFIG_ENV} replaces one already exported in the shell. If you export A8_ADMIN_TOKEN and .env also sets it, the .env value is used. Remove the key from .env when you want the shell value.

Keep default.json clean

config/default.json is version-controlled and should never contain real values. Use ${A8_*} env var references for all secrets and site-specific settings. Put actual values in your .env file or environment-specific config files (all gitignored).