Configuration¶
Quick Start¶
Recommended: Use .env file
The easiest way to get started is to copy .env.example to .env and fill in your tokens. The SDK will load it automatically:
Configuration File¶
The SDK reads configuration from a JSON file. Start with the included template:
Or generate a sample programmatically:
Configuration Keys¶
| Key | Required | Default | Description |
|---|---|---|---|
API_URL |
No | https://extapi.authentic8.com |
Base URL for the Authentic8 API |
ADMIN_TOKEN |
For browsing/org | — | Token for BrowsingAPI and OrgManagementAPI |
SYNC_TOKEN |
For user mgmt | — | Token for UserManagementAPI |
FILE_TOKEN |
For file storage | — | Token for FileAPI |
LOG_TOKEN |
For log extraction | — | Token for LogExtractionAPI |
SCRAPE_TOKEN |
For harvesting | — | Token for HarvesterAPI |
BUCKET_ID |
For file storage | — | Default file storage bucket ID |
CATCHALL_USER |
For browsing | — | Default Silo username for browsing isolation contexts |
TOP_ORG |
For the MCP server | — | Top-level organization name used as the org argument in API calls. a8_validate_config treats it as required, a8_health_check probes ADMIN_TOKEN against it, and a8_get_api_info reports it. |
HARVEST_EGRESS_ORG |
No | (empty) | Org whose egress the harvest lookups check (list_harvest_egresses, the live harvest egress check, and the MCP harvest tools). Harvest resolves egress for the org the harvest token belongs to, and a lookup sees only egress provisioned to that org and the orgs above it, so set this when the harvest token belongs to a sub-org of TOP_ORG. When empty, TOP_ORG is used. |
ORG_VANITY_URL |
No | — | Organization vanity URL |
REQUEST_TIMEOUT |
No | 30 |
HTTP request timeout (seconds) for every API except log extraction |
LOG_EXTRACT_TIMEOUT |
No | (unset) | HTTP request timeout (seconds) for LogExtractionAPI only. Ships blank in config/default.json; when unset, LogExtractionAPI falls back to 600 seconds — see Log extraction timeout |
MAX_RETRIES |
No | 3 |
Retry count on transient errors |
RETRY_DELAY |
No | 5 |
Stored on the client as retry_delay; not currently read by the SDK — synchronous retries use urllib3 exponential backoff and the async paths compute their own |
LOG_LEVEL |
No | INFO |
SDK log level (DEBUG, INFO, WARNING, ERROR) |
ENABLE_ENCRYPTION |
No | false |
Accepted and stored, but not currently read by the SDK — setting it does not change any behavior. Log decryption is driven by the functions on the Log Decryption page, not by this flag |
DEFAULT_DEST_PATH |
No | / |
Default storage destination path for harvest results |
DEBUG_MODE |
No | false |
Accepted and stored, but not currently read by the SDK — setting it does not change any behavior. Use LOG_LEVEL to get debug logging |
DEFAULT_MAX_USES |
No | 1 |
Default max_uses for research session contexts created via a8_create_research_session MCP tool |
ENABLE_STATUS_CACHE |
No | false |
Cache harvest task status responses in HarvesterAPI, so repeated status polls for the same task ID reuse the last response instead of calling the API |
STATUS_CACHE_TTL |
No | 30 |
Lifetime in seconds of a cached harvest task status entry. Only consulted when ENABLE_STATUS_CACHE is true. 0 disables caching; validate_harvester_config() rejects values above 300. |
RATE_LIMIT |
No | {"requests_per_second": 10, "burst": 20} |
Declared rate-limit budget. validate_harvester_config() checks the shape (both values must be positive), but the SDK does not throttle requests on its own — enforce the limit in your own code or at a proxy. |
DEFAULT_EGRESS_INFO |
No | {"name": "New York, NY"} |
Default egress location, used by the bundled harvest examples for their non-asset tasks. Validated against the known egress locations. Asset and wget harvest tasks draw from the six asset datacenter locations (Singapore, Dubai, Frankfurt, Sao Paulo, Johannesburg, New York, NY): pass one of those or World, since any other name only logs a warning. The demos send the asset default, {"name": "New York, NY"}, to their asset tasks instead of this key. The SDK's own defaults do not read this key: with no egress given, TaskBuilder, create_simple_task and bulk_create_and_monitor_async use "New York, NY" for asset and wget tasks and "World" (any location) for every other task type. The context scripts do not read it either: --research without --egress uses "World". |
DEFAULT_POLICY |
No | See config/default.json |
Default browsing policy — a list of {"type": ..., "params": [...]} entries. Read by the bundled examples and reported by a8_validate_config as an optional key; API methods take the policy as an argument rather than reading this key. |
ENABLE_REQUEST_LOGGING |
No | false |
Log one INFO line per outbound request with the wire commands and their parameters. Credentials are redacted — see Request logging. |
LIVE_EGRESS_VALIDATION |
No | false |
Check a requested egress against the platform's live directory. create_context checks the username's directory and raises EgressNotOfferedError (a ValidationError) if it is not offered. Harvest task creation checks the pool for the task type, for HARVEST_EGRESS_ORG (or TOP_ORG if unset), and only warns, because an org's Launcher Options hide regions from the directory but not from harvest. Costs one extra request per context, and one per pool per minute for harvest. Never blocks on an empty directory or a failed lookup. Override per call with validate_egress=. |
TLS verification is not configurable
Certificates are always verified, against the certifi trust store, on both the
synchronous and the httpx async code paths. There is no setting that turns this off.
Earlier releases shipped a VERIFY_SSL key in config/default.json that no SDK code
read; it has been removed rather than wired up, so nothing suggests a switch that
does not exist. If you are terminating TLS at a proxy with a private CA, add that CA
to the system or certifi trust store rather than disabling verification.
ORG_VANITY_URL is not the API org name
TOP_ORG holds the API org name (e.g. MyCompany) that commands such as org.get
and listusers expect. ORG_VANITY_URL is the SSO vanity slug — a different value the
API does not accept as an org name.
Log extraction timeout¶
LogExtractionAPI uses its own request timeout, LOG_EXTRACT_TIMEOUT. The key ships
blank in config/default.json, and the client falls back to 600 seconds when it is
unset. The server serializes extracts behind a per-extract lock with a 600-second
TTL, so a client that gives up sooner abandons a request that is still running while the
lock stays held — every retry then fails until the TTL expires. Pinning the client
timeout to the lock TTL means anything that does time out has also released its lock.
Two rules govern how it combines with REQUEST_TIMEOUT:
- The default raises the extraction timeout but never lowers one already set higher.
If
REQUEST_TIMEOUTis above 600 — as it might be if you raised it to work around the previous 30-second extraction limit — extraction keeps that larger value. - An explicit
LOG_EXTRACT_TIMEOUTwins outright, including a value belowREQUEST_TIMEOUT, because setting it is a deliberate choice rather than an inherited default.
REQUEST_TIMEOUT continues to govern every other API. A non-numeric or non-positive
LOG_EXTRACT_TIMEOUT raises ConfigurationError when the configuration is loaded or the
client is constructed, rather than failing on the first request.
Request logging¶
Set ENABLE_REQUEST_LOGGING to true to have the client log one INFO line per
outbound request, naming the HTTP method, the URL and the command array being sent:
INFO - API request: POST https://extapi.authentic8.com/api payload=[{"command": "setauth", "data": "***REDACTED***"}, {"command": "listusers", "org": "MyCompany"}]
This is a separate switch from LOG_LEVEL on purpose. It answers "which commands did
the SDK send, with which parameters" — the question you have when reconciling SDK
behavior against the API — without turning on every DEBUG line in the SDK.
Credentials are removed before anything is written. Fields whose names contain token,
auth, password, secret, credential or key are masked, as is the token the
setauth command carries, and the token value is then stripped from the rendered line
by literal match as a second pass.
The key accepts a real boolean or a string. "1", "true", "yes" and "on" — in any
case, with surrounding whitespace ignored — turn it on; anything else, including "false"
and "0", leaves it off. The string handling matters because a value supplied through
${VAR} substitution arrives as text, and a plain truth test on the string "false" would
switch the flag on for someone who had just switched it off.
Because it is a diagnostic you turn on for one run and off again, it is settable from the
environment as A8_ENABLE_REQUEST_LOGGING — no config file edit, and nothing left switched
on in a file you might commit:
The same variable works for the MCP server on every transport. Its streamable-HTTP mode
reads no config file — it builds its configuration from credentials.yaml — so the
variable is consulted there as the fallback for a key that file does not mention.
Two limits are worth knowing:
- Requests only. Responses are not logged. Response bodies carry user records, file contents and log extracts, so logging them is a data-classification decision rather than a debugging one and does not ride along on this flag.
- Non-credential parameters are logged verbatim — usernames, org names, file paths and search patterns all appear in the line. Treat the resulting log at the same sensitivity as the data your calls touch.
Environment Variables¶
All configuration values can be set via environment variables. The A8_ prefix is the convention for all SDK variables:
| Environment Variable | Config Key | Description |
|---|---|---|
A8_API_URL |
API_URL |
API base URL |
A8_ADMIN_TOKEN |
ADMIN_TOKEN |
Browsing / org management token |
A8_SYNC_TOKEN |
SYNC_TOKEN |
User management token |
A8_FILE_TOKEN |
FILE_TOKEN |
File storage token |
A8_LOG_TOKEN |
LOG_TOKEN |
Log extraction token |
A8_SCRAPE_TOKEN |
SCRAPE_TOKEN |
Harvesting token |
A8_BUCKET_ID |
BUCKET_ID |
Default file storage bucket |
A8_CATCHALL_USER |
CATCHALL_USER |
Default Silo username for browsing contexts |
A8_ORG_VANITY_URL |
ORG_VANITY_URL |
Organization vanity URL |
A8_LOG_LEVEL |
LOG_LEVEL |
SDK log level |
A8_REQUEST_TIMEOUT |
REQUEST_TIMEOUT |
HTTP request timeout in seconds (fallback: 30) |
A8_LOG_EXTRACT_TIMEOUT |
LOG_EXTRACT_TIMEOUT |
Log extraction request timeout in seconds (fallback: 600) |
A8_MAX_RETRIES |
MAX_RETRIES |
Retry count on transient errors (fallback: 3) |
A8_RETRY_DELAY |
RETRY_DELAY |
Stored on the client; not currently read by the SDK (fallback: 5) |
A8_TOP_ORG |
TOP_ORG |
Top-level org name, also read by the bundled scripts (export_logs, find_user_orgs, etc.) |
A8_HARVEST_EGRESS_ORG |
HARVEST_EGRESS_ORG |
Org harvest egress lookups check (fallback: TOP_ORG) |
A8_TARGET_ORG |
(script env var) | Destination org for move_users.py |
A8_DEFAULT_MAX_USES |
DEFAULT_MAX_USES |
Default max_uses for a8_create_research_session MCP tool (fallback: 1) |
A8_ENABLE_REQUEST_LOGGING |
ENABLE_REQUEST_LOGGING |
Log one line per outbound request, credentials redacted (fallback: false) — see Request logging |
A8_LIVE_EGRESS_VALIDATION |
LIVE_EGRESS_VALIDATION |
Check egress against the live directory in create_context, and in harvest task creation (warn-only) (fallback: false) |
A8_ENABLE_STATUS_CACHE |
ENABLE_STATUS_CACHE |
Cache harvest task status responses (fallback: false) |
A8_STATUS_CACHE_TTL |
STATUS_CACHE_TTL |
Lifetime in seconds of a cached status entry (fallback: 30) |
Environment Variable Substitution¶
Config values support bash-style env var substitution. load_config() substitutes after
it merges config/{A8_CONFIG_ENV}.json over config/default.json, so placeholders in
either file are resolved, and a default.json placeholder whose key the profile file
overrides is never read:
{
"API_URL": "${A8_API_URL:-https://extapi.authentic8.com}",
"ADMIN_TOKEN": "${A8_ADMIN_TOKEN}",
"SYNC_TOKEN": "${A8_SYNC_TOKEN}",
"FILE_TOKEN": "${A8_FILE_TOKEN}",
"LOG_TOKEN": "${A8_LOG_TOKEN}",
"SCRAPE_TOKEN": "${A8_SCRAPE_TOKEN}",
"CATCHALL_USER": "${A8_CATCHALL_USER}"
}
${VAR_NAME}— the variable's value. An unset variable resolves to an empty string; loading does not fail.${VAR_NAME:-default}— the variable's value, ordefaultif the variable is unset or set to an empty string.${VAR_NAME:-}means "empty unless set".${VAR_NAME:default}— identical to${VAR_NAME:-default}; the dash is optional.
A placeholder can sit anywhere inside a string value, and one value can hold several
("${VAR_A}-${VAR_B}"). Values nested in objects and arrays are substituted too;
keys are not. A default runs to the first }, so it can contain colons, as a URL does,
but defaults do not nest. Only the braced form is recognized: $VAR_NAME stays as
literal text. Bash's colon-less ${VAR_NAME-default} is not supported either. It is
read as a variable literally named VAR_NAME-default, which a shell cannot export, so
it resolves to an empty string.
A missing token is not caught when the configuration loads
load_config() succeeds with every A8_*_TOKEN variable unset, and each token key
holds an empty string. The error comes later, when you construct a client that
needs the token, and names the variable to set: BrowsingAPI(config) raises
ConfigurationError: Missing required configuration: ADMIN_TOKEN (set A8_ADMIN_TOKEN).
A token for an API you never construct goes unnoticed. HarvesterAPI also checks
BUCKET_ID this way, but keys that no client checks at construction, such as
TOP_ORG and CATCHALL_USER, stay empty until a call uses them. The MCP server's
a8_validate_config tool lists every required key that is empty.
Substitution always produces a string, so the loader casts the keys whose type
matters before any consumer sees them. Numeric keys are parsed and range-checked, and a
value that is not a number raises ConfigurationError. A numeric key whose placeholder
resolves to an empty string is removed instead, so the built-in default or the reading
code's own fallback applies; an empty string in any other non-boolean key, such as
API_URL, is kept as is. Boolean keys are parsed against
the spellings that mean on — 1, true, yes, on, case-insensitively and with
surrounding whitespace ignored — and anything else, including false and 0, means off.
A switch never raises: there is no unparseable value, only one that is not recognized as
on. This matters because a plain truth test on the string "false" is True, which
would read a switch as the opposite of what was set.
Environment Files (.env)¶
The SDK uses python-dotenv to automatically load .env files:
# .env (never commit this file)
A8_ADMIN_TOKEN=your-admin-token
A8_SYNC_TOKEN=your-sync-token
A8_FILE_TOKEN=your-file-token
A8_LOG_TOKEN=your-log-token
A8_SCRAPE_TOKEN=your-scrape-token
A8_BUCKET_ID=your-bucket-id
A8_CATCHALL_USER=your-catchall-username
A8_TOP_ORG=your-org-name
See .env.example in the repository root for a complete template:
Never commit .env files
.env is listed in .gitignore. Keep tokens and usernames out of source control.
The committed config/default.json uses only ${A8_*} env var references — no real values.
Loading Configuration¶
Where load_config() finds its base configuration¶
Called with no argument, load_config() first loads .env and .env.{A8_CONFIG_ENV}
from the current working directory, then uses the first of these that exists as its base
configuration, also relative to the current working directory:
config/default.json, with theconfig/{A8_CONFIG_ENV}.jsonoverlay applied on topconfig.json, thensample_config.json— a single file, with no overlay- A copy of
config/default.jsonbundled inside the installed package. Theconfig/{A8_CONFIG_ENV}.jsonoverlay still applies on top of it if that file exists.
The bundled copy holds only ${A8_*} references, the same as the file in the
repository, so an install from PyPI needs no config file at all: set the A8_*
environment variables and call load_config(). API_URL defaults to
https://extapi.authentic8.com when A8_API_URL is unset.
A config file in the working directory wins
The bundled copy is used only when none of the files above exist. A stray
config/default.json, config.json or sample_config.json in the directory
your program runs from replaces it, and a .env there overrides variables
already set in the environment.
Passing a path bypasses the profile
Only load_config() with no argument applies the A8_CONFIG_ENV profile.
load_config("my_config.json") loads exactly that one file: config/{A8_CONFIG_ENV}.json
is never read, so tokens set there are silently ignored. .env and
.env.{A8_CONFIG_ENV} are still loaded either way. The profile is also skipped when
load_config() falls back to config.json or sample_config.json, which it loads alone.
Multi-Environment Setup¶
config/
default.json # base values with env var refs — version-controlled
production.json # production overrides — version-controlled
development.json # dev overrides — gitignored
qa.json # QA overrides — gitignored
Select the environment with the A8_CONFIG_ENV variable. The name becomes the
overlay filename, so it must contain only letters, digits, hyphens, and underscores:
export A8_CONFIG_ENV=development # loads default.json then development.json
export A8_CONFIG_ENV=qa # loads default.json then qa.json
export A8_CONFIG_ENV=production # loads default.json then production.json
No overlay is applied when A8_CONFIG_ENV is unset
There is no implicit default environment. With the variable unset,
load_config() reads config/default.json alone. An overlay file that does not
exist is skipped silently, so a typo in the environment name is not an error —
it just means no overrides are applied.
The same variable also selects a second environment file for secrets: .env is
loaded first, then .env.{A8_CONFIG_ENV} on top of it. You can set A8_CONFIG_ENV
in .env itself; a value exported in the shell takes precedence, so a default in
.env never overrides a profile you select for one command. An A8_CONFIG_ENV set
inside .env.{A8_CONFIG_ENV} is ignored, because that file was chosen by the profile
name, so the .env and config/ files always come from the same profile.
.env overrides other shell variables
For every other variable, a value in .env or .env.{A8_CONFIG_ENV} replaces one
already exported in the shell. If you export A8_ADMIN_TOKEN and .env
also sets it, the .env value is used. Remove the key from .env when you want the
shell value.
Keep default.json clean
config/default.json is version-controlled and should never contain real values.
Use ${A8_*} env var references for all secrets and site-specific settings.
Put actual values in your .env file or environment-specific config files (all gitignored).