Skip to content

How to Export Logs

This guide covers extracting Silo audit logs using the SDK and the scripts/export_logs.py standalone script.

The Silo Logs Reference Guide on the Authentic8 support portal describes each log type and its fields.

Two approaches

  • SDK directly — Best for programmatic integration and custom processing
  • export_logs.py script — Best for one-time exports, scheduled exports, and resume support

Using the SDK Directly

1. Check what's available

from silo_sdk import LogExtractionAPI, load_config

config = load_config()
logs = LogExtractionAPI(config)

seq_info = logs.get_log_sequence_info("my_org")
print(f"Start from seq: {seq_info['min_seq']}")
print(f"More logs available: {seq_info['is_more']}")
# Note: max_seq is always None — the API does not expose total log count
print(f"Valid types: {LogExtractionAPI.VALID_LOG_TYPES}")

VALID_LOG_TYPES is compiled into the SDK, so it reflects the platform as of the SDK release. To ask the platform you are actually talking to, call get_log_types():

info = logs.get_log_types()
print(info["log_types"])  # e.g. ['A8SS', 'ADMIN_AUDIT', 'APP_LAUNCH', ...]
print(info["version"])    # date-stamped version of the list, e.g. 20260817

Names come back in SDK form, with underscores, so they can go straight into extract_logs(). The four types that use spaces on the wire — BLOCKED URL, CASE MANAGER, LOCATION CHANGE, POST DATA — are translated for you.

Don't use version as a cache key yet

get_log_types() accepts a version argument as a change check, but two deployments exposing different type lists have been seen reporting the same version. Call it without version until that is fixed, or you may miss a newly added type.

2. Extract logs

# Extract AUTH and SESSION logs starting from min_seq
records = logs.extract_logs(
    org="my_org",
    start_seq=seq_info.get("min_seq", 0),
    log_types=["AUTH", "SESSION"],
    limit=500,
)
print(f"Got {len(records['logs'])} records")
if records["is_more"]:
    print(f"Next page starts at seq {records['next_seq']}")

extract_logs() returns a dictionary with logs, is_more, and next_seq. Pass next_seq as the next call's start_seq to page forward.

2a. Extract several types when one may fail

A combined extract_logs() request fails as a whole if any one type is unavailable — most often ENC on an organization without encryption — even though the other types would succeed. extract_logs_per_type() makes one request per type and merges the pages. A type that fails is reported in type_errors instead of failing the call; the call raises only if every type fails.

page = logs.extract_logs_per_type(
    org="my_org",
    start_seq=seq_info.get("min_seq", 0),
    log_types=["AUTH", "SESSION", "ENC"],
    limit=500,
)
print(f"Got {len(page['logs'])} records")
for log_type, error in page.get("type_errors", {}).items():
    print(f"{log_type} failed: {error}")

Two things to plan for when you page through the merged result:

  • Deduplicate by seq_id. Each type paginates independently, so the merged next_seq is the lowest resume point among the types that still report is_more. Resuming there skips no rows, but the next page can repeat rows of types that were further ahead. When no type reports more, next_seq is the lowest next_seq any type reported, so a poller still advances.
  • Failed types are not covered by next_seq. A type listed in type_errors returned nothing, and is_more says nothing about it. Re-request failed types from the original start_seq; resuming them from the merged next_seq would skip their rows.

If you make the single-type requests yourself (for example, to add your own retry), pass the per-type results to merge_per_type_extracts() from silo_sdk.logging to get the same merged page.

3. Export to file

logs.export_logs_to_file(
    org="my_org",
    output_path="logs.json",
    log_types=["AUTH", "SESSION", "URL"],
    output_format="json",
)
logs.export_logs_to_file(
    org="my_org",
    output_path="logs.csv",
    log_types=["AUTH"],
    output_format="csv",
)

CSV output uses per-type field schemas from log_schemas.py to produce consistent column ordering.

4. Extract all logs (with automatic pagination)

from datetime import datetime

all_records = logs.extract_all_logs(
    org="my_org",
    start_seq=0,
    log_types=["AUTH", "URL"],
    start_date=datetime(2026, 1, 1),
    end_date=datetime(2026, 1, 31),
)
print(f"Total: {len(all_records)} records")

start_date and end_date take a datetime or an integer number of days ago; date strings are not accepted.

Using the export_logs.py Script

The standalone script adds resume support, ENC log handling, and a proxy option.

Basic usage

source .venv/bin/activate
export A8_LOG_TOKEN="your-log-token"
export A8_TOP_ORG="my_org"

# Check sequence info and valid log types
python scripts/export_logs.py --info

# Extract all logs to JSON
python scripts/export_logs.py --output logs.json --format json

# Extract specific types to CSV
python scripts/export_logs.py --log-types AUTH SESSION --output auth_session.csv --format csv

Resume support

The script saves the last extracted sequence to logextract_resume.json. Re-running picks up where it left off:

# First run: extracts from seq 0
python scripts/export_logs.py --output logs.json

# Second run: continues from last seq
python scripts/export_logs.py --output logs.json

Proxy support

python scripts/export_logs.py --proxy https://proxy.company.com:8080

CSV enhancements

When exporting to CSV, you can expand JSON-encoded fields and parse hierarchy strings into separate columns:

# Expand JSON headers into individual columns (headers.User-Agent, etc.)
python scripts/export_logs.py --log-types URL --format csv --expand-json

# Flatten nested dict fields (egress_info.protocol, task_params.request_type)
python scripts/export_logs.py --log-types HARVEST --format csv --expand-nested

# Parse LAUNCHER egress_region into world/region/country/city columns
python scripts/export_logs.py --log-types LAUNCHER --format csv --parse-egress

# Combine all enhancements
python scripts/export_logs.py --log-types ALL --format csv \
    --expand-json --expand-nested --parse-egress

Encrypted logs

The SDK handles ENC log extraction and decryption natively — no external package required:

python scripts/export_logs.py --encrypted --decrypt --pvtkey pvtkey.txt
python scripts/export_logs.py --encrypted --show-enc  # include raw ENC blocks

For a standalone example of the full extract-decrypt workflow, see examples/decrypt_logs.py:

# Extract and decrypt ENC logs, print per-type breakdown
python examples/decrypt_logs.py --org my_org --pvtkey pvtkey.txt --limit 100

# Write decrypted logs to JSON
python examples/decrypt_logs.py --org my_org --pvtkey pvtkey.txt --output decrypted.json

Copy examples/pvtkey.txt.example to pvtkey.txt and fill in your private key values. See the file for format details (Legacy passphrase and Standard EC PEM formats).

Schema Validation

Validate the SDK's log_schemas.py field definitions against real log data from a live org:

# Validate all 26 log types against ExampleOrg
python scripts/dev/validate_log_schemas.py --org ExampleOrg

# Validate specific types only
python scripts/dev/validate_log_schemas.py --org ExampleOrg --types AUTH,SESSION,URL

# Write report to file
python scripts/dev/validate_log_schemas.py --org ExampleOrg --output validation_report.txt

The script compares LOG_TYPE_FIELDS schemas against real extracted log fields and reports any extra fields in the real data that should be added to the schema. Run this after Authentic8 ships a new logtype_headers.ini to catch schema drift.

Log type coverage

Log availability depends on org activity; not all types may have data in every organization.

Multi-Type Extraction

For extracting multiple log types with automatic lock-contention retry, use scripts/extract_all_logs.py:

# Extract all log types for the last 7 days
python scripts/extract_all_logs.py --org my_org --types ALL --days 7

# Extract specific types with custom limits
python scripts/extract_all_logs.py --org my_org --types AUTH,SESSION,URL --days 30 --limit 500

# Custom retry and pause settings
python scripts/extract_all_logs.py --org my_org --types ALL \
    --pause 15 --retry-delay 90 --max-retries 5

The script extracts one type at a time with configurable pauses between types, and automatically retries on backend lock contention errors.

Log Analytics

After extracting logs, use scripts/log_stats.py to generate analytics:

# Domain frequency analysis
python scripts/log_stats.py --input logs/ --analysis domains --top 20

# Authentication pattern analysis
python scripts/log_stats.py --input logs/ --analysis auth

# Run all analyses
python scripts/log_stats.py --input logs/ --analysis all

# Output as JSON or CSV
python scripts/log_stats.py --input logs/ --analysis domains --format json
python scripts/log_stats.py --input logs/ --analysis egress --format csv

Available analyses: domains, auth, users, egress, transfers, harvest, apps, nexus, sessions, bypasses, user_agents.

Log Parsing Utilities

The silo_sdk.logging.log_utils module provides helpers for working with extracted log data:

from silo_sdk.logging import (
    parse_json_field,
    parse_all_json_fields,
    extract_user_agents,
    extract_content_types,
    parse_egress_hierarchy,
    normalize_log_types,
    expand_json_fields,
    flatten_nested_dicts,
)

# Parse JSON-encoded header fields
entry = parse_all_json_fields(url_log_entry)
print(entry["headers"]["User-Agent"])

# Get User-Agent frequency table
ua_counts = extract_user_agents(url_logs)
print(ua_counts.most_common(5))

# Parse LAUNCHER egress hierarchy
location = parse_egress_hierarchy(
    "World /  / North America /  / United States /  / New York, NY"
)
# {'world': 'World', 'region': 'North America',
#  'country': 'United States', 'city': 'New York, NY'}

Wire Format Note

Space-delimited log types

Four log types use spaces on the wire: BLOCKED URL, CASE MANAGER, LOCATION CHANGE, POST DATA. The SDK accepts both forms — pass BLOCKED_URL (with underscores) and the SDK converts to spaces automatically when sending to the API.

Log Types Reference

The SDK ships with 26 log types; a newer deployment may support more, which is what get_log_types() is for. Common ones:

Type Description
AUTH Login/logout events
SESSION Silo session lifecycle
URL Web requests from Silo sessions
EXPLOIT Security exploit detections
CLIPBOARD Clipboard copy/paste events
HARVEST Web harvesting task events
APP_LAUNCH Application launch events
EVENT General platform events
ENC Encrypted log entries (requires decryption)
NEXUS Nexus AI conversation events

For the full list, ask the platform with LogExtractionAPI.get_log_types(), or read the built-in LogExtractionAPI.VALID_LOG_TYPES if you need an answer offline.