Architecture

System architecture and core components of TrailBase

Architecture

TrailBase is a single-executable backend platform that combines multiple technologies into a cohesive system. This document explains the core architecture and how components interact.

Overview

TrailBase’s architecture is built on four main pillars:

┌─────────────────────────────────────────────────────────┐
│                    TrailBase Server                     │
├─────────────────────────────────────────────────────────┤
│  Admin UI (Embedded)    │    REST APIs (Axum Router)   │
├─────────────────────────────────────────────────────────┤
│  AppState (Shared State & Configuration)               │
├───────────────┬─────────────────┬──────────────────────┤
│  Auth System  │  Record APIs    │  WASM Runtime        │
├───────────────┴─────────────────┴──────────────────────┤
│  Connection Manager (SQLite Pool)                       │
├─────────────────────────────────────────────────────────┤
│  SQLite Database + Extensions                           │
└─────────────────────────────────────────────────────────┘

Core Components

Server Layer

Server Initialization

The Server struct manages the HTTP server, routers, and TLS configuration.

TrailBase uses Axum (a web framework built on Tokio and Hyper) for HTTP handling. The server initialization process:

1. Configuration Loading: Reads config from traildepot/config.textproto
2. Database Initialization: Sets up SQLite with extensions and migrations
3. Router Setup: Configures routes for APIs, admin UI, and custom endpoints
4. Runtime Initialization: Prepares WASM runtime for custom handlers

// Defined in server/mod.rs:97-106
pub struct Server {
  pub state: AppState,
  pub main_router: (String, Router),
  pub admin_router: Option<(String, Router)>,
  pub tls: Option<(CertificateDer<'static>, PrivateKeyDer<'static>)>,
}

Application State

The AppState is the central nervous system of TrailBase, containing all shared state and configuration.

Defined in app_state.rs:82-85, the AppState struct holds:

• Data Directory: File system paths for data, uploads, and configuration
• Database Connections: Connection manager with pooling and caching
• Authentication: JWT helper, auth options, and OAuth configuration
• Record APIs: Configured table/view APIs with access control
• Job Registry: Scheduled tasks and background jobs
• WASM Runtimes: WebAssembly execution environments
• Object Store: File upload storage (local or S3)
• Mailer: Email sending configuration

// Key AppState methods from app_state.rs:199-311
pub fn data_dir(&self) -> &DataDir
pub fn connection_manager(&self) -> ConnectionManager
pub fn lookup_record_api(&self, name: &str) -> Option<RecordApi>
pub fn auth_options(&self) -> Arc<AuthOptions>
pub fn jwt(&self) -> &JwtHelper

SQLite Core

Connection Manager

Manages SQLite connections with multi-database support and metadata caching.

TrailBase uses a custom SQLite connection layer (trailbase_sqlite) with:

• Connection Pooling: Manages multiple connections for different database combinations
• Read/Write Separation: Optimizes concurrency with dedicated read threads
• Metadata Caching: Schema information cached per connection
• Attached Databases: Support for multiple SQLite databases in a single connection

// From connection.rs:84-82
pub struct ConnectionManager {
  state: Arc<ConnectionManagerState>,
}

// Key capabilities:
- main_entry() -> ConnectionEntry  // Primary database connection
- get_entry(...) -> ConnectionEntry  // Get/create cached connections
- rebuild_metadata() -> Result<(), ConnectionError>  // Refresh schema cache

Connection Features:

• Extensions: JSON functions, full-text search, WASM functions
• Migrations: Automatic schema migrations using Refinery
• Triggers: File deletion triggers for blob cleanup
• Pragmas: Optimized settings (WAL mode, synchronous, mmap_size)

WASM Runtime

WASM support requires the wasm feature flag to be enabled at compile time.

TrailBase integrates Wasmtime for running WebAssembly components. The WASM runtime provides:

// From wasm/mod.rs:19
pub(crate) use trailbase_wasm_runtime_host::{
  HttpStore,    // HTTP request handling
  KvStore,      // Key-value storage
  Runtime,      // WASM execution environment
  SharedState,  // Shared state between instances
};

WASM Capabilities:

1. HTTP Handlers: Custom API endpoints written in WASM
2. Job Handlers: Scheduled background tasks
3. SQLite Functions: Custom SQL functions implemented in WASM
4. File System Access: Sandboxed FS access for WASM components

// WASM runtime initialization from wasm/mod.rs:71-116
pub(crate) fn wasm_runtimes_builder(
  data_dir: DataDir,
  conn: Connection,
  rt: Option<tokio::runtime::Handle>,
  runtime_root_fs: Option<PathBuf>,
  shared_kv_store: Option<KvStore>,
  dev: bool,
) -> Result<WasmRuntimeBuilder, AnyError>

Admin UI

The admin interface is embedded in the binary as static assets using trailbase_assets::AdminAssets. It provides:

• Schema management (tables, indexes, views)
• User administration
• Record API configuration
• System logs and monitoring
• OAuth provider setup

Routed at /_/admin by default, with API endpoints at /_/admin/api.

Data Flow

Request Lifecycle

1. HTTP Request → Axum Router
2. Middleware: Logging, CORS, Cookies, Rate Limiting
3. Authentication: JWT validation (if required)
4. Record API: Access control checks (ACL + Rules)
5. SQLite: Query execution
6. Response: JSON serialization
7. HTTP Response → Client

Subscription Flow (Real-time)

TrailBase supports real-time subscriptions using Server-Sent Events (SSE):

1. Client: GET /api/records/v1/{api_name}/subscribe/{record_id}
2. Server: Establish SSE connection
3. SQLite: Create trigger for record changes
4. On Change: Notify subscription manager
5. Server: Send event to client over SSE

Implemented in records/subscribe.rs with the SubscriptionManager managing active subscriptions.

Configuration System

TrailBase uses Protocol Buffers (textproto format) for configuration, defined in config.rs. The configuration includes:

Server Config

• Bind address and ports
• Site URL
• CORS settings
• TLS certificates

Auth Config

• JWT settings
• OAuth providers
• Password policies
• Session duration

Record APIs

• Table/view mappings
• Access control lists
• Access rules
• Expansion config

Storage Config

• Database paths
• S3 settings
• Upload limits
• Object storage

Configuration is validated on load and can be hot-reloaded via SIGHUP signal.

Extension System

TrailBase extends SQLite through trailbase_extension:

• JSON Schema Functions: Validation against JSON schemas
• UUID Functions: UUIDv7 generation
• GeoIP: MaxMind database queries for IP geolocation
• Custom Functions: User-defined functions via WASM

Performance Considerations

TrailBase uses several strategies to maintain high performance:

1. Connection Pooling: Reuses SQLite connections across requests
2. Metadata Caching: Schema information cached in memory
3. Read Thread Pool: Multiple reader threads for concurrent queries
4. Query Preparation: Cached prepared statements
5. Reactive Updates: Only rebuilds affected state on config changes
6. WASM Compilation: Component-level caching for WASM modules

Security Architecture

• JWT-based Authentication: EdDSA signed tokens
• CSRF Protection: Random tokens validated on state-changing operations
• Row-Level Security: SQL-based access rules per operation
• Rate Limiting: IP-based throttling on auth endpoints
• Sandboxed WASM: Isolated execution with controlled host functions

Next Steps

Data Model

Learn about database schemas, tables, and relationships

Authentication

Understand the auth system and user management

APIs

Explore REST APIs and type-safe record APIs