Architecture Overview

For Data Analysts 8 min read

Olytix Core is built on a modern, cloud-native architecture designed for scalability, performance, and extensibility. This page provides a technical overview of the key components and how they interact.

What you'll learn

Core architectural components and their roles
How semantic queries are translated to SQL
The query optimization pipeline
Warehouse adapter interface

High-Level Architecture

🏗️

Olytix Core Architecture

Click any component to explore its details

🖥️

BI Tools

📊

Power BI

🐍

Python SDK

⚛️

React Apps

🔧

REST Clients

▼

API Gateway

FastAPI + Authentication + Rate Limiting

Entry Point

🔐

REST API

HTTP/JSON

📡

GraphQL

Flexible queries

📈

DAX / XMLA

Power BI Protocol

🔑

Auth

JWT + API Keys

▼

Core Services

Business Logic & Processing

Processing

⚙️

Compiler

YAML → SQL

🗺️

Query Planner

Semantic → SQL

🔗

Lineage Service

Column tracking

📋

Metadata

Catalog & Registry

🛡️

Security Engine

RLS & Masking

⏰

Pre-aggregation

Performance cache

▼

Resilience Layer

Fault Tolerance & Performance

Protection

🔄

Circuit Breaker

Fail-fast on errors

CLOSED→OPEN→HALF

🧱

Bulkhead

Tenant isolation

🔁

Retry Policy

Exponential backoff

1s2s4s8s

⏱️

Rate Limiting

Request throttling

▼

DataFusion Query Engine

Apache Arrow-based SQL Processing

Execution

📝

SQL Parser

Parse & tokenize

→

🌳

Logical Plan

AST building

→

⚡

Optimizer

Cost-based

→

🚀

Physical Plan

Execution ready

🏹

Apache Arrow — Zero-copy columnar in-memory format

▼

Data Warehouse Adapters

Pluggable Connectivity Layer

Data Sources

🐘

PostgreSQL

✓ GA

❄️

Snowflake

✓ GA

🔷

BigQuery

✓ GA

🧱

Databricks

✓ GA

🔶

Redshift

✓ GA

🦆

DuckDB

✓ GA

Technology Stack

⚡FastAPI

🐍Python 3.11+

🏹Apache Arrow

🔥DataFusion

🍓Strawberry GraphQL

🔴Redis

📊Prometheus

🔭OpenTelemetry

Performance Targets

< 200ms

API Response (p95)

< 5s

Query Execution (p95)

< 60s

Full Compilation

> 80%

Cache Hit Rate

Core Components

API Layer

The API layer provides multiple interfaces for consuming the semantic layer:

Interface	Use Case	Protocol
REST API	General integrations, BI tools	HTTP/JSON
GraphQL API	Flexible queries, frontend apps	GraphQL
DAX API	Power BI native integration	XMLA/DAX

Semantic Layer

The semantic layer is the heart of Olytix Core:

Cubes: Define analytical entities with measures and dimensions
Measures: Aggregation expressions (SUM, COUNT, AVG, etc.)
Dimensions: Categorical and temporal attributes
Metrics: Business KPIs composed from measures
Joins: Relationships between cubes

Query Engine

Built on Apache DataFusion and Apache Arrow:

Query Planner: Translates semantic queries to optimized SQL
Optimizer: Applies automatic optimizations
Executor: Manages query execution and result streaming

Deep Dive: Query Execution

Semantic Query to SQL

The Query Planner translates semantic queries into optimized SQL:

Semantic Query (Input)
{
  "metrics": ["total_revenue"],
  "dimensions": ["order_date.month", "customer.region"],
  "filters": [{"dimension": "order_date.year", "operator": "equals", "value": 2024}]
}

Optimized SQL (Output)
SELECT
  DATE_TRUNC('month', o.order_date) AS "order_date.month",
  c.region AS "customer.region",
  SUM(o.total_amount) AS "total_revenue"
FROM fct_orders o
JOIN dim_customers c ON o.customer_id = c.customer_id
WHERE EXTRACT(YEAR FROM o.order_date) = 2024
GROUP BY 1, 2
ORDER BY 1, 2

Query Optimization

The optimizer applies several techniques automatically:

Technique	Description	Benefit
Predicate Pushdown	Filters pushed to warehouse level	Reduced data transfer
Join Elimination	Removes unnecessary joins	Faster execution
Pre-aggregation Matching	Uses cached aggregates when available	Sub-second responses
Subquery Flattening	Simplifies nested queries	Cleaner execution plans

Performance Tip

Enable pre-aggregations for frequently queried measure/dimension combinations to achieve sub-second response times on large datasets.

Adapter Interface

Olytix Core uses a pluggable adapter architecture for warehouse connectivity:

src/olytix_core/adapters/base.py
class WarehouseAdapter(ABC):
    """Abstract interface for warehouse implementations."""

    async def execute(self, sql: str) -> pa.RecordBatch
    async def execute_iter(self, sql: str) -> AsyncIterator[pa.RecordBatch]
    async def get_schema(self, table: str) -> dict[str, str]
    def get_dialect(self) -> SQLDialect

Supported Warehouses

Warehouse	Adapter	Status
PostgreSQL	`postgresql`	Production
Snowflake	`snowflake`	Production
BigQuery	`bigquery`	Production
DuckDB	`duckdb`	Beta

This abstraction allows Olytix Core to support multiple data warehouses while maintaining a consistent query execution model based on Apache Arrow.

Data Flow

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Client    │────▶│   API       │────▶│   Query     │
│   Request   │     │   Layer     │     │   Planner   │
└─────────────┘     └─────────────┘     └──────┬──────┘
                                               │
                                               ▼
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Response  │◀────│   Result    │◀────│   Warehouse │
│   (Arrow)   │     │   Processor │     │   Adapter   │
└─────────────┘     └─────────────┘     └─────────────┘

High-Level Architecture​

Olytix Core Architecture

Core Components​

API Layer​

Semantic Layer​

Query Engine​

Deep Dive: Query Execution​

Semantic Query to SQL​

Query Optimization​

Adapter Interface​

Supported Warehouses​

Data Flow​

Related Pages​

High-Level Architecture

Core Components

API Layer

Semantic Layer

Query Engine

Deep Dive: Query Execution

Semantic Query to SQL

Query Optimization

Adapter Interface

Supported Warehouses

Data Flow

Related Pages