Unified Project
A Olytix Core project unifies data transformation and semantic layer capabilities in a single, coherent structure. This page explains how the components work together.
Project Philosophy
Traditional data stacks separate transformation (dbt) from semantic layer (Cube, Looker). This creates:
- Disconnected metadata
- Manual synchronization
- Incomplete lineage
- Duplicate configurations
Olytix Core's unified approach means:
- Single source of truth for definitions
- Automatic synchronization
- End-to-end lineage
- One configuration language
Project Components
A Olytix Core project contains these interconnected components:
Unified Project Structure
Explore the hierarchy of an Olytix project with sources, models, cubes, and metrics.
Sources
Sources define external data connections:
# sources/raw.yml
sources:
- name: raw
database: production
schema: public
tables:
- name: orders
columns:
- name: id
- name: customer_id
- name: amount
Purpose:
- Document raw data schema
- Enable
source()references - Track freshness
- Provide lineage starting point
Models
Models transform data using SQL:
-- models/fct_orders.sql
{{ config(materialized='table') }}
SELECT
id AS order_id,
customer_id,
amount AS order_amount,
created_at AS order_date
FROM {{ source('raw', 'orders') }}
WHERE amount > 0
Purpose:
- Clean raw data
- Join multiple sources
- Calculate derived fields
- Create analytical tables
Cubes
Cubes define semantic abstractions:
# cubes/orders.yml
cubes:
- name: orders
sql: "SELECT * FROM {{ ref('fct_orders') }}"
measures:
- name: total_revenue
type: sum
sql: order_amount
- name: order_count
type: count
dimensions:
- name: order_date
type: time
sql: order_date
- name: customer_id
type: number
sql: customer_id
Purpose:
- Define measures (aggregations)
- Define dimensions (slicers)
- Create joins between entities
- Enable semantic querying
Metrics
Metrics define business KPIs:
# metrics/revenue.yml
metrics:
- name: monthly_revenue
type: simple
expression: orders.total_revenue
time_grain: month
description: "Total revenue by month"
- name: revenue_growth
type: derived
expression: |
(current.monthly_revenue - prior.monthly_revenue)
/ prior.monthly_revenue
description: "Month-over-month growth"
Purpose:
- Define business-level KPIs
- Add time intelligence
- Create derived calculations
- Standardize definitions
How Components Connect
The components form a dependency chain:
Source → Model → Cube → Metric
Connection: Source → Model
Models reference sources using source():
-- In model SQL
SELECT * FROM {{ source('raw', 'orders') }}
This creates:
- Compile-time dependency
- Lineage tracking
- Freshness awareness
Connection: Model → Model
Models can reference other models using ref():
-- In model SQL
SELECT * FROM {{ ref('stg_orders') }}
This creates:
- Execution order dependency
- Model-level lineage
- Cascading refreshes
Connection: Model → Cube
Cubes reference models in their SQL:
cubes:
- name: orders
sql: "SELECT * FROM {{ ref('fct_orders') }}"
This creates:
- Schema inference
- Column lineage to measures/dimensions
- Refresh dependencies
Connection: Cube → Cube
Cubes can join to other cubes:
cubes:
- name: orders
joins:
- cube: customers
sql: "{orders}.customer_id = {customers}.customer_id"
relationship: many_to_one
Connection: Cube → Metric
Metrics reference cube measures:
metrics:
- name: monthly_revenue
expression: orders.total_revenue # cube.measure
The UnifiedProject Object
When you compile a project, Olytix Core creates a UnifiedProject object:
# Conceptual structure
class UnifiedProject:
name: str
sources: Dict[str, Source]
models: Dict[str, Model]
cubes: Dict[str, Cube]
metrics: Dict[str, Metric]
graph: DependencyGraph
lineage: LineageGraph
This object:
- Validates all references
- Builds dependency graph
- Computes lineage
- Enables queries
Project Configuration
The olytix-core_project.yml file configures the entire project:
name: my_analytics
version: 1.0.0
# Database connection
warehouse:
type: postgresql
host: ${OLYTIX_DB_HOST}
database: ${OLYTIX_DB_NAME}
user: ${OLYTIX_DB_USER}
password: ${OLYTIX_DB_PASSWORD}
# Model defaults
models:
my_analytics:
+materialized: view
staging:
+schema: staging
marts:
+materialized: table
+schema: analytics
# Cube defaults
cubes:
+meta:
owner: analytics-team
# Variables
vars:
start_date: '2020-01-01'
Project Lifecycle
1. Development
Create and modify YAML/SQL files:
# Create new model
touch models/marts/fct_orders.sql
# Create new cube
touch cubes/orders.yml
2. Compilation
Validate and build project:
olytix-core compile
This:
- Parses all files
- Validates references
- Builds dependency graph
- Generates manifest
3. Execution
Run models and start server:
# Build models
olytix-core run
# Start API
olytix-core serve
4. Querying
Query via API:
curl -X POST http://localhost:8000/api/v1/query \
-d '{"metrics": ["monthly_revenue"], "dimensions": ["orders.order_date.month"]}'
Best Practices
1. Layered Architecture
Organize models in layers:
models/
├── staging/ # 1:1 with sources, cleaning only
├── intermediate/ # Complex transforms, joins
└── marts/ # Final fact and dimension tables
2. Clear Naming
Use consistent naming:
stg_<source>_<table> → stg_raw_orders
int_<description> → int_order_items
fct_<entity> → fct_orders
dim_<entity> → dim_customers
3. Single Responsibility
Each component should have one job:
- Source: Define schema
- Model: Transform data
- Cube: Define semantics
- Metric: Define KPIs
4. Documentation
Document everything:
- name: monthly_revenue
description: |
Monthly revenue calculation.
Excludes refunds and cancelled orders.
meta:
owner: finance-team
Next Steps
Now that you understand the unified project: