Migrating from dbt

For Data Analysts

This guide walks you through migrating an existing dbt project to Olytix Core. Olytix Core is designed to be dbt-compatible, so most projects can be migrated with minimal changes while gaining semantic layer capabilities.

Migration Overview

Aspect	Effort Level	Notes
Models	Low	Mostly compatible as-is
Sources	Low	Same YAML format
Tests	Low	Same test syntax
Macros	Medium	Some adjustments needed
Project config	Low	Similar structure

Before You Begin

Prerequisites

• Python 3.11+
• Olytix Core installed (pip install olytix-core)
• Existing dbt project
• Database credentials

Backup Your Project

Before making changes, create a backup:

cp -r my_dbt_project my_dbt_project_backup

Step 1: Initialize Olytix Core in Your Project

Navigate to your dbt project directory and initialize Olytix Core:

cd my_dbt_project

# Create Olytix Core configuration alongside dbt
olytix-core init --preserve-existing

This creates olytix-core_project.yml without modifying your existing files.

Step 2: Migrate Project Configuration

dbt_project.yml to olytix-core_project.yml

dbt_project.yml:

name: 'my_analytics'
version: '1.0.0'
config-version: 2

profile: 'my_profile'

model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"
clean-targets:
  - "target"
  - "dbt_packages"

models:
  my_analytics:
    staging:
      +materialized: view
    marts:
      +materialized: table

olytix-core_project.yml:

name: my_analytics
version: 1.0.0

# Direct warehouse connection (replaces profiles.yml)
warehouse:
  type: postgresql
  host: ${OLYTIX_DB_HOST}
  port: 5432
  database: ${OLYTIX_DB_NAME}
  user: ${OLYTIX_DB_USER}
  password: ${OLYTIX_DB_PASSWORD}

# Paths (Olytix Core uses same conventions)
paths:
  models: models
  sources: sources
  cubes: cubes        # New: semantic layer
  metrics: metrics    # New: business metrics
  tests: tests
  seeds: seeds
  macros: macros

# Default materializations
models:
  my_analytics:
    staging:
      materialized: view
    marts:
      materialized: table

Migrate Database Credentials

dbt uses profiles.yml while Olytix Core uses environment variables:

profiles.yml (dbt):

my_profile:
  target: dev
  outputs:
    dev:
      type: postgres
      host: localhost
      user: analyst
      password: secret123
      database: analytics
      schema: public

Olytix Core approach:

# Set environment variables
export OLYTIX_DB_HOST=localhost
export OLYTIX_DB_USER=analyst
export OLYTIX_DB_PASSWORD=secret123
export OLYTIX_DB_NAME=analytics

# Or use a .env file (add to .gitignore)
echo "OLYTIX_DB_HOST=localhost" >> .env
echo "OLYTIX_DB_USER=analyst" >> .env
echo "OLYTIX_DB_PASSWORD=secret123" >> .env
echo "OLYTIX_DB_NAME=analytics" >> .env

Step 3: Migrate Sources

Sources use the same format in both dbt and Olytix Core:

sources/raw.yml (works in both):

version: 2

sources:
  - name: raw
    database: analytics
    schema: public
    tables:
      - name: orders
        description: Raw e-commerce orders
        columns:
          - name: id
            description: Unique order identifier
          - name: customer_id
            description: Reference to customer
          - name: amount
            description: Order total in USD
          - name: created_at
            description: Order timestamp

No changes required for most source definitions.

Step 4: Migrate Models

Basic Models

Most models work without modification:

-- models/staging/stg_orders.sql
-- Works in both dbt and Olytix Core

{{ config(materialized='view') }}

SELECT
    id AS order_id,
    customer_id,
    amount,
    created_at AS order_date
FROM {{ source('raw', 'orders') }}

Incremental Models

Incremental models use the same syntax:

-- models/marts/fct_orders.sql
-- Works in both dbt and Olytix Core

{{ config(
    materialized='incremental',
    unique_key='order_id'
) }}

SELECT
    order_id,
    customer_id,
    amount,
    order_date
FROM {{ ref('stg_orders') }}
{% if is_incremental() %}
WHERE order_date > (SELECT MAX(order_date) FROM {{ this }})
{% endif %}

Model Schema Files

Schema files are compatible:

# models/staging/schema.yml
version: 2

models:
  - name: stg_orders
    description: Staged orders with cleaned column names
    columns:
      - name: order_id
        description: Unique order identifier
        tests:
          - not_null
          - unique
      - name: customer_id
        description: Reference to customer
        tests:
          - not_null

Step 5: Migrate Macros

Most macros work without changes. Here are common patterns:

Simple Macros

-- macros/cents_to_dollars.sql
-- Works in both dbt and Olytix Core

{% macro cents_to_dollars(column_name) %}
    ({{ column_name }} / 100.0)
{% endmacro %}

Macros with dbt-Specific Functions

Some dbt utilities need adjustment:

dbt version:

{% macro generate_schema_name(custom_schema_name, node) %}
    {%- set default_schema = target.schema -%}
    {%- if custom_schema_name is none -%}
        {{ default_schema }}
    {%- else -%}
        {{ custom_schema_name | trim }}
    {%- endif -%}
{% endmacro %}

Olytix Core version:

{% macro generate_schema_name(custom_schema_name, node) %}
    {%- set default_schema = var('schema', 'public') -%}
    {%- if custom_schema_name is none -%}
        {{ default_schema }}
    {%- else -%}
        {{ custom_schema_name | trim }}
    {%- endif -%}
{% endmacro %}

Common Macro Replacements

dbt Macro	Olytix Core Equivalent
target.schema	var('schema')
target.database	var('database')
target.name	var('target')
run_started_at	var('run_started_at')

Step 6: Migrate Tests

Tests are fully compatible:

# tests/schema.yml
version: 2

models:
  - name: fct_orders
    columns:
      - name: order_id
        tests:
          - not_null
          - unique
      - name: customer_id
        tests:
          - not_null
          - relationships:
              to: ref('dim_customers')
              field: customer_id

Singular tests also work:

-- tests/assert_positive_amounts.sql
SELECT *
FROM {{ ref('fct_orders') }}
WHERE amount < 0

Step 7: Add Semantic Layer (New)

This is where Olytix Core adds value. Create cubes to define your semantic layer:

Create Cubes

# cubes/orders.yml
cubes:
  - name: orders
    sql: "SELECT * FROM {{ ref('fct_orders') }}"
    description: "Order analytics"

    measures:
      - name: count
        type: count
        description: "Total number of orders"

      - name: total_revenue
        type: sum
        sql: amount
        format: currency
        description: "Sum of all order amounts"

      - name: average_order_value
        type: avg
        sql: amount
        format: currency
        description: "Average order amount"

    dimensions:
      - name: order_id
        type: number
        sql: order_id
        primary_key: true

      - name: order_date
        type: time
        sql: order_date
        granularities:
          - day
          - week
          - month
          - quarter
          - year

      - name: customer_id
        type: number
        sql: customer_id

Create Metrics

# metrics/revenue.yml
metrics:
  - name: monthly_revenue
    type: simple
    expression: orders.total_revenue
    time_grain: month
    description: "Total revenue aggregated by month"

  - name: revenue_per_customer
    type: derived
    expression: "{orders.total_revenue} / {orders.unique_customers}"
    description: "Average revenue per customer"

Add Joins

# cubes/customers.yml
cubes:
  - name: customers
    sql: "SELECT * FROM {{ ref('dim_customers') }}"

    measures:
      - name: count
        type: count

    dimensions:
      - name: customer_id
        type: number
        sql: customer_id
        primary_key: true

      - name: country
        type: string
        sql: country

# Update orders cube with join
# cubes/orders.yml
cubes:
  - name: orders
    sql: "SELECT * FROM {{ ref('fct_orders') }}"

    joins:
      - name: customers
        relationship: many_to_one
        sql: "{orders}.customer_id = {customers}.customer_id"

    # ... measures and dimensions

Step 8: Validate and Compile

Run Olytix Core compilation:

# Compile the project
olytix-core compile

# Expected output:
# Compiling project...
# Loaded 2 sources
# Compiled 5 models
# Registered 2 cubes
# Created 2 metrics
# Built column-level lineage
# Generated manifest at target/manifest.json

Fix any errors before proceeding.

Step 9: Run and Verify

Run Models

# Run all models
olytix-core run

# Run specific models
olytix-core run --select staging.*

Test the Semantic Layer

Start the API server:

olytix-core serve

Query your new semantic layer:

curl -X POST http://localhost:8000/api/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "metrics": ["total_revenue", "count"],
    "dimensions": ["order_date.month"]
  }'

Migration Checklist

Use this checklist to track your migration progress:

• Backup existing project
• Initialize Olytix Core configuration
• Migrate dbt_project.yml to olytix-core_project.yml
• Set up environment variables for credentials
• Verify sources compile
• Verify models compile
• Update any incompatible macros
• Verify tests pass
• Create cubes for key models
• Create business metrics
• Add joins between cubes
• Test semantic queries via API
• Document changes

Maintaining Both Systems (Optional)

If you need to run both dbt and Olytix Core during transition:

my_project/
├── dbt_project.yml       # Keep for dbt
├── olytix-core_project.yml      # Add for Olytix Core
├── profiles.yml          # Keep for dbt
├── .env                  # Add for Olytix Core
├── sources/
├── models/
├── cubes/                # New: Olytix Core only
├── metrics/              # New: Olytix Core only
└── tests/

Run each tool independently:

# dbt workflow
dbt run
dbt test

# Olytix Core workflow
olytix-core compile
olytix-core run
olytix-core serve

Common Migration Issues

Unsupported dbt Packages

Some dbt packages may not be compatible. Common solutions:

Package	Solution
dbt_utils	Most macros work; some need adjustment
dbt_date	Use Olytix Core time intelligence instead
dbt_expectations	Tests work as-is

Target Variable References

Replace target.* references:

-- dbt
{% if target.name == 'prod' %}

-- Olytix Core
{% if var('target') == 'prod' %}

Snapshots

Snapshots have similar syntax but check compatibility:

-- May need adjustment
{% snapshot orders_snapshot %}
    {{ config(
        strategy='timestamp',
        unique_key='order_id',
        updated_at='updated_at'
    ) }}

    SELECT * FROM {{ source('raw', 'orders') }}
{% endsnapshot %}

Next Steps

After migration:

1. Add pre-aggregations for query performance
2. Connect BI tools to your semantic layer
3. Set up row-level security if needed
4. Explore column-level lineage

---

Need help? Check the Troubleshooting guide or FAQ.