Skip to main content

Audit & Compliance

For Business Users

Olytix Core provides comprehensive audit capabilities to help organizations meet regulatory requirements, pass audits, and maintain data governance. This guide covers how to leverage Olytix Core's audit features.

Audit Capabilities Overview​

┌─────────────────────────────────────────────────────────────────────┐
│                    Olytix Core Audit Architecture                          │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│   ┌─────────────┐    ┌─────────────┐    ┌─────────────────────┐    │
│   │   Access    │    │   Change    │    │    Data Quality     │    │
│   │   Logs      │    │   History   │    │    Evidence         │    │
│   └──────┬──────┘    └──────┬──────┘    └──────────┬──────────┘    │
│          │                  │                       │               │
│          └──────────────────┼───────────────────────┘               │
│                             │                                        │
│                             ▼                                        │
│                    ┌─────────────────┐                              │
│                    │  Audit Store    │                              │
│                    │                 │                              │
│                    │  • Immutable    │                              │
│                    │  • Queryable    │                              │
│                    │  • Exportable   │                              │
│                    └────────┬────────┘                              │
│                             │                                        │
│            ┌────────────────┼────────────────┐                      │
│            │                │                │                      │
│            ▼                ▼                ▼                      │
│    ┌─────────────┐  ┌─────────────┐  ┌─────────────┐               │
│    │   Reports   │  │    Alerts   │  │   Export    │               │
│    └─────────────┘  └─────────────┘  └─────────────┘               │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

Configuring Audit​

Enable Audit Logging​

# olytix-core_project.yml
audit:
  enabled: true

  # What to log
  log:
    query_access: true        # All queries
    data_export: true         # Data exports
    definition_changes: true  # Metric/cube changes
    permission_changes: true  # Access control changes
    admin_actions: true       # Admin operations

  # Retention
  retention:
    days: 2555  # 7 years for SOX
    archive_after_days: 365

  # Storage
  storage:
    type: postgresql
    connection: audit_database
    table_prefix: audit_

  # Real-time alerting
  alerts:
    enabled: true
    channels:
      - security-team@company.com

Audit Log Structure​

Every audit event contains:

{
  "event_id": "uuid-123-456",
  "timestamp": "2024-01-20T10:30:45.123Z",
  "event_type": "QUERY_EXECUTED",

  "actor": {
    "user_id": "analyst@company.com",
    "ip_address": "10.0.1.50",
    "user_agent": "Olytix Core-Python-Client/1.0",
    "session_id": "session-789"
  },

  "resource": {
    "type": "metric",
    "name": "monthly_revenue",
    "version": "v3.0"
  },

  "action": {
    "type": "read",
    "query": {
      "metrics": ["monthly_revenue"],
      "dimensions": ["region"],
      "filters": [],
      "row_count": 4
    }
  },

  "context": {
    "application": "executive-dashboard",
    "request_id": "req-456",
    "duration_ms": 45
  }
}

Access Audit​

Query Access Logs​

Track who queries what data:

Query Access Report - January 2024
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Summary
────────────────────────────────────────────────────────────
Total Queries:        125,430
Unique Users:             234
Peak Hour:           10:00 AM (8,540 queries)
Avg Response Time:       45ms

Top Accessed Metrics
────────────────────────────────────────────────────────────
Metric                  │ Queries │ Users │ Avg Rows
────────────────────────┼─────────┼───────┼──────────
monthly_revenue         │  45,230 │   156 │     12
customer_count          │  32,100 │   189 │    248
order_count             │  28,450 │   145 │     35
avg_order_value         │  12,340 │    98 │     12

Access by User Role
────────────────────────────────────────────────────────────
Role           │ Queries │ Users │ Data Volume
───────────────┼─────────┼───────┼─────────────
analyst        │  78,000 │   125 │    2.3 GB
executive      │  12,000 │    15 │    0.8 GB
api_service    │  35,000 │    10 │    5.2 GB
developer      │     430 │    84 │    0.1 GB

Unusual Activity Detected
────────────────────────────────────────────────────────────
⚠ bulk_export_user: 50,000 rows exported in single query
⚠ analyst@company: Query volume 300% above normal
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Query Audit API​

# Get query audit logs
curl "http://localhost:8000/api/v1/audit/queries?start=2024-01-01&end=2024-01-31"

# Filter by user
curl "http://localhost:8000/api/v1/audit/queries?user=analyst@company.com"

# Filter by metric
curl "http://localhost:8000/api/v1/audit/queries?metric=monthly_revenue"

Change Audit​

Definition Change History​

Track all changes to metrics and cubes:

Change Audit - monthly_revenue
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Version History
────────────────────────────────────────────────────────────

v3.0 (Current) — 2024-01-15 14:32:01
  Changed by: john.smith@company.com
  Approved by: cfo@company.com
  Ticket: FIN-1234

  Changes:
  + Added filter: status != 'refunded'
  ~ Updated description

  Diff:
  - expression: orders.gross_amount
  + expression: orders.net_amount

  Impact Analysis:
  • Affected dashboards: 12
  • Affected reports: 8
  • Users notified: 156

────────────────────────────────────────────────────────────

v2.0 — 2023-07-01 09:15:22
  Changed by: data-team@company.com
  Approved by: finance-director@company.com
  Ticket: DATA-567

  Changes:
  + Added digital revenue
  ~ Changed time_grain to day

────────────────────────────────────────────────────────────

v1.0 — 2023-01-01 00:00:00
  Created by: finance-team@company.com
  Approved by: cfo@company.com
  Ticket: FIN-001

  Initial definition
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Permission Change History​

Track access control changes:

Permission Change Audit
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

2024-01-20 14:30:00 — ROLE_CREATED
  Actor: admin@company.com
  Role: new_analyst_role
  Permissions: [read_orders, read_customers]
  Approved by: security-team@company.com

2024-01-19 10:15:00 — USER_ROLE_ASSIGNED
  Actor: manager@company.com
  User: new.hire@company.com
  Role: analyst
  Expiry: 2024-07-19

2024-01-18 16:45:00 — PERMISSION_REVOKED
  Actor: security@company.com
  User: departed.employee@company.com
  Reason: Employee departure
  Ticket: HR-789

2024-01-17 09:00:00 — RLS_POLICY_UPDATED
  Actor: data-team@company.com
  Policy: region_restriction
  Change: Added APAC region access
  Affected users: 45
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Compliance Reports​

SOX Compliance Report​

SOX Compliance Report — Q4 2024
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Control Summary
────────────────────────────────────────────────────────────
Control                          │ Status │ Evidence
─────────────────────────────────┼────────┼────────────
Data accuracy validation         │   ✓    │ Test results
Change management process        │   ✓    │ Approval logs
Access control enforcement       │   ✓    │ RBAC config
Audit trail completeness         │   ✓    │ Audit logs
Segregation of duties            │   ✓    │ Role matrix

Financial Metrics Audit
────────────────────────────────────────────────────────────
Metric              │ Certified │ Owner    │ Tests │ Changes
────────────────────┼───────────┼──────────┼───────┼─────────
monthly_revenue     │    ✓      │ Finance  │ 12/12 │    2
quarterly_revenue   │    ✓      │ Finance  │ 8/8   │    1
net_income          │    ✓      │ Finance  │ 10/10 │    0
operating_expenses  │    ✓      │ Finance  │ 6/6   │    1

Change Control Evidence
────────────────────────────────────────────────────────────
Total changes this quarter: 4
All changes approved: ✓
All changes documented: ✓
All changes tested: ✓

Access Review Evidence
────────────────────────────────────────────────────────────
Users with financial data access: 45
Access reviews completed: 45/45 (100%)
Terminated user access revoked: 3/3 (100%)

Report generated: 2024-01-20T10:00:00Z
Prepared by: Olytix Core Audit System
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

GDPR Data Inventory​

GDPR Data Inventory Report
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Personal Data Categories
────────────────────────────────────────────────────────────
Category          │ Columns │ Tables │ Lawful Basis
──────────────────┼─────────┼────────┼───────────────
Email addresses   │      12 │      5 │ Contract
Names             │       8 │      4 │ Contract
Phone numbers     │       4 │      2 │ Consent
IP addresses      │       6 │      3 │ Legitimate int.
Location data     │       3 │      2 │ Consent

Data Subject Access
────────────────────────────────────────────────────────────
Total access requests (YTD): 23
Average response time: 2.3 days
Completed within 30 days: 23/23 (100%)

Data Retention
────────────────────────────────────────────────────────────
Category          │ Retention │ Records │ Due for Deletion
──────────────────┼───────────┼─────────┼──────────────────
Customer PII      │ 2 years   │ 50,000  │ 2,340 (next 30d)
Transaction logs  │ 7 years   │ 1.2M    │ 0
Marketing consent │ 3 years   │ 25,000  │ 890 (next 30d)

Data Processing Activities
────────────────────────────────────────────────────────────
Activity          │ Legal Basis │ Recipients │ Transfers
──────────────────┼─────────────┼────────────┼───────────
Order processing  │ Contract    │ Internal   │ None
Marketing emails  │ Consent     │ Mailchimp  │ US (SCCs)
Analytics         │ Legit. int. │ Internal   │ None
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Audit Alerts​

Configure Audit Alerts​

audit:
  alerts:
    # Unusual access patterns
    - name: bulk_data_access
      condition: "rows_returned > 100000"
      severity: warning
      channels: [security-team@company.com]

    # After-hours access
    - name: after_hours_access
      condition: "hour < 6 OR hour > 22"
      severity: info
      channels: [audit-log@company.com]

    # Sensitive data access
    - name: pii_access
      condition: "accessed_pii = true"
      severity: info
      channels: [privacy-team@company.com]

    # Failed access attempts
    - name: access_denied
      condition: "result = 'denied'"
      severity: warning
      channels: [security-team@company.com]

    # Admin actions
    - name: admin_action
      condition: "action_type IN ('delete', 'modify_permissions', 'drop')"
      severity: high
      channels: [security-team@company.com, audit-committee@company.com]

Audit Export​

Export for External Audit​

# Export audit logs for external auditor
olytix-core audit export \
  --start 2024-01-01 \
  --end 2024-03-31 \
  --format csv \
  --include-changes \
  --include-access \
  --include-permissions \
  --output audit_q1_2024.zip

# Export specific control evidence
olytix-core audit export-control \
  --control SOX-404 \
  --period Q1-2024 \
  --format pdf \
  --output sox_evidence_q1.pdf

Auditor Self-Service Portal​

# Configure auditor access
roles:
  - name: external_auditor
    permissions:
      - view_audit_logs
      - view_change_history
      - view_access_reports
      - export_audit_data
      - view_lineage

    restrictions:
      - no_data_access
      - read_only

    validity:
      start: 2024-01-15
      end: 2024-04-15
      reason: "Annual SOX audit"

Best Practices​

1. Comprehensive Logging​

Log everything relevant:

audit:
  log:
    # Always log these
    required:
      - queries
      - changes
      - access_grants
      - access_revocations

    # Log these for compliance
    compliance:
      - pii_access
      - financial_data_access
      - bulk_exports

2. Immutable Storage​

Ensure audit logs cannot be modified:

audit:
  storage:
    immutable: true
    write_once: true
    checksums: sha256
    backup: true

3. Regular Reviews​

Schedule audit reviews:

audit:
  reviews:
    - type: access_review
      frequency: quarterly
      reviewers: [security-team, data-owners]

    - type: change_review
      frequency: monthly
      reviewers: [data-governance]

    - type: compliance_review
      frequency: annually
      reviewers: [compliance-team, external-auditor]

4. Retention Compliance​

Match retention to requirements:

audit:
  retention:
    default: 7 years  # SOX requirement
    pii_access: 3 years  # GDPR
    financial: 7 years  # SOX
    security: 5 years  # SOC 2

Next Steps​

Ready to implement audit compliance?

  1. Configure authentication →
  2. Set up RBAC →
  3. Enable detailed audit logging →
Audit Preparation

Don't wait for an audit to prepare. Run compliance reports regularly and address issues proactively.