MDO Data Warehouse

This project is the central data warehouse for Minerals Development Oman (MDO). It integrates data from multiple sources including the ERP system, Excel files, and other business systems, providing a single source of truth for reporting and analytics.

Architecture Overview

The data warehouse pipeline consists of:

• Airbyte – for data ingestion from various sources (ERP, Excel files, APIs, etc.)
• dbt – for data transformation and modeling
• Hydra – for orchestration and task scheduling
• Grafana Loki + Promtail – for centralized logging and monitoring

Data Sources

• ERP System – Core business operations data
• Excel Files – Ad-hoc reports and manually maintained data
• Other Systems – Additional business systems as required

Contributing

To contribute to this project:

Clone the repository:

git clone git@github.com:rihal-om/mdo-dwh.git