Near Real-Time Crypto Price ETL Pipeline with Power BI DirectQuery

This project involves building an ETL pipeline to extract real-time cryptocurrency prices from the CoinGecko API and load the data into Google BigQuery. The dataset is then connected to Power BI using DirectQuery, enabling real-time visualization and analysis of crypto market trends without the need for data refreshes.

Note: This Power BI near real-time report will fetch the latest data because it uses the DirectQuery feature of Power BI. However, automatic page refresh only applies to Power BI Pro users and accounts within the same tenant.

Crypto Currency Price Report Architecture

Tools & Frameworks: Python, SQL, Power BI Direct Query, BigQuery, AWS EC2, Cron Job

ETL Script at GitHub: https://github.com/zawmyohtet215/etl_coingecko_to_bigquery