Built a comprehensive Extract, Transform, Load pipeline with Python, APIs, SQL and Power BI visualization for minimizing technical involvement with data processing and analytics.
This project involves fetching data from a sql table using python fastapi, transforming it using pandas and loading it in excel sheet. The processed data is then visualized in Power BI template for reporting and insights.
Developed Python fastAPIs to fetch data from sql table using custom optimised queries.
Used Pandas for data manipulation and transformation. Applied data validation rules, cleaned inconsistencies, and standardized formats across all data sources.
loading the data in excel format to get the processed results
Created interactive dashboards in Power BI to visualize the processed data and provide actionable insights.
99.8% Reliability
no technical involvement
40% Faster
99.5% Accuracy
Built comprehensive validation rules to check data quality, detect anomalies, and handle edge cases before loading to production database.
Created direct Power BI to excel connection to the processed data for real-time dashboards and reporting capabilities.