I have extensive experience as a Senior Data Engineer, specializing in enhancing dashboard performance, developing ETL queries, and building data lakes. I have also worked on innovative data integration solutions.
Teikametrics – Bengaluru, India (Remote) August 2021 – Present
Senior Data Engineer
● Enhance Looker dashboard performanc...
Teikametrics – Bengaluru, India (Remote) August 2021 – Present
Senior Data Engineer
● Enhance Looker dashboard performance by creating a read-optimized presentation layer on Unity Catalog and successfully migrated complex queries from Snowflake to Databricks
● Develop ETL queries using DBT (Python, SQL)
● Collaborate with business and leadership teams to understand mission critical business requirement and developed interactive dashboards and visualizations using Looker
● Develop Data Lake using Databricks delta format ingesting Amazon, Walmart API data through Kafka.
● Develop ETL scripts to clean and transform data into Snowflake, orchestrated the ETLs on Airflow.
● Actively learning machine learning ML models and natural language processing techniques for potential integration into data systems.
St. Jude Children’s Research Hospital – Memphis, TN December 2019 – August 2021
Data Engineer
Project: Build St. Jude Clo...
St. Jude Children’s Research Hospital – Memphis, TN December 2019 – August 2021
Data Engineer
Project: Build St. Jude Cloud for the global research community
● Worked closely with the SJCRH researchers, SMEs, business analysts, and technical leads/architects to lead the implementation of innovative data integration solutions across on-premises and cloud-based environments.
● Built ETL systems using python. The raw data and image files are pulled from shared location and then processed using python to be loaded into MS Azure cloud cosmos DB storage in JSON document format.
● Architected and implemented a highly scalable data pipeline using Delta Lake, capable of efficiently processing 1TB of daily data ingestion via Spark Structured Steaming and batch processing on Databricks.
● Worked cross-functionally with teams and mentored junior engineers on best practices in data engineering and BI analytics.
Cigna Health Spring – Nashville, TN October 2019 – December 2019
Business Intelligence Consultant/ Systems Analyst
Proj...
Cigna Health Spring – Nashville, TN October 2019 – December 2019
Business Intelligence Consultant/ Systems Analyst
Project: Building Cigna Operational Support System
● Collaborate with data modelers to develop, document and implement data models for OLTP and OLAP systems.
● Deliver throughout the full software development life cycle from scoping, analysis, design, implementation, and quality assurance to delivery and support.