Azure Migration and ETL
Migrated the on-premises services and data pipelines into Azure. Designed and implemented various frameworks for data quality and data m...
Migrated the on-premises services and data pipelines into Azure. Designed and implemented various frameworks for data quality and data movements.
Technologies: PySpark, Azure Data Factory, Azure Data Lake, Python, Azure Databricks
Implemented transformation frameworks that can move data from various sources into Azure Data Lake.
Designed and developed rule based Data Quality Framework.
Python
Azure
Apache Spark
View more
Python
Azure
Apache Spark
Azure Data Factory
View more
Document Indexing and Searching
Data pipeline to collect and refresh data from various sources using AWS Services.
Technologies: Python, AWS Glue, AWS Comprehend, AWS ...
Data pipeline to collect and refresh data from various sources using AWS Services.
Technologies: Python, AWS Glue, AWS Comprehend, AWS Textract, AWS CloudSearch
Retrieved data from various sources and saved them into a data warehouse.
Indexed the documents using NLP techniques and expose the data for fast lookup.