How to truncate/delete a database table from the AWS Glue job?
Truncating/Deleting a Mysql/MariaDB/Postgres table from AWS Glue using python
5
Rakesh Kaswan
Mentor
(64 reviews)
US$20.00
For every 15 mins
126
Sessions/Jobs
ABOUT ME
Data Engineer | Full stack developer |Python & SQL Expert
Data Engineer | Full stack developer |Python & SQL Expert
I am an expert Data Engineer having 10 years of experience with exposure on in-house, hybrid & AWS, GCP cloud services. I hold Data Architect Nanodegree from Udacity. My Primary skills include -
- Languages - Python, PHP, Ruby, Javascript, HTML, CSS
- Packages - PySpark, Pandas, Numpy, Flask, Jquery, Bokeh
- Tools - Spark, Hadoop, Hive, Mysql, Postgres, Airflow
- File Formats - json, csv, txt, Parquet
- GCP - Cloud functions, DataProc, Bigquery, GCS
- AWS - Lambda functions, API Gateway, Athena, Cloudwatch, Glue, DataBrew, Kinesis, S3, Redshift, Redshift Spectrum, SNS, SQS
I mostly worked on developing, maintaining, optimising ETL/Data pipelines, used variety of tools & process during pipeline execution. Apart from building pipelines i holds good hands on building custom scripts or solutions.
I am a quick learner & having positive mindset which helps me to tackle a complex problem in easy way. During my experience i optimised numerous pipelines where improved time by ~70% & saved cost with magnitude of 10th part.
If you are looking for any help in learning, solving complex data problem, you are at right place. I am happy to help here.
Thanks.
Hindi, English
New Delhi (+05:30)
Joined March 2017
EXPERTISE
6 years experience | 3 endorsements
I am having both development & teaching experience in python. Worked on big data processing & web development using python. My experience...
I am having both development & teaching experience in python. Worked on big data processing & web development using python. My experience include both on python 2.X and python 3.X.
3 years experience | 1 endorsement
I worked as full stack developer too using php5 & php7. Having exposure on Laravel, yii2, codeigniter
I worked as full stack developer too using php5 & php7. Having exposure on Laravel, yii2, codeigniter
4 years experience | 3 endorsements
I worked on numerous AWS services.
I worked on numerous AWS services.
7 years experience | 7 endorsements
I have good understanding of relational query processing. I can write complex analytical queries which includes ranking, grouping level s...
I have good understanding of relational query processing. I can write complex analytical queries which includes ranking, grouping level sum/avg, statistical etc.
5 years experience | 3 endorsements
I worked on complex topics like physical level joins, equi distribution of data, caching, check pointing etc..
I worked on complex topics like physical level joins, equi distribution of data, caching, check pointing etc..
5 years experience | 1 endorsement
I worked on Glue using Glue studio & non studio mode. Converted SQL stored procs. to Glue job.
I worked on Glue using Glue studio & non studio mode. Converted SQL stored procs. to Glue job.
6 years experience
As a Data engineer I worked on interesting projects using AWS & GCP cloud infra.
As a Data engineer I worked on interesting projects using AWS & GCP cloud infra.
REVIEWS FROM CLIENTS
(64 reviews)
Myriam Naeem
July 2024
Rakesh is very friendly and educational. It is very efficient and helped me a lot, I recommend it!
Manuel
March 2024
great help, thanks Rakesh
Lisa Perl
March 2023
Great job. Very helpful and patient.
Tianyu
February 2023
Thank you so much for the wonderful help!
Riyad
January 2023
very good!
Saffi Keagan
January 2023
Great help!
Sandeep
January 2023
Rakesh is very knwoledgeble , patient and knows what he is doing. He guided me in a data engineering project, I would definitely recommend him.
Brent Robbins
December 2022
Rakesh was super knowledgable and patient. Highly recommend.
A T
December 2022
Very smart and patient guy, willing to explain everything he wrote!
Lucas Hunt
December 2022
Rakesh is the most knowledge Python person I've met, and he was able to help with a timely manner, will definitely work with him again.
SOCIAL PRESENCE
EMPLOYMENTS
Data Engineer
FromAtoB
2019-11-01-2021-03-01
* Improved performance of existing ETL processes, SQL queries & SPARK execution plan running daily to
publish into BigQuery around 30%
* ...
* Improved performance of existing ETL processes, SQL queries & SPARK execution plan running daily to
publish into BigQuery around 30%
* Leading research on Insight publication across product teams & self data exploration tools using python,
flask, bokeh
* Translate business propositions into quantitative queries and collect/clean the necessary data
* Collaboration with Stakeholders & Business for Custom Insights & Dashboarding using DataStudio
* Improving Dataproc Cluster configurations for efficiently utilizing resources
HTML/CSS
Python
jQuery
View more
HTML/CSS
Python
jQuery
SPARQL
Google BigQuery
Docker
Google Cloud Platform
JavaScript
Apache Spark
AWS
View more
Data Science Engineer
Dunnhumby
2016-12-01-2019-11-01
* Leading Migration of Analytics & core data science solutions from SAS & Excel to Automation, Spark,
Hadoop
* Acting as SME for Advance...
* Leading Migration of Analytics & core data science solutions from SAS & Excel to Automation, Spark,
Hadoop
* Acting as SME for Advance Spark (Optimization & Handling Big Data), SQL & Python
* Optimized deliveries by 75% as compared to previous turnaround in Customer Engagement
* Initiated training of colleagues on Spark processing, optimization, simplifying DAG, Python, Optimal
handling Big Data processing
* Collaboration with Stakeholders & Data Scientists in Customer Engagement
* Developed Anonymised data generation Algorithm to on-board Contractors
Python
SQL
SAS
View more
Python
SQL
SAS
SPARQL
Google Cloud Platform
Data Migration
Apache Spark
Apache Hive
View more
Software Engineer
Housefull
2015-10-01-2016-12-01
* Co-developed Marketplace Backend & Order Management System
* Initiated API development in Python for Frontend & Mobile integration
* Co...
* Co-developed Marketplace Backend & Order Management System
* Initiated API development in Python for Frontend & Mobile integration
* Collected 25GB of Data within 2 months by applying webhook in request & response which helped to
optimize API & Bug fixing
Python
jQuery
MySQL
View more
Python
jQuery
MySQL
Yii
JavaScript
PHP 5
View more
PROJECTS
Python Training
Students/Professionals
2021
In this project i provided Python training, where i covered
- Python Basic
- Python Structures
- Python classes or Object Orentations
- ...
In this project i provided Python training, where i covered
- Python Basic
- Python Structures
- Python classes or Object Orentations
- Working with CSV/JSON/TXT
- Working with Modules
- Working with Packages
- Working with Pandas/Numpy/Pyspark
Python
NumPy
Apache Spark
View more
Python
NumPy
Apache Spark
View more
Streaming Data Pipeline
2022
In this pipeline,
- mobile events were collected using API Gateway
- API Gateway send data to kinesis
- Kinesis sends raw data to s3
- G...
In this pipeline,
- mobile events were collected using API Gateway
- API Gateway send data to kinesis
- Kinesis sends raw data to s3
- Glue schema
- Athena queries for aggregation & Reporting
- Glue Job
- Loading data to Redshift
Amazon S3
Amazon Redshift
AWS Lambda
View more
Amazon S3
Amazon Redshift
AWS Lambda
AWS
View more