Ling Dye

Being a full-time employee for over 22 years, I'm looking forward to new job experiences now. I've developed many successful products in the past, including Central Client Manager for FalconStor and Pinpoint Mobile for Flatirons. I'll offer best services as a code mentor.

I have been doing object-oriented design and programming since 2002.

I worked on the IETP product when I was in Flatirons. It's an aerospace IETP for various fleets like Boeing 737, 747 and Airbus A330. The size of the XML documents for a single manual is so large, usually over 1 GB. And the xsl templates we developed are really complicated.

I have a long experience of Agile project management with Scrum and Kanban. The team members play story points cards to make an agreement of the user story points by leveraging Fibonacci numbers in the Sprint planning meeting.

Setting up the HDFS environment and the Spark cluster. Purifying and transforming the semi-structured MongoDB data of auto parts, and saving the normalized data to ORC, ClickHouse and Elasticsearch with Spark. There are more than 13,000 columns in the most complex MongoDB collection and some columns contain arrays of thousands of nested structures. Using Spark to process user customized data, including adding constant columns, adding calculated columns, merging columns with different structures, pivoting (transforming tall table to wide table, converting rows to columns), unpivoting (transforming wide table to tall table, converting columns to rows) and all sorts of joins (inner, full, left, right, left semi, left anti and cross).

Managing and maintaining the Kubernetes cluster. Submitting jobs and monitoring the statuses of Kubernetes objects like no, po, svc, sparkapp and scheduledsparkapp.

I have been designing and normalizing database schemas, as well as writing SQL since 2002.