Crop Picture
NOTE: Do not upload any image that violates our Picture Policy. Doing so will result in the removal of the concerned image without prior notification.
概要
Results-driven data engineer with expertise in data scraping, data modeling, and data pipeline development. Experienced in utilizing technologies such as Scrapy, Docker, Kubernetes, AWS Ecosystem, SQL and in-house products like Lazarus. Proficient in building and maintaining ETL processes, designing ODL and ADL for raw data, and leveraging APIs for data modeling.
项目
工作经历
Data Engineer
Led statistics scraping to extract competitor tendencies, using Scrapy framework in a Docker kubernetes and AWS environment.
Implemented synchronization of source statistics into the RDL using in house Product Lazarus.
Built the physical data model for a classified ads platform and designed and evolved data models for ODL and ADL to ensure efficient data processing, analysis, and retrieval.
Built and maintained numerous information pipelines of ETL on Matillion
Developed a facts pipeline on Matillion to extract user information from the ODL based on business necessities Uploaded this facts to MoEngage for sending targeted notifications, optimizing consumer engagement and increase revenue
ETL and Automation Specialist
Analyzed and documented client requirements, collaborating with the onsite team.
Developed scripts to extract data from diverse websites and stored it in a MySQL database.
Transformed and prepared the data for data lakes, ensuring compatibility and consistency.
Automated ETL jobs with script-based solutions for improved efficiency.
Ensured data accuracy through validation mechanisms and indexing in MySQL tables.
Utilized reverse engineering techniques to handle JSON, Ajax, and other web technologies for efficient big data processing.
Created a Rails application to scrape data from mobile and broadband companies.
Applied object-oriented programming techniques to enhance ETL jobs in the company\'s Hamster product.
Implemented a boot system for OpenCorporates projects, retrieving, transforming, and loading data efficiently.
Intern
In this company my role was as a Full Stack Developer and I was serving this company help to build their web development projects
