Migration of Existing Architecture to AWS Serverless Architecture• Migrate existing Data pipeline to Serverless Architecture. - AWS Lambda - S3 - Athena - Redshift- Cloud Watch - SQS services- AWS Glue• Implemented ETL Jobs and Transformations to load data from different sources topre_staging table, Cleansing data process, moving to stage area, and then to target table usingPentaho Data Integration.• Data wrangling and Change data capture.• Developed a complete ETL pipeline, which included data extraction from tabular and non-tabulardata sources and performed Merging of Data Streams, Data Cleansing, Data Validation, SendingEmail, and Error Handling in the ETL pipeline using Pentaho Data Integration.• Schedule ETL jobs using a Custom-built scheduler in Pentaho• Providing support, and optimization on running ETL processes• Collaborating with the customer’s technical team to gather technical requirements such as performance, maintainability, and scalability.• Producing design documentation.• Managing the approval and acceptance process for the designs and implementation in cooperation with the client.• Resolving ongoing maintenance issues and bug fixes; monitoring daily scheduled jobs and performance tuning of transformation and jobs.• PostgreSQL DB to perform queries + PLSQL queries on dataMigration of Existing Architecture to AWS Serverless Architecture • Migrate existing Data pipeline to Serverless Architecture.   - AWS Lambda - S3 - Athena - Redshift - Cloud Watch - SQS services - AWS Glue • Implemented ETL Jobs and Transformations to load data from different sources to pre_staging table, Cleansing data process, moving to stage area, and then to target table using Pentaho Data Integration.
• Data wrangling and Change data capture.
• Developed a complete ETL pipeline, which included data extraction from tabular and non-tabular data sources and performed Merging of Data Streams, Data Cleansing, Data Validation, Sending Email and Error Handling in the ETL pipeline using Pentaho Data Integration.
• Schedule ETL jobs using a Custom-built scheduler in Pentaho
• Provide support, and optimization on running ETL processes
• Collaborate with the customer’s technical team to gather technical requirements such as performance, maintainability, and scalability.
• Producing design documentation.
• Managing the approval and acceptance process for the designs and implementation in cooperation with the client.
• Resolving ongoing maintenance issues and bug fixes; monitoring daily scheduled jobs and performance tuning of transformation and jobs.
• PostgreSQL DB to perform queries + PLSQL queries on data












Skills: Apache Airflow · Pentaho · Snowflake Cloud · Data Warehouse Architecture · Python (Programming Language) · Extract, Transform, Load (ETL) · SQL · Amazon Web Services (AWS) · PostgreSQL