ShopDev is a full-service technology and design company specializing in software & product development and technology acceleration. We have 150+ employees working with enterprise clients and growth stage startups globally. We strategize, design, and build best-in-class digital products, transform teams, and co-create disruptive business models. We work as technology innovation partners with our clients, helping them adapt and thrive in the digital era.
About The Role:
At Shopdev, we love to craft high-end technology products for our clients. Our fast-paced Agile programming environment is exciting and well integrated, we work closely with industry experts to deliver meaningful experiences using the most cutting edge technology.
We are looking for a Technical Lead in Data Engineering with 6+ years of experience - ideally in the E-Commerce domain. As a Lead Data Engineer – EDW and Analytics, you will design, implement, and manage Data Models, ETLs/ELTs and Reports/Dashboards/Visualizations that will enable customers - turn data into actionable insights. Ideal candidate is an expert of data design and analysis, building data pipelines and ETL processes with a passion to enable stakeholders with data-driven solutions and insights with highest quality and attention to detail.
- Lead large-scale full-lifecycle enterprise data warehousing projects. Proven skills in data warehousing design/modeling, tuning, data cleansing, ETL process development, including scalability, performance, and high availability.
- Design, develop, deploy, and maintain Reports/Dashboards/Visualizations that can scale
- Technically lead end-to-end Data Warehousing architecture (i.e. Kimball/Dimensional modeling techniques in Oracle/AWS/Snowflake architecture and best practices).
- Troubleshoot technical issues with platforms, data discrepancies, data quality etc
- Develop and maintain documentation of the data architecture, data flow and data models.
- Be responsible for preparing estimates and defining technical solutions to proposals (RFPs).
- Ability to take ownership and drive end-to-end deliverables.
- Desire to mentor junior team members and develop their skills. Job Requirements and Skills
- B.S. degree in Information Technology, Computer Science, Engineering or equivalent experience.
- 6+ years of overall work experience in data engineering.
- At least 2+ years of hands-on experience leading projects on Snowflake platform.
- Must have Snowflake expertise in Design and Implementation of highly scalable, highly available, Cloud (PaaS, and SaaS) services and solutions.
- Analyze, architect, design, and actively develop cloud data warehouses, data lakes, data vaults and other cloud-based data solutions.
- Experience in Big Data technologies and concepts on Hadoop, Apache Spark, AWS and Snowflake platform and willing/adapting to future technologies. Experience with high-speed messaging frameworks and streaming (Kafka etc.).
- Experience with data driven frameworks that use cloud native technologies (SQS/SNS or similar).
- Experience using big data infrastructure tools such as Python, SQS and Redshift. A suitable candidate will be proficient in Scala, Spark, Spark Streaming, AWS and EMR.
- Strong experience in relational databases like Teradata/ Postgres/ Oracle / SQL Server etc. and object data stores such as MongoDB, Cassandra etc.
- Expertise in one or more ETL tools such as Informatica, Talend, Tableau, Matillion etc.
- Must have experience with a wide range of persistence stores such as columnar MPP, relational, and NoSql/document databases.
- Strong communication skills.