AI Data Engineer

Join FactualIQ as an AI Data Engineer and play a pivotal role in building and maintaining the data infrastructure that powers our enterprise Decision Engine. You will develop scalable data pipelines, optimize high-performance data systems, and enable intelligent workflows that support mission-critical AI decision-making for Fortune 1000 clients. This hands-on technical position offers the opportunity to collaborate with senior engineers while advancing your expertise in advanced data engineering tailored for AI applications.

Key Responsibilities
- Implement and maintain robust data pipelines supporting hundreds of concurrent agent workflows with reliable, real-time performance.
- Build and optimize relational (PostgreSQL, MySQL) and vector databases (Pinecone, Weaviate, Qdrant), including indexing strategies and query optimization.
- Develop semantic search capabilities, including embedding pipelines, hybrid search combining vector and traditional methods, and result ranking systems.
- Design and manage data processing workflows for agent context management, such as ETL pipelines, batch processing, and incremental updates.
- Implement data models for AI workflows, addressing temporal patterns, multi-tenant structures, and schema versioning.
- Develop data quality and monitoring systems, including validation frameworks, data drift detection, pipeline health checks, and automated alerts.
- Build multi-level caching systems for semantic data with cache invalidation and performance tuning.
- Create feature engineering pipelines encompassing feature computation, versioning, and low-latency serving infrastructure.
- Implement data governance components like data classification, PII detection and handling, and audit logging to meet compliance requirements.
- Optimize data system performance through query tuning, index design, and cost-effective storage strategies.
- Participate actively in sprint planning, technical discussions, and maintain clear documentation of data systems and processes.
- Stay updated on emerging best practices in data engineering and AI data systems, integrating new knowledge into daily work.

Required Qualifications
- Bachelor’s degree in Computer Science, Data Engineering, Statistics, or a related technical field, or equivalent practical experience.
- Minimum of 4 years of production experience in data engineering, including hands-on development of data pipelines, database systems, and data processing workflows.
- Strong proficiency in Python and familiarity with data processing libraries such as Pandas, Polars, and DuckDB, with attention to performance optimization.
- Experience with relational databases (PostgreSQL, MySQL), including query optimization, index design, and scaling techniques.
- Practical knowledge of vector databases and semantic search tools (Pinecone, Weaviate, Qdrant, ChromaDB), including embedding-based retrieval implementations.
- Familiarity with data pipeline orchestration tools like Airflow, Prefect, or Dagster, including workflow design and error handling.
- Advanced SQL skills, including complex queries, window functions, and CTEs, with the ability to optimize query performance.
- Understanding of data modeling techniques such as normalized schemas and dimensional modeling, with awareness of trade-offs.
- Experience working with cloud data platforms (AWS RDS/Aurora, GCP Cloud SQL, Azure Database) and infrastructure provisioning.
- Knowledge of embedding models and semantic processing, including model selection and quality evaluation.
- Experience building Retrieval Augmented Generation (RAG) systems, including chunking strategies and context optimization.
- Strong grasp of data quality practices, including monitoring, validation, and incident response.
- Experience in multi-tenant environments with considerations for data isolation and tenant-level optimization.
- Excellent communication skills and ability to collaborate effectively with engineers and stakeholders.

Preferred Qualifications and Additional Attributes
- Experience with knowledge graphs and graph databases (Neo4j, Neptune).
- Familiarity with streaming data systems such as Kafka or Kinesis and real-time processing patterns.
- Background in machine learning concepts, including feature engineering and data augmentation.
- Experience with time-series or analytical databases (InfluxDB, TimescaleDB, ClickHouse, DuckDB).
- Understanding of data mesh principles and domain-oriented data design.
- Knowledge of database internals, including storage engines and query execution.
- Experience mentoring junior engineers and contributing to team knowledge sharing.
- Contributions to open-source projects or technical community involvement.
- Relevant certifications such as AWS Database Specialty or GCP Data Engineer.

What We Value
We seek individuals with a growth mindset who continuously learn and adapt. Creativity and entrepreneurial flexibility are essential, along with resilience to quickly recognize and correct mistakes. We prioritize speed to solutions through rapid, well-planned iterations and design-forward approaches supported by thorough testing. Transparent, frequent, and constructive communication is critical, as is low-ego collaboration where feedback is welcomed and every voice is heard. Above all, we value mission alignment and a commitment to delivering the highest quality to support our clients’ success.

This role currently reports to FactualIQ’s President, with plans to transition reporting to a Tech Lead or Senior Tech Lead as the engineering team expands. Career progression from this position leads to a Senior AI Data Engineer role.