Manager - Data Engineer AI
AXA ดูงานทั้งหมด
- กรุงเทพมหานครและปริมณฑล
- งานประจำ
- ฟูลไทม์
- Develop, establish, and enforce data engineering standards for data pipelines, emphasizing modular design, code reuse, testability, and maintainability.
- Align the data engineering roadmap with the organization’s strategic objectives to drive data-driven decision making and AI/ML model development.
- Lead modernization initiatives, such as transitioning from Data Lake to Lakehouse architectures and adopting advanced orchestration and streaming technologies, to ensure long-term scalability.
- Oversee the creation and maintenance of comprehensive documentation, runbooks, and governance artifacts to support audits, compliance, and operational excellence.
- Implement and oversee comprehensive testing strategies, including data transformations, quality gates, and end-to-end validation processes.
- Build and manage CI/CD pipelines for data code and infrastructure, ensuring automated testing, version control, and rollback capabilities.
- Standardize on orchestration and transformation frameworks (e.g., Talend ETL, Spark, Databricks, AWS Glue) to promote consistency, reusability, and best practices.
- Ensure the stable, reliable, and efficient day-to-day operation of data platforms and pipelines, including proactive monitoring, incident management, change management, and SLA adherence.
- Drive data privacy, security, and compliance by implementing robust access controls, encryption, audit trails, and regulatory adherence.
- Optimize cloud data infrastructure for cost efficiency and performance, including effective data lifecycle management and storage tiering.
- Foster the development and deployment of scalable data pipelines and feature stores that enable AI and machine learning models, ensuring data quality, lineage, and accessibility for AI/ML teams.
- Bachelor's Degree in Computer Science, Computer Engineering, other technical discipline, or equivalent work experience
- 5-10 years of experience in Data Engineering, Data Warehousing, or related fields, with hands-on experience designing and orchestrating data workflows using Talend ETL or Spark, along with a solid understanding of cloud-based data engineering.
- Proven experience in building data pipelines and feature stores to support AI and machine learning models.
- Advanced experience designing and orchestrating data integration workflows using Talend ETL.
- Deep understanding of modern data architecture, including batch and streaming ingestion patterns.
- Extensive experience with big data processing frameworks such as Apache Spark (preferably on AWS Glue and Databricks).
- Proficiency in programming and scripting languages, especially Python, Scala, or Java.
- Practical experience with AWS cloud ecosystem, including services like S3, RDS, Redshift, Athena, and AWS Glue.
- Strong SQL skills, with experience managing enterprise relational databases (e.g., MSSQL, Oracle, PostgreSQL, and NoSQL).
- Proven experience developing data pipelines and feature stores to support AI and machine learning workflows.
- Good English skills at business level