Senior Data Engineer
You will design, develop, and maintain real-time and batch data processing pipelines using Apache Flink. You will architect streaming systems, support feature engineering and feature stores for production machine learning models, collaborate with AI, compliance, risk, and engineering stakeholders, and resolve performance and data integrity issues.
Responsibilities
- Design, develop, and maintain real-time and batch data processing pipelines using Apache Flink
- Design and develop Flink streaming system architecture
- Support feature engineering and feature store development for production machine learning models
- Collaborate with AI, compliance, risk, and engineering teams
- Troubleshoot performance bottlenecks in real-time stream processing
- Improve system performance and data integrity
Requirements
- Apache Flink
- Real-time data streaming and processing
- Streaming architecture
- Alibaba Cloud Hologres
- Doris
- ClickHouse
- Spark
- Feature engineering
- Feature store development
- At least 10 years of relevant work experience
- FinTech or blockchain background
- Strong English communication skills