Staff Senior Staff Web3 Big Data Engineer
You own the architecture, development, and optimization of a big data platform for on-chain data, trading behavior, and user profiles. You build real-time and batch pipelines, data warehouses and lakes, AI infrastructure, and intelligent data applications while optimizing reliability and collaborating with AI, product, risk, and growth stakeholders.
Responsibilities
- Own the architecture, development, and optimization of the big data platform
- Design and build real-time and batch pipelines for on-chain data
- Build and maintain the data warehouse and data lake
- Define data-layering standards and ensure data quality, consistency, and timeliness
- Build AI-driven applications for anomaly detection, risk modeling, behavior prediction, and Text2SQL
- Build vector databases, feature platforms, and embedding pipelines
- Partner with AI teams on training data and feature engineering pipelines
- Optimize big data job performance and resource efficiency
- Track developments in big data, Web3 infrastructure, and AI
- Communicate and document technical work in English
Requirements
- Bachelor's degree or above in Computer Science, Software Engineering, or a related field
- 7+ years of big data development experience
- Proficiency with Hadoop, Spark, and Flink
- Experience building batch and real-time data warehouses
- Familiarity with Hive, Kafka, HBase, ClickHouse, and Doris
- Large-scale cluster tuning experience
- Big data development skills in Java, Scala, or Python
- Strong SQL tuning ability
- Understanding of LLM fundamentals, prompt engineering, and RAG
- Experience with vector databases or embedding data processing
- Experience connecting big data platforms with AI/ML training and inference pipelines
- Understanding of blockchain fundamentals and on-chain data structures
- Fluent English reading and writing
- Knowledge of Solidity is a plus
Benefits
- Comprehensive insurance coverage for employees and their dependants