- Apply DataOps practices including CI CD version control automated testing release management and production support
- Design build and maintain batch and streaming data pipelines for ingestion transformation validation and publishing
- Develop curated reusable and well documented data products for BI analytics and ML
- Enable AI ML and GenAI teams with governed data access patterns
- Implement data quality checks observability lineage metadata management and monitoring
- Optimize pipeline performance storage compute cost and reliability
- Prepare feature datasets vector ready datasets and document corpora for AI ML and GenAI
- Support data governance privacy access control and compliance for enterprise data
- Translate business analytics and AI use cases into scalable data engineering solutions
- Use cloud data engineering tools to deliver resilient data pipelines
- Write production code in Python and SQL using data engineering frameworks