Cloud Engineer
Key Responsibilities
- Manage and optimize cloud-based data storage solutions including Amazon S3, Amazon RDS, Amazon Redshift, Amazon DynamoDB, databases, data lakes, and data warehouses.
- Design, manage, and optimize Databricks Delta Lake environments and data processing solutions.
- Implement and manage Informatica Intelligent Data Management Cloud (IDMC) for metadata management, data cataloging, integration, data quality, and governance.
- Develop, maintain, and optimize data pipelines for data ingestion, processing, transformation, and movement using AWS Glue, AWS Data Pipeline, AWS Lambda, Databricks, and Informatica IDMC.
- Integrate data from multiple internal and external sources into AWS and Databricks environments while ensuring data consistency, accuracy, quality, and reliability.
- Design and manage ETL/ELT processes to cleanse, transform, enrich, and prepare data for analytics and reporting.
- Leverage Databricks Apache Spark capabilities for large-scale data processing and transformation.
Qualifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, Data Engineering, or a related discipline.
- 6+ years of professional experience in Cloud Engineering, Data Engineering, or Cloud Data Platforms.
- Strong hands-on experience with AWS, Databricks, and Informatica IDMC.
- Strong analytical, troubleshooting, and problem-solving skills.
- Ability to work independently as well as collaboratively in a cross-functional environment.