Databricks Data Engineer
Salary: Competitive
- Remote - Auckland, Wellington & Christchurch
- Immediate start available
- Rare opportunity for a certified consultant
Databricks Data Engineer
The opportunity: In this role, you will design and build analytics solutions that deliver significant business value. You will collaborate with other data and analytics professionals, management, and stakeholders to ensure that technical requirements align with business needs. Your responsibilities will include creating scalable data architecture and modeling solutions that support the entire data asset lifecycle.
Your Key Responsibilities: As a Senior Data Engineer, you will play a pivotal role in transforming data into actionable insights. Your time will be spent on various responsibilities, including:
- Designing, building, and operating scalable on-premises or cloud data architecture.
- Analyzing business requirements and translating them into technical specifications.
- Optimising data flows for target data-platform designs.
- Design, develop, and implement data engineering solutions using Databricks on cloud platforms (e.g., AWS, Azure, GCP).
- Collaborate with clients to understand their data needs and provide tailored solutions that meet their business objectives.
- Lead end-to-end data pipeline development, including data ingestion, transformation, and storage.
- Ensure data quality, integrity, and security throughout the data lifecycle.
- Provide technical guidance and mentorship to junior data engineers and team members.
- Communicate effectively with stakeholders, including technical and non-technical audiences, to convey complex data concepts.
- Manage client relationships and expectations - ensuring high levels of satisfaction and engagement.
- Stay updated with the latest trends and technologies in data engineering and cloud computing.
- This role offers the opportunity to work with cutting-edge technologies and stay ahead of industry trends, ensuring you gain a competitive advantage in the market. The position may require regular travel to meet with external clients.
Skills And Attributes For Success
To thrive in this role, you will need a blend of technical and interpersonal skills. Your ability to communicate effectively and build relationships will be crucial. Here are some key attributes we look for:
Essential requirements:
- Live and work in NZ
- Holder of Databricks certification
- Solid experience with Delta Live Tables
- Solid experience with Asset Bundles
- Solid experience with Delta Lake and Unity Catalogue.
- Solid experience with CI/CD pipelines
- Solid experience with infrastructure as code.
- Solid client-facing experience.
- Strong analytical and decision-making skills.
- Proficiency in cloud computing and data architecture design.
- Experience in data integration and security.
- Ability to manage complex problem-solving scenarios.
- A Bachelor’s degree in Computer Science, Engineering, or a related field (4-year degree). Master’s degree preferred
- Typically, no less than 2 - 4 years of relevant experience in data engineering, with a focus on cloud data solutions.
- 5+ years of experience in data engineering, with a focus on cloud data solutions.
- Expertise in Databricks and experience with Spark for big data processing.
- Proven experience in at least two end-to-end data engineering implementations, including:
- Implementation of a data lake solution using Databricks, integrating various data sources, and enabling analytics for business intelligence.
- Development of a real-time data processing pipeline using Databricks and cloud services, delivering insights for operational decision-making.
- Strong programming skills in languages such as Python, Scala, or SQL.
- Experience with data modeling, ETL processes, and data warehousing concepts.
- Excellent problem-solving skills and the ability to work independently and as part of a team.'
- Each Applicant must possess strong communication and interpersonal skills, with a focus on client management.
Thank you for considering this role today. All those who meet Client criteria will be contacted asap for an initial confidential chat.