Senior Data Software Engineer with Databricks and Azure
We are looking for a Senior Data Software Engineer to join the Data Engineering CoE, building and scaling data pipelines in Databricks with PySpark and Python to deliver data products for the client's stakeholders.
Responsibilities
- Support a team of data engineers to build pipelines used by MLOps and ML Engineers on ML modeling teams
- Develop, optimize, and maintain data transformation pipelines in Databricks (PySpark)
- Work with data stored in ADLS Gen2 and SAP HANA Data Lake, primarily in Delta/Parquet format
- Implement and maintain data quality checks, including schema validation, deduplication, enrichment, and tagging
- Communicate with stakeholders to understand business processes and model input data
- Tune performance for large-scale datasets
Requirements
- 3+ years of experience in data engineering or a related field
- Proficiency in Python, PySpark, and Databricks, including Delta Lake
- Familiarity with software version control tools such as GitHub and Git
- Experience with CI/CD frameworks such as GitHub Actions
- Knowledge of data lake technologies
- Experience working with MS Azure
- English proficiency at B2 level or higher
Nice to have
- Familiarity with at least one other programming or scripting language, such as Java, SQL, or Scala
Benefits
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
- Discounts in local language schools, including online courses for the Kazakh language
We cover it all:
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Benefits package (sports activities, a variety of stores and services)
- Housing support program: preferential mortgage access via Otbasy Bank partnership