Senior Data Engineer

We are looking for an experienced senior data engineer

with strong expertise in PySpark, Python, and Informatica BDM development to support large-scale data & analytics initiatives within an enterprise banking environment. The ideal candidate should possess hands-on experience in building and maintaining robust ETL pipelines, data marts, and scalable data processing solutions across structured, semi-structured, and unstructured data sources.

The role requires strong analytical capabilities, end-to-end SDLC experience, and the ability to work closely with cross-functional teams throughout development, testing, deployment, and production support activities.

Key responsibilities

Design, develop, and maintain scalable ETL pipelines and data mart solutions.
Develop high-performance data processing solutions using PySpark and Python.
Perform end-to-end SDLC activities including development, UAT support, bug fixing, production deployment, and post-production support.
Build and optimize data transformation pipelines for large-scale enterprise data platforms.
Perform data analysis, code debugging, and performance tuning across PySpark and SQL-based solutions.
Collaborate with business, analytics, and engineering teams to understand and implement data requirements.
Ensure data quality, integrity, scalability, and reliability across data pipelines.
Work with structured, semi-structured, and unstructured datasets in enterprise environments.
Participate in code reviews and implement software engineering best practices.
Support CI/CD implementation and data pipeline deployment activities.
Troubleshoot production issues and implement effective resolutions.
Contribute to technical documentation and knowledge-sharing initiatives.

Required technical skills

Programming & big data technologies

Python
PySpark
Apache Spark
Informatica BDM (Big Data Management)
Hadoop
MapReduce
Hive
Pandas

Data engineering & ETL

ETL pipeline development
Data mart development
Data warehousing concepts
Data transformation & processing
Data pipeline optimization

Databases & query languages

Oracle SQL
SQL
NoSQL databases
Strong analytical and query-writing skills

Tools & platforms

Jupyter Notebook
Git / version control
CI/CD pipelines
Testing & validation frameworks

Required experience

Minimum 5+ years of commercial experience in data engineering or data analytics projects.
Strong hands-on experience in PySpark and Python-based ETL development.
Experience building enterprise-grade ETL pipelines and data mart solutions.
Strong experience in Informatica BDM development.
Experience handling end-to-end SDLC activities including development, UAT, production deployment, and post-production support.
Strong expertise in Oracle SQL and data analysis.
Hands-on experience debugging PySpark code and optimizing data processing workflows.
Experience working with production-grade data pipelines and large datasets.
Strong understanding of software engineering principles and coding best practices.
Experience working with Agile delivery environments.

Preferred domain experience

Banking
Financial services
Digital products
Data & analytics platforms

Nice to have

Experience working with enterprise data lake and big data ecosystems.
Exposure to cloud-based data platforms.
Experience working with CI/CD and automated data pipeline deployments.
Knowledge of modern data engineering best practices and scalable architectures.

Daily tech stack

The selected candidate will work extensively with:

Python
PySpark
Informatica BDM
Apache Spark
Jupyter Notebook
Oracle SQL
SQL & NoSQL databases
Hadoop ecosystem (Hive, MapReduce)
CI/CD tools
ETL & data warehousing technologies

Functional competencies

Strong problem-solving and analytical skills.
Excellent debugging and troubleshooting capabilities.
Ability to work independently in a fast-paced Agile environment.
Strong ownership mindset and attention to detail.
Effective stakeholder communication and collaboration skills.
Ability to manage multiple priorities and production support activities.

Show