Senior Solution Architect - Databricks
End-to-End Data Architecture
• Collaborate with Databricks PS and customer
• on comprehensive end-to-end data architecture on Databricks platform
• Design and implement data ingestion strategies from various source systems
• Architect scalable storage solutions leveraging Delta Lake and lakehouse architecture
• Develop efficient data processing frameworks using Apache Spark and Databricks workflows
• Create data consumption layers for analytics, reporting, and machine learning use cases
• Ensure seamless data flow across the entire data lifecycle
Governance, Security & Compliance
• Collaborate with Databricks PS on implementing robust governance frameworks
• Design and enforce security policies and access controls within Databricks environments
• Ensure compliance with regulatory requirements and industry standards
• Implement data lineage, audit trails, and monitoring capabilities
• Optimize performance through best practices in cluster configuration and query optimization
• Establish data quality standards and validation processes
Data Modelling & Design
• Define business-aligned data models that support organizational objectives
• Design Delta table structures optimized for performance and scalability
• Create dimensional and normalized data models based on business requirements
• Implement medallion architecture (Bronze, Silver, Gold layers) for data refinement
• Develop data schemas that support both analytical and operational workloads
• Document data models and maintain data dictionaries
Technical Leadership & Collaboration
• Lead technical discussions with clients and stakeholders to understand business requirements
• Provide technical guidance to development teams on Databricks best practices
• Work closely with Infrastructure (Infra), Applications (Apps), and Cyber teams for integrated delivery
• Mentor junior architects and engineers on Databricks technologies
• Present solution architectures and technical recommendations to client leadership
Solution Implementation
• Oversee implementation of Databricks solutions from design through deployment
• Ensure solutions meet performance, scalability, and reliability requirements
• Support proof-of-concepts and pilot implementations
• Conduct architecture reviews and ensure alignment with enterprise standards