Senior Solution Architect - Databricks

Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success we have evolved to become one of the top trusted providers in Singapore and service a client base across banking and financial services, insurance, information technology, healthcare, retail, and supply chain.We are currently looking to hire Senior Solution Architect - Databricks.This is an exciting opportunity to expand your skill set, achieve job satisfaction and work-life balance. More details as below.Job Role:Job DescriptionEssential Technical Skills• Databricks Platform: Proven hands-on experience with Databricks platform including workspace administration, cluster management, and workflow orchestration• Unity Catalog: Demonstrated experience implementing Unity Catalog for unified governance, including metastore configuration, catalog/schema design, and access control policies• Data Modelling: Strong expertise in designing and implementing data models, including dimensional modeling, data vault, and lakehouse architectures• Delta Lake: Deep understanding of Delta Lake features including ACID transactions, time travel, and optimization techniques• Apache Spark: Proficiency in Spark SQL, DataFrames, and performance tuning• Programming: Strong skills in Python, SQL, and/or Scala for data processing• Cloud Platforms: Experience with Azure, AWS, or GCP cloud servicesAdditional Technical Skills• Data pipeline development and ETL/ELT processes• Data governance frameworks and metadata management• Performance optimization and troubleshooting• CI/CD practices for data platforms• Data quality and validation frameworks• Understanding of data security and compliance requirementsProfessional Experience• Minimum 8-10 years in data architecture, engineering, or analytics roles• At least 3-5 years of hands-on experience with Databricks platform• Databricks Certified Associate Developer for Apache Spark (mandatory)• Proven track record implementing Unity Catalog in enterprise environments• Experience designing and implementing complex data models for large-scale systems• Background working with Databricks Professional Services or similar partnerships preferred• Experience across multiple industry sectors (Public Service, Financial Services, Healthcare, etc.) is advantageousCertifications (Preferred) - mandatory to have at least one certification• Data Engineer Professional• Cloud platform certifications (Azure Data Engineer, AWS Data Analytics, or GCP Data Engineer)• Relevant data management or analytics certificationsSoft Skills• Excellent communication and presentation abilities• Strong analytical and problem-solving capabilities• Ability to work collaboratively with Partners and cross-functional teams• Client-focused mindset with relationship-building skills• Self-motivated with ability to manage multiple priorities• Experience working in agile environmentsPriority for candidates in SingaporeEnd-to-End Data Architecture• Collaborate with Databricks PS and customer• on comprehensive end-to-end data architecture on Databricks platform• Design and implement data ingestion strategies from various source systems• Architect scalable storage solutions leveraging Delta Lake and lakehouse architecture• Develop efficient data processing frameworks using Apache Spark and Databricks workflows• Create data consumption layers for analytics, reporting, and machine learning use cases• Ensure seamless data flow across the entire data lifecycleGovernance, Security & Compliance• Collaborate with Databricks PS on implementing robust governance frameworks• Design and enforce security policies and access controls within Databricks environments• Ensure compliance with regulatory requirements and industry standards• Implement data lineage, audit trails, and monitoring capabilities• Optimize performance through best practices in cluster configuration and query optimization• Establish data quality standards and validation processesData Modelling & Design• Define business-aligned data models that support organizational objectives• Design Delta table structures optimized for performance and scalability• Create dimensional and normalized data models based on business requirements• Implement medallion architecture (Bronze, Silver, Gold layers) for data refinement• Develop data schemas that support both analytical and operational workloads• Document data models and maintain data dictionariesTechnical Leadership & Collaboration• Lead technical discussions with clients and stakeholders to understand business requirements• Provide technical guidance to development teams on Databricks best practices• Work closely with Infrastructure (Infra), Applications (Apps), and Cyber teams for integrated delivery• Mentor junior architects and engineers on Databricks technologies• Present solution architectures and technical recommendations to client leadershipSolution Implementation• Oversee implementation of Databricks solutions from design through deployment• Ensure solutions meet performance, scalability, and reliability requirements• Support proof-of-concepts and pilot implementations• Conduct architecture reviews and ensure alignment with enterprise standardsCONSULTANT DETAILSConsultant Name: Abinaya R Reg No: R1765546 Avensys Consulting Pte Ltd EA Licence 12C5759 Privacy Statement: Data collected will be used for recruitment purposes only. Personal data provided will be used strictly in accordance with the relevant data protection law and Avensys' privacy policy.