Senior Data Architect - Databricks / Data Lakehouse / GenAI

D L Resources · Singapore

Sector
AI
Function
Product & Engineering
Level
Mid-Level
Employment type
Contract
Posted
2026-11-17
Source
mycareersfuture

Senior Data Lakehouse ArchitectClient: Bank Sector Client About the RoleD L Resources is supporting a leading banking-sector client in hiring an experienced Senior Data Lakehouse Architect to lead the end-to-end architecture, design, and evolution of an enterprise Lakehouse platform.The role will be responsible for defining the technical vision, target architecture, and engineering standards for modern data platforms, including data products, data marketplace, knowledge layers, real-time data processing, Generative AI, RAG, vector search, graph technologies, and agentic workloads.The successful candidate should bring deep experience in large-scale enterprise data architecture, distributed computing, cloud and hybrid platforms, performance engineering, data governance, and modern DevSecOps practices.Key ResponsibilitiesOwn the end-to-end architecture and technical roadmap for the enterprise Data Lakehouse platform.Design and evolve platform capabilities supporting:Data products and data marketplaceKnowledge and semantic layersStructured, semi-structured, and unstructured dataReal-time and streaming workloadsRAG and Generative AI use casesVector and graph-based data servicesAgentic AI and autonomous workflow patternsDefine target architectures for applications and platform services with a focus on reusability, scalability, resilience, security, and operational efficiency.Develop reusable architecture patterns, frameworks, and technical accelerators for:Unstructured and multimodal content extractionBatch and streaming architecturesLambda and event-driven architecturesRetrieval-Augmented Generation (RAG)Agentic workloads and AI-driven data processingPartner with business and technology stakeholders to define data contracts, SLAs, data quality standards, and governance requirements for enterprise data products.Provide architecture oversight and quality assurance to ensure solutions comply with the client’s software engineering, security, and delivery standards.Review solution designs, technical specifications, non-functional requirements, and implementation approaches produced by engineering teams.Participate in technology and product evaluations, proof-of-concepts, and RFP processes.Guide installation, customization, integration, and operationalization of enterprise software platforms and technologies.Lead performance engineering, capacity planning, scalability reviews, and optimization of distributed data workloads.Partner with infrastructure, security, application, cloud, AI/ML, and operations teams to deliver integrated technology solutions.Drive continuous service improvement, engineering automation, platform standardization, and operational excellence.Produce architecture documentation, solution designs, implementation guidelines, operational standards, and technical runbooks.Required Experience10–15 years of experience in enterprise Data Engineering, Big Data, Data Architecture, Data Lake, or Lakehouse implementations.Strong experience designing and delivering large-scale Data Lakehouse platforms, preferably within banking, financial services, or another highly regulated industry.Proven experience across one or more leading data and cloud platforms such as:Databricks, Snowflake, Cloudera, Azure, AWS, Google Cloud Platform, Huawei Cloud, or Alibaba Cloud.Strong experience designing distributed compute and MPP workloads across on-premise, hybrid, and cloud environments.Deep understanding of enterprise data architecture, scalability, resilience, security, governance, and performance optimization.Core Lakehouse & Data Architecture SkillsStrong experience in several of the following areas:Open Table Formats: Apache Iceberg, Apache Hudi, Delta LakeObject Storage: Cloud and enterprise object storage, including hot/warm/cold tiering strategiesData Federation: Trino, Denodo, DremioDistributed Query Technologies: Hive, Impala, Apache Kudu and similar platformsData Processing: Spark, PySpark, SQL, Java, Python, ScalaReal-Time & Streaming: Apache Kafka, Confluent, Azure Event Hubs, Amazon Kinesis, Apache Flink, Spark Streaming, Structured Streaming, Apache NiFiWorkflow & Scheduling: Airflow, Control-MData Modelling & Governance: Enterprise data modelling, metadata, lineage, data contracts, data quality, and governance frameworksGenerative AI, RAG & Agentic ArchitectureExperience designing or supporting modern AI-enabled data architectures, including:Retrieval-Augmented Generation (RAG)Embedding strategies and vectorizationVector databases and vector searchGraph databases and knowledge graphsPrompt and context managementAgentic workflow orchestrationKnowledge and semantic layersAI-driven analytics and Generative BIRelevant technologies may include:Vector Search / Vector DatabasesDatabricks Vector SearchAzure AI SearchPineconeChromaDBWeaviateSnowflake CortexGraph DatabasesNeo4jJanusGraphTigerGraphMicrosoft Fabric / Cosmos DBAmazon NeptuneStardogAgentic & AI Orchestration FrameworksLangGraphOpenAI Agents SDKMicrosoft Agent FrameworkLlamaIndex WorkflowsGoogle Agent Development Kit (ADK)Data Products & Data MarketplaceExperience designing and delivering foundation and business data products.Experience defining and implementing data contracts, service levels, governance, and quality controls.Ability to expose data products through:APIsPublish/subscribe and event-driven architecturesReal-time dashboardsBI and Generative BI platformsData marketplace capabilitiesExperience designing data products for enterprise consumption, reuse, discoverability, and governance.Cloud & Hybrid ArchitectureStrong understanding of cloud and hybrid architecture patterns, including:Workload placement and cloud optimization strategiesPrivate and dedicated cloud connectivity such as AWS Direct Connect and Azure ExpressRouteData egress and network cost optimizationInfrastructure-as-CodeHybrid and multi-cloud data architectureSecurity and network integrationHigh availability and disaster recoveryDevOps, Platform Engineering & AutomationExperience with modern DevOps and software delivery practices, including:CI/CD: Jenkins, Azure Pipelines, AWS CodePipeline, Google Cloud Build / DeploySource Control: Git, BitbucketCode Quality: SonarQubeArtifact Repositories: JFrog Artifactory, AWS CodeArtifact, Amazon ECR, Azure Artifacts, Google Artifact RegistryInfrastructure-as-Code: Terraform, AWS CloudFormation, Azure ARMContainerization: Docker, Kubernetes, OpenShiftDeployment: Helm, KustomizeMonitoring: AWS CloudWatch, Azure Monitor, Google Cloud MonitoringIncident / Service Management: Remedy or equivalent platformsTesting / Defect Management: JIRA, QuerySurge or similar toolsProgramming & AutomationStrong knowledge of one or more of the following:PythonScalaJavaSQLJavaScript / Node.jsShell scriptingGroovyExperience automating engineering and operational processes is strongly preferred.Migration & Modernization ExperienceExperience with migration and modernization programs involving legacy or MPP data platforms will be advantageous, including:TeradataGreenplumNetezzaOther enterprise MPP platformsExperience with bulk migration, workload modernization, automated migration tooling, and AI-assisted migration accelerators is a plus.EducationBachelor’s degree in Computer Science, Engineering, Information Technology, or a related discipline.Equivalent relevant professional experience may also be considered.Preferred CertificationsCandidates with relevant architecture, data, and cloud certifications will have an advantage. Examples include:Databricks Certified Data Engineer / Data ArchitectMicrosoft Azure certificationsAWS Cloud / Data certificationsGoogle Cloud certificationsDAMA Certified Data Management Professional (CDMP)Data modelling certifications such as ErwinRelevant Kubernetes, DevOps, data engineering, or architecture certificationsWhat Will Help You SucceedStrong architectural thinking with the ability to balance business outcomes, engineering quality, cost, scalability, security, and operational requirements.Ability to understand enterprise-wide technology landscapes and translate them into practical technical roadmaps.Strong analytical, troubleshooting, and decision-making capabilities.Ability to resolve complex architecture and integration challenges.Strong focus on engineering quality and continuous improvement.Excellent communication skills, including the ability to explain complex technical concepts to non-technical stakeholders.Strong stakeholder management and collaboration skills across business, technology, vendors, and distributed engineering teams.Experience working in Agile and modern software delivery environments.Ability to manage multiple initiatives and priorities in a fast-paced enterprise environment.Key Technology StackLakehouse & Data Platforms:Databricks, Snowflake, Cloudera, Iceberg, Hudi, Delta Lake, Trino, Denodo, Dremio, Hive, Impala, KuduCloud:Azure, AWS, GCP, Huawei Cloud, Alibaba CloudData Processing & Streaming:Spark, PySpark, Python, Scala, Java, SQL, Kafka, Confluent, Flink, Spark Streaming, Structured Streaming, NiFiAI / GenAI:RAG, Vector Search, Embeddings, Graph Databases, Knowledge Graphs, LangGraph, OpenAI Agents SDK, LlamaIndex, Microsoft Agent Framework, Google ADKDevOps & Platform Engineering:Kubernetes, OpenShift, Docker, Terraform, Helm, Kustomize, Jenkins, Git, SonarQube, CI/CDData Architecture & Governance:Data Products, Data Marketplace, Data Contracts, Data Quality, Metadata, Lineage, Data Modelling, Governance

Apply on mycareersfuture →
AI Project Team Management Application Architecture AI Agents Scalability Demonstrate resilience Security Data Quality Standards