AI Data Engineer & Analyst

OverviewWe are seeking a highly skilled AI Data Engineer & Analyst to lead the design and implementation of an AI-powered strategic account intelligence platform. This is a greenfield build within the Microsoft ecosystem (Microsoft Fabric, OneLake, Azure AI, Power BI, Purview, Entra ID), delivering an end-to-end solution from data ingestion and semantic modeling through to AI-driven recommendations and executive-ready dashboards.This role is ideal for a hands-on technical professional who thrives at the intersection of data engineering, machine learning, and business intelligence — someone who can architect a robust data platform and bring it to life with actionable AI-powered insights.You will work in close partnership with an external Microsoft-certified consultancy engaged to support the full project rollout. The consultancy will provide expert guidance and hands-on support across all project phases, from strategy and planning through infrastructure evaluation and technical execution, ensuring the platform is built to enterprise standards and aligned with Microsoft best practices.Key ResponsibilitiesData Architecture & Engineering (40%)Design and implement the canonical object model in Microsoft Fabric / OneLake (Lakehouse / Data Warehouse)Build and maintain automated ingestion pipelines using Data Factory in Fabric / Azure Data FactoryImplement identity resolution across disparate source systems (deterministic matching on shared keys + probabilistic/ML-based entity resolution)Create golden records with source-confidence scoring per attribute and conflict resolution rulesEnsure schema validation, data quality checks, and attribute-level completeness thresholds across all pipelinesDesign for scalability to hundreds of accounts with near-real-time query performanceAI / Machine Learning (30%)Develop the opportunity scoring model using Azure AI Services / Azure Machine Learning / Fabric Data ScienceBuild NLP-based news signal classification and entity extraction from external feedsImplement the NBA recommendation engine with explainable evidence chains linking scores to underlying data points, signals, and model factorsDesign and implement decision-lineage capture and the continuous learning feedback loopProduce ranked recommendations with confidence intervalsPlan for quarterly model review cycles assessing recommendation quality, bias, and driftBusiness Intelligence & Visualization (20%)Develop interactive Power BI dashboards in Direct Lake mode over OneLakeBuild account-level opportunity score dashboards with drill-through to evidence chainsCreate tracking and reporting dashboards for recommendation outcomes, acceptance rates, pipeline impact, and model performanceGenerate auto-formatted sales-ready outputs (one-page account briefs, exec summaries, talking points)Governance, Security & Collaboration (10%)Implement Role-Based Access Control (RBAC) via Microsoft Entra ID with object-level permissionsConfigure data lineage tracking through Microsoft Purview (Data Map, Unified Catalog)Ensure full audit logging of data access, recommendation generation, and user actionsCollaborate with business stakeholders (Account Directors, Sales Leaders, Strategic Planning teams) to validate data models, refine scoring logic, and iterate on outputsPartner with an external Microsoft-certified consultancy providing end-to-end project support, from strategy and infrastructure planning through to technical execution, leveraging their specialist Microsoft expertise to accelerate delivery, validate architectural decisions, and ensure best-practice implementation across the full Microsoft stackDocument architecture decisions, data flows, and operational runbooksRequired QualificationsBachelor's degree in Computer Science, Data Science, Information Systems, Statistics, or a related technical fieldExperience:5+ years of professional experience in data engineering, data analytics, or business intelligence3+ years working within the Microsoft data ecosystem (Azure / Fabric / Power BI)Demonstrated experience building end-to-end data platforms from ingestion through modeling to dashboardsProven track record with AI/ML model development and deployment in a production or near-production environmentExperience with entity resolution / identity matching across multiple data sourcesTechnical Skills (Required): Data PlatformMicrosoft Fabric (OneLake, Lakehouse, Data Warehouse, Data Factory, Fabric Data Science)AI / MLAzure AI Services, Azure Machine Learning, model training & scoring pipelinesBI & DashboardsPower BI (Direct Lake mode, DAX, advanced data modeling, report design)Data GovernanceMicrosoft Purview (Data Map, Unified Catalog, Lineage), source-confidence scoringIdentity & AccessMicrosoft Entra ID (RBAC, object-level permissions)ProgrammingPython (pandas, scikit-learn, or equivalent ML frameworks), SQL (T-SQL / Spark SQL)Data FormatsDelta Lake, Parquet, JSONData ModelingCanonical / semantic data modeling, star schema, entity-relationship design