Data Engineer
Kopi Recruit · Singapore
Job DescriptionWe are seeking a Data Engineer to join our data team at a government agency. The rolefocuses on designing and building data models, curated datasets and pipelines to supportbusiness intelligence, analytics and data science use cases.You will work directly with business stakeholders to understand data requirements, translatebusiness needs into technical solutions, and deliver reliable, well-structured and reusabledata products.Our technology stack is primarily built on Databricks and AWS.Key ResponsibilitiesEngage directly with business stakeholders to understand requirements and translatethem into data models, datasets and pipelines.Design and develop data models and transformation pipelines to integrate, clean,harmonise and curate data from multiple source systems.Build and maintain scalable, reliable and reusable data pipelines using Databricks, Sparkand SQL.Design datasets and data structures optimised for downstream reporting, analytics anddata science applications.Implement data quality checks, validation and reconciliation to ensure data accuracy andconsistency.Monitor and troubleshoot pipeline performance and operational issues.Document data models, datasets, transformations and pipeline dependencies.Work with data engineers, analysts and data scientists to deliver data solutions alignedwith business needs.RequirementsStrong proficiency in SQL and experience with relational databases and datawarehousing.Strong experience in data modelling, including dimensional modelling and designinganalytical datasets.Proficiency in Python for data processing and pipeline development.Hands-on experience with Databricks, Apache Spark and Delta Lake.Experience designing and developing ETL/ELT pipelines and data transformationworkflows.Experience working with AWS data services and cloud-based data platforms.Ability to engage business stakeholders, understand business processes and translaterequirements into technical data solutions.Familiarity with Git, CI/CD and data pipeline orchestration.Preferred QualificationsExperience with Databricks Workflows, Unity Catalog and Delta Lake.Experience designing data products for business intelligence and self-service analytics.Familiarity with data governance, access control and data quality practices.Experience working in a government or other highly regulated environment.Degree in Computer Science, Information Technology, Data Engineering or a relateddiscipline.