Data Engineer

Data Engineer RESPONSIBILITY- Design, develop and deploy data tables, views and marts in datawarehouses, operational data store, data lake and data virtualization.- Perform data extraction, cleaning, transformation, and flow. Web scrapingmay be also a part of the work scope in data extraction.- Design, build, launch and maintain efficient and reliable large-scale batchand real-time data pipelines with data processing frameworks.- Integrate and collate data silos in a manner which is both scalable andcompliant.- Collaborate with Project Manager, Data Architect, Business Analysts,Frontend Developers, Designers and Data Analyst to build scalable datadriven products.- Be responsible for developing backend APIs & working on databases tosupport the applications.- Work in an Agile Environment that practices Continuous Integration andDelivery.- Work closely with fellow developers through pair programming and codereview process.EXPERIENCE AND SKILLS NEEDED- Proficient in general data cleaning and transformation (e.g. SQL,pandas,R, etc) to ensure data accuracy and consistency.- Proficient in building ETL pipeline (eg. SQL Server Integration Services(SSIS), AWS Database Migration Services (DMS), Python, AWS Lambda,ECS Container task, Eventbridge, AWS Glue, Spring).- Proficient in database design and various databases (e.g. SQL,PostgreSQL, AWS S3, Athena, mongodb, postgres/gis, mysql, sqlite,voltdb, cassandra, etc).- Experience in cloud technologies such as GPC, GCC (i.e. AWS, Azure,Google Cloud).- Experience and passion for data engineering in a big data environmentusing Cloud platforms such as GPC, GCC (i.e. AWS, Azure, Google Cloud).- Experience with building production-grade data pipelines, ETL/ELT dataintegration.- Knowledge about system design, data structure and algorithms.- Familiar with data modelling, data access, and data storage infrastructurelike Data Mart, Data Lake, Data Virtualisation and Data Warehouse forefficient storage and retrieval.- Familiar with rest api and web requests/protocols in general.- Familiar with big data frameworks and tools (eg. Hadoop, Spark,Kafka,RabbitMQ).- Familiar with W3C Document Object Model and customized web scraping(e.g. BeautifulSoup, CasperJS, PhantomJS, Selenium, Nodejs, etc).- Familiar with data governance policies, access control and security bestpractices.- Comfortable in at least one scripting language (eg. SQL,Python).- Comfortable in both windows and linux development environments.When you apply, you voluntarily consent to the disclosure, collection and use of your personal data for employment/recruitment and related purposes in accordance with the Tech Aalto Privacy Policy, a copy of which is published at Tech Aalto’s website (https://www.techaalto.com/privacy/)Confidentiality is assured, and only shortlisted candidates will be notified for interviews.Tech Aalto Pte Ltd | 24S2130 EA |Pushpanjli Kir | R1657306.