N° Référence: MM-BBBH12697
Présentation société Voir les autres offres
   
 

MyNextCompany - Data Engineer

Réf.: MM-BBBH12697

We are looking for a Data Engineer or Data Scientist for a 6 month mission with extensions in Brussels. You will be developping a platform that will centralise data on energy efficiency.

 

 

Technical context

 

The platform is set up on Amazon Web Services and uses various PaaS and IaaS solutions. The data lake / Data Warehouse that is used for all kinds of reporting and analysis purposes uses the following Amazon Web services

  • Data pipelines and EMR for loading and processing data.
  • S3 for saving files
  • Amazon Redshift (DWH)
  • Tableau as a reporting environment
  • Athena for querying flat files in the data lake
  • SQL Server as backend database of the Web application
  • Lambda for running backend services
  • DMS for synchronization of data e.g. from SQL Server to redshift

For example, for processing the consumption data that is supplied on a daily basis, Python and Spark is used as an ETL process to merge the new consumption data in the Redshift DWH with already existing data.

 

In addition, links are made with numerous other data sources to enrich the Data Warehouse.

 

GIT is used to Version Control the code and the release process (dev, stg, prod) uses MS DevOps. The latter is also used for following up sprints, user stories, tasks, ...

 

 

Your profile

  • You manage the existing stack of data flows and pipelines in the AWS cloud environment that uses S3, EMR, Spark, Redshift, Lambda, Python, among others.
  • You will (further) build complex data models to generate further analytical insight
  • You write high-quality code to further expand the data platform. The platform is scalable and easy to maintain.
  • You will link / integrate new data sources from various origins into the existing solution.
  • You analyze existing processes and advise where performance gains and cost savings can be achieved.
  • You implement data quality controls and alert stakeholders so that the necessary actions can be taken.
  • All projects are developed according to the Agile Scrum methodology. As a data engineer you work closely with the other members of the multidisciplinary team.
  • You report to the Project Manager of the team.

 

Interested to know more?

 

Then apply with your latest CV via the button below!

Description société

Recruiting people is as crucial as putting together a crew for a space mission. With the right team, the possibilities are endless.

At MyNextCompany, we go further than traditional recruitment. To efficiently match talents and companies we use Smart Recruiting®, an innovative technology we set up for efficient and transparent recruitment.

Smart Recruiting® is a combination of Employer Branding and Inbound Recruitment, and it offers candidates greater transparency on employers we partner with.

MyNextCompany. Aim high. Explore more.

Lire la suite
Offres d'emploi similaires
Offres similaires basées sur les critères : (Big) Data Expert IaaS / PaaS / SaaS Amazon Web Services (AWS) Cloud