Description
You will contribute to the development of our data pipelines within our Big Data platform. You will join an Agile team with the mentality of a startup. In addition to the technical development of pipelines, you can also understand how to extract value from these pipelines.
You can retrieve data. You can first gain exploratory insights and build production-value models. Open Constructive content and technical adjustments are in your nature. So we expect more than just technical implementation.
You have experience with technical integrations and are able to think and build (API, event based, pipeline sheduling and transaction handling...). You think along functionally. You are at least 4/5 available to work on-site in Mechelen.
Context
- Design, build and deploy a set of batch data applications in Spark / Azure batch.
- Knowledge of data science and machine learning algorithms so you can ensure that your data deliveries are ideal for building models.
- Experience in workflow mgmt and job scheduling of data loads (airflow, Azure data factory)
- Experience in real time data streaming and processing is a plus (Azure functions / event hubs / streaming analytics)
- Experience in cloud Azure is a plus. Overall cloud experience is a must
- Knowledge of ETL for large data sets on distributed systems
- Thorough knowledge in programming languages (Python, SQL, HCL, ...) and in exposing data (API, SQL Server, event, etc.).
- Experience with data exploration: Jupyter notebooks, dashboarding ... and extracting value from data. We expect to be able to identify business opportunities or service improvements through your own analyses and insights.
- You have the nerve to put forward new assumptions. You have demonstrable experience with business rules and experience with ETLs with aggregations, deduplication and linking to identities
- Experience in build automation and DevOps CI/CD tooling: GIT, TerraForm/ARM templates, Circle CI
- Experience with Web environments using Big Data intel/recommendations. JavaScript, Java, Node js, API low to exhibit data.
Requirements
- Proven experience in build automation and DevOps CI/CD tooling: GIT, TerraForm/ARM templates, Circle CI
- Proven experience in workflow mgmt and job scheduling of data loads (airflow, Azure data factory)
- Demonstrable experience with cloud is a must.
- Proven experience with data exploration and extracting value from data
- Proven experience in programming languages (Python, SQL, HCL, ...) and in exposing data (API, SQL Server, event streams, ...)
- Demonstrable experience with Web environments using Big Data intel/recommendations. Javascipt, Java, Node js, API layer to expose data.
- NL at a native level + ENG
