Data Engineer
Role details
Job location
Tech stack
Job description
This will be an opportunity to join a newly established team which has the opportunity to work within a number of interesting projects including Data Science, Market Data and Pricing Transformation, Customer Data and Analytics. As this is a new role, you will be meeting with key stakeholders to gather requirements, design and implement the solutions, and lead the development life cycle., * Develop and maintain scalable data pipelines to power new initiative data products
- Use the latest data analytics and quality tools to ensure data meets strict data standards.
- Design and implement new data pipelines using best in class tools such as Airflow, DBT etc.
- Build a scalable cloud infrastructure on Azure using modern technologies such as terraform
- DevOps experience, such as using Git, CI/CD and Unit Testing
- Working on iterative/agile development methodologies such as SCRUM
- Bring your own idea to show how we can do things better.
Requirements
Do you have experience in Unit testing?, * Strong hands-on development experience with Python
- Strong SQL experience (Any No-SQL and/or Snowflake experience)
- Experience with DBT or Airflow
- Have experience with ETL, Analytical/Reporting tools
- Have cloud-based experience, preferably with Azure
- Big Data Frameworks experience, such as Hadoop and Apache Spark
- Experience of working within small teams or a startup environment
- Interest in working closely with Artificial Intelligence/Machine Learning