Lead Azure Databricks / Data Engineer – New Data Lakehouse Build
We are hiring for a Lead Azure Databricks / Data Engineer with 5 years experience in the Azure Databricks space, and 1-2 years experience as a Lead Data Engineer where you have been focussed on developing data pipelines, data warehouses, data lakehouses, and similar data repositories for BI, Reporting and Analytics use.
Our customer is building a new data lakehouse which will see you work closely with four other slightly more junior Data Engineers, to work to the already approved strategy, ensuring that the solution is scalable and that the correct monitoring is in place. You will be the ‘go-to’ person for the business for all things datalake focussed, therefore, you will have a good solid technical understanding of this together with a demonstrable background of building solutions in the commercial world.
You will need to have expertise in the following areas to be considered; Solid SQL Skills, Solid Python Skills (able to use the Python programming concepts in Databricks Notebooks for data loading, transformation and exploration including familiarity with Pandas), as well as a strong understanding of core Databricks and Spark concepts such as Delta Lake, Medallion Architecture, Dataframes, Workflows, Unity Catalogue, Delta Live Tables, and UDF’s. You will also be experienced with ADF, ADLS Gen2, and the Azure Cloud Ecosystem
This role is a remote position however, on occasions when the team get together you will be required to attend these meetings.
Salary: up to £80,000 Per Annum DOE
Location: Remote – UK/Europe
