We are seeking a skilled Databricks Data Architect to drive our clients data architecture strategy, design, and implementation on the Databricks platform. The successful candidate will play a pivotal role in building and maintaining robust, scalable, and high-performance data solutions that support our business's strategic objectives. 9 months contract with a view to extend on Sydney's north shore.
Responsibilities:
- Develop and maintain the organization's Databricks-based data architecture by understanding business needs, current systems, and data movements.
- Collaborate with stakeholders, data engineers, and business analysts to gather requirements and translate them into technical solutions on the Databricks platform.
- Design and implement scalable data models, data pipelines, and ETL processes using Databricks to support various analytical and operational requirements.
- Optimize and maintain the performance and stability of data solutions on the Databricks platform ensuring reliability, efficiency, and security.
- Provide expertise in Databricks best practices, configuration, and tuning to maximize the platform's capabilities and benefits.
- Lead and mentor a team of data engineers, providing guidance and knowledge transfer on Databricks architecture and development best practices.
- Stay current with industry trends, emerging technologies, and best practices in data engineering and architecture, and leverage this knowledge to continually improve our data solutions.
Ideally you will have:
- Bachelor's degree in computer science, information technology, or a related field. A Master's degree is a plus.
- Proven experience as a Data Architect, including at least 3 years of hands-on experience with Databricks architecture and development.
- Expertise in building and optimizing data pipelines and ETL processes using Databricks, Apache Spark, and related technologies.
- Strong understanding of cloud-based data platforms, particularly Azure, and proficiency in leveraging Databricks in these environments.
- In-depth knowledge of big data technologies, data warehousing, and data modeling concepts, as well as experience with structured and unstructured data formats.
- Proficiency in programming and scripting languages such as Python, SQL, Scala, or Java, with a focus on data manipulation and processing.
- Experience in Azure data factory and Power BI also preferred.
- Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams and convey complex technical concepts to non-technical stakeholders.
- Proven leadership and mentoring skills, with the ability to guide and develop a team of data engineers in implementing Databricks data solutions.
Apply below or for more details call Mark on 0438 198 849
