Key Deliverables:
• Build and optimize scalable data pipelines using Apache Spark on Databricks
• Implement ETL/ELT processes with Databricks Notebooks and Delta Lake
• Ensure data quality, integrity, and compliance across platforms
• Support CI/CD, cloud migrations, and infrastructure automation
Role Responsibilities:
• Collaborate with data scientists and analysts to deliver data products
• Maintain and optimize large-scale distributed data jobs
• Use Azure DevOps, GitHub Actions, and Terraform for workflow automation
• Work with structured and unstructured data using Lakehouse architecture
