My client from retail industry is looking for a data engineer for migrating data pipelines over cloud platform.
Responsibilities:
• Migrate data pipelines from Informatica to Databricks, ensuring seamless integration and efficient data processing.
• Design, develop, and optimize data processing workflows using SQL, PL/SQL, or python.
• Implement and enforce data governance and security measures to protect sensitive data and ensure compliance.
• Establish and maintain data lineage documentation to track the origin, transformations, and destinations of data. Collaborate with cross-functional teams to identify data requirements and implement optimal solutions.
• Monitor and troubleshoot data pipelines to ensure smooth and uninterrupted data flow.
• Conduct testing and validation of data pipelines to ensure accuracy, completeness, and data quality.
• Optimize data storage, retrieval, and archival processes in collaboration with database administrators and infrastructure teams.
• Stay updated with industry trends and advancements in data engineering, big data technologies, and cloud computing.
Qualifications:
• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in migrating data pipelines from Informatica to Databricks.
• Strong proficiency in Oracle 12c, ADW, MSSQL, and MYSQL.
• Proficient in programming languages such as Python, Scala, or SQL for data processing and transformation.
• Solid understanding of data governance and security principles.
• Experience in maintaining data lineage documentation.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration skills.
• Ability to adapt to changing priorities and work effectively in a fast-paced environment.
My client from retail industry is looking for a data engineer for migrating data pipelines over cloud platform
