Role: Data QA
Location: Toronto, CA
Key Responsibilities:
• Develop and implement test strategies, test cases, and automation scripts to validate data pipelines in Azure and Databricks environments.
• Perform data validation, reconciliation, and comparative analysis between source and target systems.
• Validate ETL/ELT pipelines built using ADF and Databricks.
• Collaborate with Data Engineers and Product Owners to understand STM (Source-to-Target Mapping) and ensure transformation logic is correctly implemented.
• Monitor and validate data quality across Delta tables, and Data Warehouses.
• Identify data anomalies, document defects, and drive them to resolution with the engineering team.
• Support CI/CD pipelines by integrating data testing into DevOps workflows.
• Contribute to test data management, metadata validation, and regression testing.
• Provide regular reporting on test execution results, defect metrics, and QA health
Required Skills:
• Proven experience in data QA/validation in cloud-based data platforms.
• Strong knowledge of Azure Data Factory, Databricks.
• Proficiency in SQL and scripting languages such as Python.
• Hands-on experience with data profiling, data reconciliation, and schema validation.
• Understanding of SCD Type 2 and data transformation logic.
• Familiarity with DevOps tools like Azure DevOps or GitHub Actions for CI/CD integration.
• Experience working with large datasets, performance testing, and data lineage tools.
