Required Skills:
Very good knowledge in working with Azure Databricks services for data engineer
• Creating Azure Databricks services
• Reading & writing data from/to Azure Blob storage & Azure Synapse Gen2 storage using Azure Databricks services after receiving in different formats (e.g. csv, Parquet, Jason etc.) from Snowflake DWH platform
• Hands-on experience in working with Databricks File system (DBFS)
• Experience in working with RDD, Dataframe, Azure SQL database
• Experience in constructing ingestion, transformation & egree pipelines using Azure Data Factory
• Experience in developing application service using Pyspark & Spark SQL
