Responsibilities
• Design, develop, and optimize large-scale data pipelines and ETL processes to efficiently extract, transform, and load data from various sources including cloud platforms like AWS and Azure Data Lake.
• Build and maintain scalable data warehouses utilizing technologies such as SQL Server, Oracle, Hadoop, Apache Hive, Spark, and Big Data frameworks to support analytics and reporting needs.
• Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions using Python, Java, Shell Scripting, Bash (Unix shell), Talend, Informatica, and RESTful APIs.
• Implement data models and database designs that ensure high performance, reliability, and security for enterprise data systems.
• Develop automated workflows for model training, analysis tasks, and data validation to streamline operations.
• Integrate linked data sources to enhance data richness and enable comprehensive analysis across disparate datasets.
• Utilize analytics tools such as Looker for visualization and reporting to deliver actionable insights to stakeholders.
• Participate in Agile development cycles to continuously improve data infrastructure while ensuring compliance with best practices in database management.
• Conduct troubleshooting of complex data issues and perform performance tuning on large datasets.
• Document processes thoroughly to ensure maintainability and knowledge sharing within the team.
Qualifications
• Proven experience designing and implementing ETL workflows using tools like Talend, Informatica or custom scripting languages such as Python or Shell Scripting.
• Strong proficiency in SQL (Microsoft SQL Server, Oracle) along with expertise in database design principles for building efficient Data Warehouses.
• Hands-on experience working with Big Data technologies including Hadoop ecosystem (HDFS), Apache Hive, Spark, and Azure Data Lake.
• Familiarity with cloud platforms such as AWS (Amazon Web Services) and Azure for deploying scalable data solutions.
• Knowledge of programming languages including Java and Python for developing robust data processing applications.
• Experience working with RESTful APIs for integrating external systems or services into the data architecture.
• Ability to analyze complex datasets using advanced analysis skills to identify trends or anomalies that inform business strategies.
• Understanding of linked data concepts to connect diverse datasets effectively.
• Skills in database modeling, schema design, and optimization techniques to support high-performance environments.
• Experience with analysis tools like Looker or similar BI platforms for creating dashboards and reports.
• Familiarity with Shell Scripting (Bash), VBA is a plus for automating tasks or integrating legacy systems.
• Knowledge of Agile methodologies to collaborate effectively within fast-moving project teams. Join us if you're eager to leverage your technical expertise in a vibrant environment where innovation meets impact!
Job Type: Full-time
Pay: $67,850.26-$144,698.27 per year
