Duties
• Design, develop, and optimize Extract, Transform, Load (ETL) processes to facilitate efficient data movement across various platforms including cloud environments such as AWS and Azure Data Lake.
• Collaborate with cross-functional teams to gather requirements and translate them into scalable data workflows using tools like Informatica, Talend, and Apache Hive.
• Build and maintain data pipelines leveraging big data frameworks such as Hadoop, Spark, and Apache Hive to handle large-scale datasets effectively.
• Develop and implement database schemas, models, and designs for data warehouses using Microsoft SQL Server, Oracle, and other relational databases.
• Integrate linked data sources through RESTful APIs and ensure interoperability across diverse systems.
• Write complex SQL queries, Python scripts, Bash shell scripts, and VBA macros to automate processes and enhance data analysis capabilities.
• Monitor pipeline performance, troubleshoot issues promptly, and refine processes for optimal efficiency.
• Support model training activities by preparing datasets and ensuring the integrity of data used in analytics projects.
• Participate in Agile development cycles to deliver iterative improvements aligned with project goals.
Skills
• Strong proficiency in SQL programming for data extraction and manipulation across multiple database platforms including Microsoft SQL Server and Oracle.
• Hands-on experience with ETL tools such as Informatica or Talend for building scalable data workflows.
• Familiarity with cloud-based storage solutions like AWS (Amazon Web Services), Azure Data Lake, and related services for big data processing.
• Expertise in big data technologies including Hadoop ecosystem components (HDFS), Spark, Apache Hive, and related frameworks for processing large datasets.
• Knowledge of programming languages such as Java, Python, Bash (Unix shell), Shell Scripting, VBA for automation and custom development tasks.
• Experience working with Looker or similar BI tools for creating dashboards and visualizations that support analytics initiatives.
• Understanding of RESTful API integration techniques for linked data sourcing from external systems.
• Ability to design efficient database schemas and optimize query performance in complex environments.
• Familiarity with Agile methodologies to foster collaborative development cycles in fast-paced projects.
• Strong analysis skills to interpret large datasets accurately and derive actionable insights. Join us to be part of a forward-thinking team dedicated to transforming raw data into strategic assets! Your expertise will empower our organization’s decision-making processes while advancing your career in a vibrant technological environment focused on innovation and excellence.
Job Type: Full-time
Pay: $75,598.37-$123,084.89 per year
