Role: Big Data/ Hadoop Engineer
Location: San Francisco, CA
Duration: 4 Months +
Looking for candidates with at least 10 years of experience
This is a very Senior role and Client is Looking for full Hadoop Ecosystem experience and Java experience to help create the big data platforms.
As part of the technology transformation, we have also embarked on a journey for enabling data driven decision culture at StubHub and started transitioning to and innovating on a Hadoop eco- system based data platform that meets our both online as well as offline use case challenges.
We are looking for a SENIOR BIG DATA PLATFORM ENGINEER WITH JAVA EXPERTISE to help build our next generation data platform. This highly motivated individual needs to be a self-starter with hands on in relevant experience (must). Also, the candidate expected to be working agile environment with competing priorities and expect to learn new technologies part of the delivery. This is an excellent opportunity for the right individual to have a significant impact on the organization. Specific responsibilities include:
Primary Skills:
Language: Java, Scala, Python,
Enterprise Software Development Exposure: (Eclipse, GitHub, Test-Driven Development and Server Side Framework Programing)
Big Data (Data Science & Machine Learning): Hadoop, HDFS, Spark, Hive, Pig, Oozie, Hbase,
ToolKits: Datameer, NLP, TensorFlow
Database: Oracle or Similar
Experience:
Big Data:
o Data Science, Machine Learning, Text Mining & Natural Language Processing Framework.
o Enterprise Data Processing - Extract meaningful data from structured, RDBMS, text and unstructured data.
o Experience in writing, scheduling, debugging Pig, Hive, and Spark jobs at scale.
o high-volume real-time data ingestion frameworks and automate ingesting various data sources into Hadoop.
o Research, develop, Optimize and Innovate frameworks and related components for enterprise scale data analysis and computations.
o Develop data validation frameworks, proactive monitoring solutions to detect data ingestion failures in big data platform and take appropriate remedies.
Machine Learning:
o Design, build, deploy Machine Learning applications to solve real-world problems empirically
o Work with any kind of practical data, including Image, Audio, Text, Video, Motion Capture & other high dimensional data
o Established track record of successfully employing novel Machine Learning (ML) approaches to automatically discover insights from high-volume, high-dimensional temporal data (e.g., log records from computer systems, feeds from social media, dynamics of neural networks or protein folding); designing interactive visualization tools for data/system analysis; and leveraging the gained insights to automatically determine or guide management policies for complex systems
o Deep expertise in Artificial Intelligence (AI) (neuromorphic computing, reinforcement learning, discrete optimization, graph analytics, human-computer interaction) and its applications in Game theory (strategic reasoning, multi-agent systems, incentive-design, auctions, e-commerce)
Collaborate with people working on various technologies and ensure consistency for the data exposed through these different channels.
Ownership of the end-to-end development life cycle with high quality of solution/code you develop and evangelize the test driven development - (tests, code coverage, etc.)
Minimum 8+ years of experience in requirements analysis, design, development and testing of distributed, enterprise-class applications/platforms with particular attention to scalability and high performance, with demonstrable experience
Knowledge and experience with RDBMS, O-R mapping, and application of distributed caching technologies
