Position: Lead Software Engineer - Data (Ai & Nlp)
Founded and funded by Temasek,
• * Splore
• * is an early-stage venture focused on reimagining search and discovery for a Metaverse Generation. The Metaverse Generation refers to online-first users who value immersive digital experiences, utilize AI and other technologies in novel ways, and interact within online communities.
- Our mission is to enable users creating and consuming content in this new world, like gamers, to find and discover content beyond the confines of traditional search._
We are seeking a talented and experienced Lead/Senior Software Engineer to join our Search Engine AI Team.
You will be responsible for designing, developing, and implementing data-driven solutions utilizing artificial intelligence and machine learning to solve complex problems.
You will work closely with cross-functional teams, including product managers, AI Teams, and UX designers, to build best-in-class Search Engine experiences for Gamers.
• * In this role, you’ll need to**:
- Utilize your deep understanding of data technologies, including databases, data warehousing, data pipelines, and data processing frameworks, to design and develop robust and scalable data solutions
- Work with various data technologies, such as SQL and No
SQL databases, to efficiently store and retrieve large volumes of structured and unstructured data
- Leverage data processing frameworks like Hadoop, Spark, and Kafka to analyze and process data at scale, ensuring high performance and reliability
- Design and implement data pipelines to efficiently and reliably move and transform data between systems and components of the data infrastructure
- Optimize data processing and query performance through indexing, partitioning, and other performance-tuning techniques
- Implement data quality checks and ensure data governance practices are followed to maintain data integrity and compliance
- Implement appropriate data security measures and access controls to protect sensitive data
- Collaborate with cross-functional teams, including data scientists, data analysts, and other engineers, to understand requirements and deliver high-quality data solutions
- Identify and resolve data-related issues, performance bottlenecks, and other technical problems through effective debugging and troubleshooting techniques
- Stay updated with emerging data technologies and industry trends. Continuously enhance your skills and knowledge to drive innovation and efficiency in data engineering
- Bachelor's or Master's degree in computer science, Information Technology, or a related field
- 8 to 10 years of relevant experience as a Software Engineer, with a focus on data technologies
- Deep understanding of data technologies, including databases, data warehousing, data pipelines, and data processing frameworks
- Experience and/or familiarity with a range of data technologies, such as SQL, No
SQL, Hadoop, Spark, Kafka, Air Byte, Airflow, Data Hub, Great Expectations, DBT, and Elasticsearch
- Proficiency in programming languages commonly used in data engineering, such as Python, Java, or Scala
- Experience in designing and developing data pipelines, data processing, and data integration solutions
- Strong knowledge of data modeling and database design principles
- Experience setting up end-to-end data infrastructure like a data lake with a focus on data quality monitoring and data lineage tracking
- Familiarity with data warehousing concepts and technologies
- Strong problem-solving and analytical skills
It will be good to have:
- Ownership mindset:
Reliable, takes pride in the quality of their work.
- Hunger: A desire to explore and raise the bar in building search experiences and eagerness to embrace the new and unknown
- Communicator:
Skills to communicate with stakeholders on engineering, product, business, and client teams
