Big Data Platform Architect

San Francisco 25 days agoFull-time External
Negotiable
The mission of our team (UIP & BDI - Unified Intelligence Platform & Big Data Infrastructure) is to make using data at scale easy and enabling, by marrying together cutting-edge technology with an exceptional user experience. Massive amounts of data are generated each day at Salesforce. Crafting a platform that enables our users to understand, optimize, and re-envision how the business is and should operate is critical. Modern distributed systems (e.g. Spark, Presto, & Airflow), coupled with AI tech (e.g. Agents and Jupyter), and custom data applications (e.g. a high-scale data ingestion service and a comprehensive data catalog) are a few of the ingredients this team weaves together to make this possible. We are looking for a technical ace and thought leader in the big data space to join the team. You will be working with exceptional peers that you can teach and learn from. This is a rare opportunity to leverage and extend your expertise, as we address internal-facing and external-facing big data platform needs. This is just a small sample of the interesting challenges ahead. We are constantly extending and innovating our platform. Some current areas of focus include: • How can we ensure we provide highly compelling, “first class” big data offerings (e.g. easy to use, highly enabling, and well integrated with Salesforce’s centralized infrastructure / tooling)? • How can we efficiently and effectively support our big data infrastructure within and across the major cloud providers? • How can we strike the right balance of isolation and cost efficiency for the hundreds of teams and thousands of users that leverage our data analytics platform? • How should we weave in agents to ease and accelerate platform usage and support? • How can we ensure we serve as the de facto destination for internal analytics and as a turnkey data refinery for downstream systems? • How can we best implement and extend the Iceberg Rest Catalog and related table management services? • How should we gather, distill, and present data lineage information? Required Skills / About you: • Extensive (big) data platform architecting & development experience • Extensive experience working in the public cloud • Ability to transform high-level needs into well-thought-out system designs • Thrive on taking full ownership of projects, driving them forward to completion • Exceptional software development skills (e.g. Python, Java, and / or Scala) • Love getting things done (e.g. hands-on prototyping a new approach) • Expert problem solver • Great communicator • Good listener / curious / adaptable / collaborative • Ability to lead and influence across all levels of the organization • Proven experience thinking outside the box and solving tough strategic challenges • LI-Y