Staff Data Scientist, Launch Evaluation Quality

Rigorous behavioral evaluation of the Waymo Driver is a critical part of scaling our ride hailing service and achieving Waymo’s ambitious goals. In this role, you will lead key initiatives for measuring the quality and trustworthiness of the behavior evaluation portfolio at Waymo, continuously identify and drive opportunities for improving the evaluation and testing methodologies, and partner closely with product and engineering teams to develop scalable solutions for monitoring and enhancing the health of the evaluation ecosystem. You will play a key role to ensure the behavior evaluation at Waymo is efficient, scientifically rigorous, and supporting Waymo’s growth priorities.

In this hybrid role you will report to the DS Lead for Pre-Release Evaluation.

You will:

• Lead a roadmap to develop methodologies and Key Performance Indicators to measure the quality, cost efficiency, and business relevance for various behavior evaluation solutions at Waymo.

• Directly influence the portfolio of evaluation signals relied on by hundreds of software engineers to ensure their changes in the Waymo Driver are having positive intended impacts.

• Conduct deep dive analysis to understand performance bottlenecks in current evaluation methodologies, propose and prototype improvements in metrics, sampling strategy, statistical inference, etc.

• Become an expert in Waymo’s evaluation ecosystem for onboard software. Understand how a broad variety of evaluation signals affect Waymo’s launch decisions, develop strategies for evaluating trade-offs, and provide actionable insights for designing launch criteria.

• Collaborate with product and engineering teams to build solutions for assessing Waymo’s evaluation portfolio at scale and accelerate the development for new evaluation.

• Collaborate with owners of various evaluation solutions, develop feedback loops to surface insights on evaluation quality to stakeholders and drive actionable plans for continuous improvement.

You have:

• Degree in a quantitative field (e.g. Statistics, Mathematics, Physics)

• Either a PhD in a quantitative field and 5+ years of industry experience, or 7+ years of industry experience solving data science problems

• Solid statistical background. Expertise using advanced statistical methods in an applied setting (e.g., experimentation design, causal inference, etc); familiarity with ML systems/models

• Demonstrated knowledge of data analysis libraries and packages in Python, R, and/or SQL

• Strong communication/leadership to address a high surface area across different organizations and products

• Ability to navigate through ambiguous business requirements and propose a clear technical solution

We prefer:

• PhD in a quantitative field

• Experience developing production-grade software and tools

• Experience as the technical lead for a broad data science area. A demonstrated track record of independently driving data science projects to deliver business value

• Experience with large-scale evaluation frameworks or experimentation/data platform for software development and machine learning systems

• Experience solving problems related to Autonomous Driving or Ride Hailing