Databricks
About this role
Databricks customers create enormous numbers of assets on the platform: tables, notebooks, dashboards, pipelines, SQL queries and models, sometimes hundreds of millions of them. Finding the right one is a core user journey, and this role leads the quality of that search. The work is search ranking, query understanding and rewriting, retrieval, and building the evaluation framework that proves a ranking change actually improved things offline and online. There is explicit LLM applied to search relevance in the requirement list. Ten years of building search relevance systems at scale is the bar, either in production or in high impact research environments, and a Master's or PhD is preferred though a Bachelor's is the stated minimum. Bengaluru.
Who this is for
What the posting asks for:
- BS or above in computer science or a related field. Master's or PhD preferred.
- 10+ years developing search relevance systems at scale, either in production or in high impact research environments.
- Experience applying LLMs to search relevance.
- Experience in one or more of: query understanding, NLP, text mining, recommendations, personalisation, discovery, or conversational AI.
- Strong computer science fundamentals.
- Contributions to well used open source projects.
Day to day: driving the development and deployment of ML based search and discovery relevance models integrated into Databricks products; designing automated ML and NLP pipelines covering data preprocessing, query understanding and rewriting, ranking and retrieval, and model evaluation so the team can iterate quickly; working with product managers on the search and discovery roadmap; and building a robust framework for evaluating ranking improvements both offline and online.
The problem in context: customers can hold hundreds of millions of assets on the platform, so this is a genuine large scale retrieval problem rather than a site search feature.
Location and office reality: Bengaluru. The posting does not publish an office day policy.
Honest fit guidance: ten years specifically in search relevance is narrow, and the open source contribution line is listed as a requirement rather than a nice to have. General ML engineering at ten years is not the same thing as a decade of ranking and retrieval work, and this interview will find the difference.
- BS or above in computer science or a related field. Master's or PhD preferred.
- 10+ years developing search relevance systems at scale, either in production or in high impact research environments.
- Experience applying LLMs to search relevance.
- Experience in one or more of: query understanding, NLP, text mining, recommendations, personalisation, discovery, or conversational AI.
- Strong computer science fundamentals.
- Contributions to well used open source projects.
Day to day: driving the development and deployment of ML based search and discovery relevance models integrated into Databricks products; designing automated ML and NLP pipelines covering data preprocessing, query understanding and rewriting, ranking and retrieval, and model evaluation so the team can iterate quickly; working with product managers on the search and discovery roadmap; and building a robust framework for evaluating ranking improvements both offline and online.
The problem in context: customers can hold hundreds of millions of assets on the platform, so this is a genuine large scale retrieval problem rather than a site search feature.
Location and office reality: Bengaluru. The posting does not publish an office day policy.
Honest fit guidance: ten years specifically in search relevance is narrow, and the open source contribution line is listed as a requirement rather than a nice to have. General ML engineering at ten years is not the same thing as a decade of ranking and retrieval work, and this interview will find the difference.
Apply on company site
Opens databricks.com, the employer's own application page. Applying is always free.