Databricks
About this role
This is a search quality and information retrieval seat at Databricks, and the framing is unusual because the search has two different consumers. One is human: employees looking for assets and answers. The other is AI agents that need a high precision context layer to reason over data the model was never trained on. The posting is direct that poor retrieval quality causes hallucinations, so part of the job is building the relevance scoring and guardrails that keep agent output grounded. The technical problems named are hybrid retrieval balancing keyword exactness against semantic vector search, ranking models tuned for both human readability and machine readable context, and joining structured SQL tables to unstructured documents and real time metrics. Databricks publishes no years figure anywhere in this posting, so we have not guessed one.
Who this is for
Requirements as published:
- The employer publishes no years of experience figure anywhere in this posting. We have left the experience field empty rather than guess, which means this role will not appear when you filter the board by years.
- The IR specialist: knowledge of Lucene or Elasticsearch, embeddings and ranking algorithms.
- The quality and metrics obsessive: experience building evaluation frameworks, both human in the loop and LLM based, and comfort with relevance metrics such as nDCG, MRR and Precision at K.
- The bridge builder: willingness to apply traditional information retrieval practice to retrieval augmented generation.
The actual day to day:
- Owning result quality for two different users: the AI agent that needs ground truth context, and the human who needs an intuitive, high recall search experience.
- Hybrid retrieval work, balancing keyword based search for exactness against semantic vector search for intent.
- Dual optimisation of ranking models that have to satisfy human readability and LLM ready context at the same time.
- Building the guardrails and relevance scoring that keep agent output grounded, because poor search quality shows up as hallucination.
- Connecting structured SQL tables, unstructured documents and real time business metrics.
Location and office reality:
- Bengaluru. Office days are not stated in the posting.
Honest fit guidance:
- The Senior Staff title at Databricks has historically meant 10 years or more, and its Network Platform posting in this same edition asks for 15. Since no figure is published here, assume the bar is high and read the title in Databricks terms rather than the industry average.
- This is a classical information retrieval role wearing an AI badge. The nDCG and MRR requirements are the giveaway. If your search experience is real, that is what counts.
- If you filter job boards by years, note that this role has no published figure, which is why it sits outside the experience filter here.
- The employer publishes no years of experience figure anywhere in this posting. We have left the experience field empty rather than guess, which means this role will not appear when you filter the board by years.
- The IR specialist: knowledge of Lucene or Elasticsearch, embeddings and ranking algorithms.
- The quality and metrics obsessive: experience building evaluation frameworks, both human in the loop and LLM based, and comfort with relevance metrics such as nDCG, MRR and Precision at K.
- The bridge builder: willingness to apply traditional information retrieval practice to retrieval augmented generation.
The actual day to day:
- Owning result quality for two different users: the AI agent that needs ground truth context, and the human who needs an intuitive, high recall search experience.
- Hybrid retrieval work, balancing keyword based search for exactness against semantic vector search for intent.
- Dual optimisation of ranking models that have to satisfy human readability and LLM ready context at the same time.
- Building the guardrails and relevance scoring that keep agent output grounded, because poor search quality shows up as hallucination.
- Connecting structured SQL tables, unstructured documents and real time business metrics.
Location and office reality:
- Bengaluru. Office days are not stated in the posting.
Honest fit guidance:
- The Senior Staff title at Databricks has historically meant 10 years or more, and its Network Platform posting in this same edition asks for 15. Since no figure is published here, assume the bar is high and read the title in Databricks terms rather than the industry average.
- This is a classical information retrieval role wearing an AI badge. The nDCG and MRR requirements are the giveaway. If your search experience is real, that is what counts.
- If you filter job boards by years, note that this role has no published figure, which is why it sits outside the experience filter here.
Apply on company site
Opens databricks.com, the employer's own application page. Applying is always free.