HackerRank
About this role
HackerRank's assessment platform is used by more than 2,500 companies to test developers, and this role exists because its core assumption has broken. Evaluation used to be deterministic: code either passed the test cases or it did not, and the score was binary and reproducible. With AI assistance now ambient in how developers work, the question has shifted from whether someone can write a function to how effectively they can direct AI while still holding the fundamentals. The posting is candid that what replaces the old model is genuinely unsolved and that nobody has cracked it yet. Scope spans live interviews, asynchronous assessments, AI assisted coding environments and pair programming with agents, so it is a research flavoured machine learning problem with a product attached.
Who this is for
HackerRank publishes no years of experience figure anywhere on this posting, so none has been recorded here rather than guessing. This has been consistent across HackerRank listings on this board.
What the posting describes as the problem: measuring skill when AI is already in the room. Software engineering has moved from writing code to using AI to solve problems, and the frameworks used to evaluate developers have not kept up. For more than a decade, skills based hiring relied on deterministic evaluation where a candidate's code either passed test cases or did not, producing a binary and reproducible score. The role is hired to work out what replaces that, and the posting states plainly that this is unsolved rather than a matter of implementation.
Scope covers every context in which someone is trying to judge how good a developer actually is: live interviews, asynchronous assessments, AI assisted coding environments and pair programming with agents.
Location is Bengaluru on a hybrid basis.
Good fit if you are a machine learning engineer who is drawn to open ended, ill defined problems and wants to design evaluation methodology rather than tune a known model against a known benchmark. Measurement, fairness and validity are the substance of the work here.
Not a fit if you want clear specifications and a defined success metric, because the posting says outright that the framework does not exist yet. Also worth noting that because HackerRank states no experience figure, this listing will not appear under any experience filter on this page.
What the posting describes as the problem: measuring skill when AI is already in the room. Software engineering has moved from writing code to using AI to solve problems, and the frameworks used to evaluate developers have not kept up. For more than a decade, skills based hiring relied on deterministic evaluation where a candidate's code either passed test cases or did not, producing a binary and reproducible score. The role is hired to work out what replaces that, and the posting states plainly that this is unsolved rather than a matter of implementation.
Scope covers every context in which someone is trying to judge how good a developer actually is: live interviews, asynchronous assessments, AI assisted coding environments and pair programming with agents.
Location is Bengaluru on a hybrid basis.
Good fit if you are a machine learning engineer who is drawn to open ended, ill defined problems and wants to design evaluation methodology rather than tune a known model against a known benchmark. Measurement, fairness and validity are the substance of the work here.
Not a fit if you want clear specifications and a defined success metric, because the posting says outright that the framework does not exist yet. Also worth noting that because HackerRank states no experience figure, this listing will not appear under any experience filter on this page.
Apply on company site
Opens job-boards.greenhouse.io, the employer's own application page. Applying is always free.