RLHF coding prefs
Rank model code completions for correctness, style, and safety. Produce preference pairs labs can train on.
Coding · Illustrative record
Coding expert
Elite competitive programmer and systems thinker for AI coding evaluation, preference ranking, and algorithm review. Exceptional, not merely good.
Work this expert does
Rank model code completions for correctness, style, and safety. Produce preference pairs labs can train on.
Score solutions on complexity, edge cases, and clarity. Catch subtle bugs that generalist annotators miss.
Build and apply domain rubrics for SWE agents: APIs, concurrency, testing, and documentation quality.
Other domains