All pull requests

Pull requests

• fix(types): allow None for evaluation seeds

  #4297·palindromeRice opened Oct 1, 2026·

  ·
  1

• feat(tasks): support sliced dataset splits

  #4296·palindromeRice opened Oct 1, 2026·

  ·
  1

• fix(metrics): propagate weighted group uncertainty from subtask errors

  #4295·Afloat16 opened Oct 1, 2026·

  ·
  1

• fix(scripts): repair prompt export and add coverage

  #4294·palindromeRice opened Oct 1, 2026·

  ·
  1

• fix(metrics): run all requested bootstrap resamples in bootstrap_stderr

  #4293·barlowa124 opened Oct 1, 2026·

  ·
  1

  1

• feat(tasks): add LiveCodeBench code-generation benchmark

  #4292·barlowa124 opened Oct 1, 2026·

  ·
  1

• feat(hf): support independent PEFT revisions

  #4291·palindromeRice opened Oct 1, 2026·

  ·
  1

• fix(api): keep an all-whitespace context from encoding to no tokens

  #4290·feiiiiii5 opened Oct 1, 2026Contributor·

  ·
  1

• fix(ruler): correct partial-answer scoring

  #4289·palindromeRice opened Oct 1, 2026·

  ·
  1

• fix(minerva_math): keep the sympy check working where SIGALRM is unavailable

  #4287·MohammadHijjawi97 opened Oct 1, 2026·

  ·
  1

• fix(habana): restore max length after generation

  #4285·sdivyanshu90 opened Oct 1, 2026Contributor·

  ·
  1

  1

• fix(mamba): correct missing dependency guidance

  #4284·sdivyanshu90 opened Oct 1, 2026Contributor·

  ·
  1

  1

• Warn when metric_list names a metric the output type never emits

  #4281·JOhnsonKC201 opened Oct 1, 2026·

  ·
  1

• fix(models): ignore empty stop sequences in HF stopping criteria

  #4280·feiiiiii5 opened Oct 1, 2026Contributor·

  ·

• fix: acc_all_stderr groups by (paragraph, question) to match acc_all

  #4279·raunak2007 opened Oct 1, 2026·

  ·
  2

• Add canine_geroscience tasks: short-answer QA on aging research in companion dogs

  #4275·w0lph opened Sep 30, 2026·

  ·
  1

• fix(filters): fall back when a regex capture is only whitespace

  #4273·feiiiiii5 opened Sep 30, 2026Contributor·

  ·

• fix(litellm): forward EOS to async requests

  #4271·sdivyanshu90 opened Sep 29, 2026Contributor·

  ·
  1

  1

• fix(models): normalize null stop sequences

  #4269·sdivyanshu90 opened Sep 29, 2026Contributor·

  ·
  1

  1

• fix(openai): don't treat every model name containing "5" as a reasoning model

  #4267·DrDLyuan opened Sep 29, 2026·

  ·
  2

• fix(hf): honor generation length limits over checkpoint defaults

  #4266·huhumeng2 opened Sep 28, 2026·

  ·
  1

  1

• fix(okapi): add Chinese HellaSwag task

  #4264·cimutianxin opened Sep 28, 2026·

  ·
  1

  2

• fix(hf): size auto batch probes from generation request lengths

  #4263·huhumeng2 opened Sep 28, 2026·

  ·
  1

• fix(api): group acc_all_stderr by paragraph and question like acc_all

  #4262·simpleqt opened Sep 28, 2026·

  ·
  1

• fix(api): cap the fewshot over-sample draw at the pool size

  #4261·simpleqt opened Sep 28, 2026·

  ·
  1