feat: Evals with explanations (#1699)
* Add explanation template * Spike out explanations * Ruff 🐶 * Use tailored explanation prompt * Add explanation templates for all evals * Wire up prompt template objects * Update models to use new template object * Ruff 🐶 * Resolve type and linter issues * Fix more typing issues * Address first round of feedback * Extract `ClassificationTemplate` ABC * Label extraction belongs to the "template" object * Add logging for unparseable labels * Patch in openai key environment variable for tests * Refactor to address feedback * Evaluators should use PromptTemplates * Pair with Mikyo * Fix for CI * `PROMPT_TEMPLATE_STR` -> `PROMPT_TEMPLATE` * Print prompt if verbose * Add __repr__ to `PromptTemplate` * fix relevance notebook * docs: update evals * Normalize prompt templates in llm_classify * Ruff 🐶 * feat(evals): add an output_parser to llm_generate (#1736) * feat(evals): add an output_parser param for structured data extraction * remove brittle test * docs(evals): document llm_generate with output parser (#1741) --------- Co-authored-by: Mikyo King <mikyo@arize.com>
D
Dustin Ngo committed
2db814102ea27f441e201740cc75ace79c82837c
Parent: 1e70ec3
Committed by GitHub <noreply@github.com>
on 11/14/2023, 8:07:47 PM