SIGN IN SIGN UP

feat: Evals with explanations (#1699)

* Add explanation template

* Spike out explanations

* Ruff 🐶

* Use tailored explanation prompt

* Add explanation templates for all evals

* Wire up prompt template objects

* Update models to use new template object

* Ruff 🐶

* Resolve type and linter issues

* Fix more typing issues

* Address first round of feedback

* Extract `ClassificationTemplate` ABC

* Label extraction belongs to the "template" object

* Add logging for unparseable labels

* Patch in openai key environment variable for tests

* Refactor to address feedback

* Evaluators should use PromptTemplates

* Pair with Mikyo

* Fix for CI

* `PROMPT_TEMPLATE_STR` -> `PROMPT_TEMPLATE`

* Print prompt if verbose

* Add __repr__ to `PromptTemplate`

* fix relevance notebook

* docs: update evals

* Normalize prompt templates in llm_classify

* Ruff 🐶

* feat(evals): add an output_parser to llm_generate (#1736)

* feat(evals): add an output_parser param for structured data extraction

* remove brittle test

* docs(evals): document llm_generate with output parser (#1741)

---------

Co-authored-by: Mikyo King <mikyo@arize.com>
D
Dustin Ngo committed
2db814102ea27f441e201740cc75ace79c82837c
Parent: 1e70ec3
Committed by GitHub <noreply@github.com> on 11/14/2023, 8:07:47 PM