OpenAI introduces LifeVal, a benchmark of how well AI helps people complete everyday tasks
OpenAI's new research benchmark scores help with real errands rather than exam questions.
// Key points
- LifeVal measures how well AI helps people complete everyday tasks
- Examples range from finding childcare to managing finances and government paperwork
Builder's takeI've only seen OpenAI's summary, so I'll hold off on any scores until the full page is readable. The direction is right for products, though: judge an assistant by whether the errand got done. For AI Cloud Drive I'd keep a small eval set of real user errands, such as finding a contract and filling in a form from it.
// Background · from #OpenAI
Full timeline →- Nov 2 AWS shows how to point the ChatGPT desktop app and Codex at Amazon Bedrock in AWS GovCloud (US), billed per token with no seat licenses
- Nov 1 OpenAI releases GPT-6 Rosalind for life sciences research, open only to approved organizations, at $15 input and $75 output per million tokens
- Nov 1 OpenAI publishes a community safety report and new protections for teens, strengthening violence prevention in ChatGPT