Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

While it’s easier than ever to deploy automation, it’s much harder to ensure those early investments won’t hold a company back as it scales.