Is self-hosting an LLM cheaper than paying for an API?
Self-hosting beats a frontier API at about 16M tokens a day per H100, but beating cheap open-model APIs takes 240M+ a day and a busy GPU.
Self-hosting beats a frontier API at about 16M tokens a day per H100, but beating cheap open-model APIs takes 240M+ a day and a busy GPU.