I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.
I think the trouble is that Typesafe APIs don't fit into the normal OpenAI-style API that every other regular LLM provider users. You're not just providing unstructured text and getting unstructured text back. It would take a different request and response format than every other model on Open Router. Though you could shoe-horn it in some way, it'd be hacky.
But agreed it'd be very useful to see it deployed on other hubs, and it seems worth it to provide the bespoke API format. Perhaps Typesafe's API will end up becoming the standard for a new type of structured model, the way OpenAI's API did.
mushufasa · · focus · HN ↗
varenc · · focus · HN ↗
But agreed it'd be very useful to see it deployed on other hubs, and it seems worth it to provide the bespoke API format. Perhaps Typesafe's API will end up becoming the standard for a new type of structured model, the way OpenAI's API did.
anshumankmr · · focus · HN ↗