I can recall so many times I have explained people higher/ lower than me that open source does not mean “free”. You have to pay for inferencing and stuff.
I believe OS shines when hyper focused very small models are considered. In the range of ~100M to 500M. Encoder use case only though.
Decoders / text generators in this range are very bad.
Thanks so much for sharing this. 25 years ago I ran my own email server using open source code. It was great. You could customize things. It was feature rich, when a lot of the online email serviced were less than ideal. BUT… it was the most time-consuming thing I’ve ever done in my life. I’m gonna stick to online AI services for now. 😀
Oh cmon! Yes you made really good point about hidden costs. But there is another reason why you want to go OSS - control. You can actually build ai that is truly aligned with needs of your organization. And yes it will cost you. But for mission critical applications of ai you will pay premium for being sure how the ai works and that it works for you and not for OpenAI. Not for everyone. But definitely for someone
Control is important, but that’s not the point of this article. This article isn’t arguing OSS is bad, simply that it’s more expensive. And for most orgs, the tradeoff isn’t worth it.
Extremely intriguing blog! I'm building the infra platform layer to minimise such overheards for folks looking for open source models while minimising their AI bills.
Would be happy to chat with people struggling with this problem to understand your use case better!
“Open-source LLMs are not free — they just move the bill from licensing to engineering, infrastructure, maintenance, and strategic risk”
Isnt that the case with every OSS?
Databases, message brokers, data processing frameworks etc.
Isnt what you are talking about just self hosting vs managed infra?
No free lunch, and great analysis on the hidden costs.
Damn this went deep.
I can recall so many times I have explained people higher/ lower than me that open source does not mean “free”. You have to pay for inferencing and stuff.
I believe OS shines when hyper focused very small models are considered. In the range of ~100M to 500M. Encoder use case only though.
Decoders / text generators in this range are very bad.
Wow. Such a great article!
Thank you
Thanks so much for sharing this. 25 years ago I ran my own email server using open source code. It was great. You could customize things. It was feature rich, when a lot of the online email serviced were less than ideal. BUT… it was the most time-consuming thing I’ve ever done in my life. I’m gonna stick to online AI services for now. 😀
loved reading this...relates very much to the problem I faced recently
Thank you
Brilliant Deep Dive 🌟 Thanks for sharing 🌞
Oh cmon! Yes you made really good point about hidden costs. But there is another reason why you want to go OSS - control. You can actually build ai that is truly aligned with needs of your organization. And yes it will cost you. But for mission critical applications of ai you will pay premium for being sure how the ai works and that it works for you and not for OpenAI. Not for everyone. But definitely for someone
Control is important, but that’s not the point of this article. This article isn’t arguing OSS is bad, simply that it’s more expensive. And for most orgs, the tradeoff isn’t worth it.
Awesome read Devansh! Loved it :)
Extremely intriguing blog! I'm building the infra platform layer to minimise such overheards for folks looking for open source models while minimising their AI bills.
Would be happy to chat with people struggling with this problem to understand your use case better!
Great piece!!!🙌🙌