Object store is quickly becoming the new core data substrate. Lets build kafka, but on s3. Lets build github, but on s3. It feels like we going to see more and more "object-store first" systems in the next few years.
I am excited about this future. Give me stateless servers and a storage bucket over having to manage systems with disks any day.
I do wonder if we will see an expansion of the s3 api to support more of these use cases. S3 added a janky file append operation to their new express-one-zone bucket type, and limited to 10k total file append operations. I wonder what else we will get in the next few years.
The range of things you can do with blob storage and a (very simple) auth model are surprisingly broad.
We recently replaced our Docker container registry with S3 using a tiny tool [1] we built in-house. I think that even with current capabilities, we can still model a lot services as a very thin layer over object storage.
That sounds really cool — I tend to agree with you that S3 and similar are underutilized, but I remember that essentially all of the providers charge for bandwidth measured in gigabytes and I'm like no. My consumer line is measured in megabits per second and if I have to pay for my data usage the way it's paid for in data centers it would be far more expensive. Somehow consumer ISPs, who have to pay for the lines, are cheaper than than the cloud providers.
> My consumer line is measured in megabits per second and if I have to pay for my data usage the way it's paid for in data centers it would be far more expensive
Well because your provider assumes you are not using all your bandwidth constantly. Cloud bandwidth is only billed for you actually use
It's still true that "clouds" cost much more than proper internet connections at DCs. E.g. AWS wants you to pay $90/TB, Hetzner $1.50/TB, a good deal on a contract is probably half what Hetzner pays since they need profit too.
And if you have two specific endpoints you need to transfer data between at a high rate, you can get stupidly cheap cost per GB on a leased line in exchange for making all that commitment upfront.
psanford · · focus · HN ↗
I am excited about this future. Give me stateless servers and a storage bucket over having to manage systems with disks any day.
I do wonder if we will see an expansion of the s3 api to support more of these use cases. S3 added a janky file append operation to their new express-one-zone bucket type, and limited to 10k total file append operations. I wonder what else we will get in the next few years.
khazit · · focus · HN ↗
The range of things you can do with blob storage and a (very simple) auth model are surprisingly broad.
We recently replaced our Docker container registry with S3 using a tiny tool [1] we built in-house. I think that even with current capabilities, we can still model a lot services as a very thin layer over object storage.
[1]: <a href="https://github.com/Simple-Observability/grue" rel="nofollow">https://github.com/Simple-Observability/grue
tomjen3 · · focus · HN ↗
gbalduzzi · · focus · HN ↗
Well because your provider assumes you are not using all your bandwidth constantly. Cloud bandwidth is only billed for you actually use
someonebaggy · · focus · HN ↗
And if you have two specific endpoints you need to transfer data between at a high rate, you can get stupidly cheap cost per GB on a leased line in exchange for making all that commitment upfront.