GitHub is almost forever. Your custom domain disappears when you stop paying the bills, which if you’re an open source developer has a higher likelihood than GitHub disappearing.
One day, we’re all going back to vendoring dependencies.
Updates mean you have to copy over all the code into your repo, which creates a large diff, and hope you aren't overwriting any local changes someone might have made.
It bloats your repo, both with the actual code, and the large diffs when you update it.
You have to manually track new versions, without something to tell you if new versions are available, or if your version has known security vulnerabilities.
If the dependency has it's own dependencies, you have to vendor those too recursively. And if multiple dependencies have the same transitive dependency, it is up to you to deduplicate them, and make sure you have a version compatible with all dependents.
The biggest problem isn't (usually) disk space, or network bandwidth, it is that git operations slow down as the size of the repo grows. And it means that cloning or pulling the repo takes longer, which can be especially problematic for CI.
> Recursive dependencies have the same issues whether you vend them or not.
Package managers usually handle resolving recursive/transitive dependencies for you. Some have support for vendoring dependencies, but not all do. In theory, you could have similar tooling for vendoring dependencies, but in practice that often isn't the case.
Vendoring isn't a clear win for security. It may provide some protection against "supply chain" attacks, but makes it more difficult for you to respond to vulnerabilities discovered in the version you vendored. And for business continuity, in many cases depending on an external registry is an acceptable risk.
And both of those concerns can be addressed by using an internal/private registry that mirrors the packages you need.
But in any event, your original question was to elaborate on why vendoring is inconvenient. Whether or not the benefits are worth the inconvenience is a different question, to which IMHO the answer is "it depends". Sometimes it is, and sometimes it isn't.
> Vendoring isn't a clear win for security. It may provide some protection against "supply chain" attacks, but makes it more difficult for you to respond to vulnerabilities discovered in the version you vendored.
How so? Whether a dependency is vendored or not, you still have to update it to integrate a security update to that dependency, don't you? The alternative is to not pin your dependencies, but that is far riskier overall.
> Whether or not the benefits are worth the inconvenience is a different question
I would contend that it is the most important question. :-)
If I want to upgrade the disk on my MacBook Pro, I need to buy a new MacBook Pro with a larger disk. If I want to upgrade the disk on my work laptop, I’m SOL.
> Ensuring that diffs to updated dependencies remain within a vendor folder is trivial.
It’s not obvious to me how putting the dependencies in a vendor folder solves the diff problem. Does every code host allow you to hide diffs to certain directories?
And what’s the advantageous scenario for vendored dependencies? Is it just when the mod proxy and the upstream code host go down at the same time?
> If I want to upgrade the disk on my MacBook Pro, I need to buy a new MacBook Pro with a larger disk
Is this a significant risk in reality? MBPs today come with a minimum of 1TB of storage. Even 5 years ago I think the minimum was 256 GB. This is more than large enough for all but the most massive repositories, even with vendored dependencies. And you can always plug in external SSDs or HDDs or connect to a network server.
> And what’s the advantageous scenario for vendored dependencies? Is it just when the mod proxy and the upstream code host go down at the same time?
This is a useful homework assignment. Ask your favorite LLM or consult some respected release engineering books. Also consult your local AppSec and infosec teams.
> This is more than large enough for all but the most massive repositories
Consider the possibility that the drive needs to accommodate more than just a single repository?
I have lots of software, entire language toolchains, AI models, Docker images, application volumes, VMs, etc.
And this isn’t a theoretical concern; I have to free up space every few months.
> This is a useful homework assignment. Ask your favorite LLM or consult some respected release engineering books. Also consult your local AppSec and infosec teams.
So there is no advantage scenario that you’re aware of?
Because it sucks. But maybe it should suck? It would make us think twice before adding dependencies.
I use dotnet and I never liked seeing dlls and binary files in my diffs. I would argue if we are adding vendor code to our projects, we should demand the FULL source code instead of dlls. Maybe it is already possible with things like x unit. I have never given it much thought... But then that vendoree code has to come from somewhere as well, right? I mean there is something to be said about provenance or something here?
Sorry if this feels like a stream of consciousness because it is ↔
Vendoring dependencies doesn’t necessarily mean you have to include binary objects in your repo. They could be content-addressable artifacts in your org’s private blob store.
I think its a good question. The short answer is: because tooling doesn't default to this or make it easy.
The answer to the question of why THAT is the case - is not so easy to answer. The main benefit I can see with using lock files instead of vendoring is that it saves a lot of storage and diff history from entering your repository. So clones are much faster, backups smaller etc.
I think Go used to work this way (automated vendoring) but it’s the only language I can think of that ever did in terms of standard tooling. It would be instructive to learn why that changed.
Has this ever been necessary/useful? Genuinely curious because I’ve been using Go since 2012 and can’t recall a time when vendoring solved a problem better than “regular” modules. Like have there been times when the source went down and the module proxy went down (or didn’t have your dependency cached)?
Unless the field of software development somehow totally stagnates, I'm confident that at some point I will move to a different hosting service, as github goes the way of sourceforge, travis-ci, and myspace. So my personal domain will continue to be relevant (it's lasted fine 23 years).
Maybe at that point it won't be using git either, or maybe it will - who knows.
Mine has lasted over 25 at this point, but one day I will die, my domain name will lapse because what I do as a career might as well be witchcraft to anyone in my family and then my modules will be lost forever. Sucks any way you cut it. Hopefully someone who found my modules useful will fork whatever they find and keep them going.
SenHeng · · focus · HN ↗
One day, we’re all going back to vendoring dependencies.
otterley · · focus · HN ↗
metaltyphoon · · focus · HN ↗
otterley · · focus · HN ↗
thayne · · focus · HN ↗
It bloats your repo, both with the actual code, and the large diffs when you update it.
You have to manually track new versions, without something to tell you if new versions are available, or if your version has known security vulnerabilities.
If the dependency has it's own dependencies, you have to vendor those too recursively. And if multiple dependencies have the same transitive dependency, it is up to you to deduplicate them, and make sure you have a version compatible with all dependents.
Etc.
otterley · · focus · HN ↗
Recursive dependencies have the same issues whether you vend them or not.
Ensuring that diffs to updated dependencies remain within a vendor folder is trivial.
So, what’s left?
thayne · · focus · HN ↗
The biggest problem isn't (usually) disk space, or network bandwidth, it is that git operations slow down as the size of the repo grows. And it means that cloning or pulling the repo takes longer, which can be especially problematic for CI.
> Recursive dependencies have the same issues whether you vend them or not.
Package managers usually handle resolving recursive/transitive dependencies for you. Some have support for vendoring dependencies, but not all do. In theory, you could have similar tooling for vendoring dependencies, but in practice that often isn't the case.
otterley · · focus · HN ↗
thayne · · focus · HN ↗
That sounds like it's inconvenient to me.
otterley · · focus · HN ↗
thayne · · focus · HN ↗
And both of those concerns can be addressed by using an internal/private registry that mirrors the packages you need.
But in any event, your original question was to elaborate on why vendoring is inconvenient. Whether or not the benefits are worth the inconvenience is a different question, to which IMHO the answer is "it depends". Sometimes it is, and sometimes it isn't.
otterley · · focus · HN ↗
How so? Whether a dependency is vendored or not, you still have to update it to integrate a security update to that dependency, don't you? The alternative is to not pin your dependencies, but that is far riskier overall.
> Whether or not the benefits are worth the inconvenience is a different question
I would contend that it is the most important question. :-)
throwaway894345 · · focus · HN ↗
If I want to upgrade the disk on my MacBook Pro, I need to buy a new MacBook Pro with a larger disk. If I want to upgrade the disk on my work laptop, I’m SOL.
> Ensuring that diffs to updated dependencies remain within a vendor folder is trivial.
It’s not obvious to me how putting the dependencies in a vendor folder solves the diff problem. Does every code host allow you to hide diffs to certain directories?
And what’s the advantageous scenario for vendored dependencies? Is it just when the mod proxy and the upstream code host go down at the same time?
otterley · · focus · HN ↗
Is this a significant risk in reality? MBPs today come with a minimum of 1TB of storage. Even 5 years ago I think the minimum was 256 GB. This is more than large enough for all but the most massive repositories, even with vendored dependencies. And you can always plug in external SSDs or HDDs or connect to a network server.
> And what’s the advantageous scenario for vendored dependencies? Is it just when the mod proxy and the upstream code host go down at the same time?
This is a useful homework assignment. Ask your favorite LLM or consult some respected release engineering books. Also consult your local AppSec and infosec teams.
throwaway894345 · · focus · HN ↗
Consider the possibility that the drive needs to accommodate more than just a single repository?
I have lots of software, entire language toolchains, AI models, Docker images, application volumes, VMs, etc.
And this isn’t a theoretical concern; I have to free up space every few months.
> This is a useful homework assignment. Ask your favorite LLM or consult some respected release engineering books. Also consult your local AppSec and infosec teams.
So there is no advantage scenario that you’re aware of?
otterley · · focus · HN ↗
throwaway894345 · · focus · HN ↗
collabs · · focus · HN ↗
I use dotnet and I never liked seeing dlls and binary files in my diffs. I would argue if we are adding vendor code to our projects, we should demand the FULL source code instead of dlls. Maybe it is already possible with things like x unit. I have never given it much thought... But then that vendoree code has to come from somewhere as well, right? I mean there is something to be said about provenance or something here?
Sorry if this feels like a stream of consciousness because it is ↔
otterley · · focus · HN ↗
someonebaggy · · focus · HN ↗
jeremyjh · · focus · HN ↗
The answer to the question of why THAT is the case - is not so easy to answer. The main benefit I can see with using lock files instead of vendoring is that it saves a lot of storage and diff history from entering your repository. So clones are much faster, backups smaller etc.
I think Go used to work this way (automated vendoring) but it’s the only language I can think of that ever did in terms of standard tooling. It would be instructive to learn why that changed.
stackedinserter · · focus · HN ↗
st3fan · · focus · HN ↗
throwaway894345 · · focus · HN ↗
deniska · · focus · HN ↗
mkj · · focus · HN ↗
Maybe at that point it won't be using git either, or maybe it will - who knows.
sleepybrett · · focus · HN ↗
JodieBenitez · · focus · HN ↗
Also: <a href="https://proxy.golang.org/" rel="nofollow">https://proxy.golang.org/