The last paragraph says:
> At some point your docs will outgrow a single folder, and then all bets are off. You’ll want a separate repo with its own build process...
My question is, why is this taken as a given? Is it so hard to have docs and code live together in version control after a certain scale? If so, what is the specific problem and what is the cause?
I ask because I've never been that satisfied with the various ways I've tried to organize projects in git. Recently I've been trying to keep the source, tests and docs together in the same tree so that changes are more localized. It seems to be helping me keep track of things, especially with coding agents so eager to make changes all over the place. I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
On a large project you will have problems. You can maintain a monorepo anyway as many people do, and deal with the problems of a large monorepo. Or you can go to multirepo and deal with the issues of multirepo. Both have been done successfully, and both have significant problems that you need to work with.
Most people advocating a monorepo have never worked on a project large enough to see the issues with a monorepo and so are arguing for a monorepo without understanding the problems with them. For most people a monorepo is the correct answer because their project is small.
Seems like if you're small enough, a monorepo is the right way to go because it doesn't matter at that scale, and if you're big enough, you'll have the resources to throw at making monorepos scale.
Mono vs poly at scale needs resources. You have different compromises with each and so the resources go to different places. However there is no clear cut winner despite a few mono repo at scale advocates trying to claim otherwise - they are always completely ignoring the issues with a monorepo setup.
Exactly because monorepos have least overhead when they’re small, monorepos generally win because you need to be small for a long while until you get big.
By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.
So far, I’ve started a new job a handful of times by collapsing a premature polyrepo strategy: people were not experienced enough to merge two git repos without a common root.
I’ve only once went the other way, and it incurred so much overhead, it decreased developer productivity by some small but not insignificant percentage.
To be clear: I’m not a maximalist. All of my open-source work is exceedingly compartmentalised. My DNS library is separate from my external-dns webhook is separate from my fork of external-dns. They could all live in one repo. But FOSS encourages reusability, commercial software encourages clumping and vendoring.
> By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.
Conversely to your experience, I have worked at a handful of places who have a monorepo that has been creaking under its own weight for years, but its structure as a monorepo now underpins the business, and so migration to a polyrepo simply never happens, and developers are now checking out a 50GB repo in its entirety periodically.
I'm on the other side of this problem, with a company that went multi repo for bad reasons (political, not technical) and I would give you serious money if you could solve my problems by just forcing me to check out 50gb every now and then...
Instead I deal with a 30+ repo clusterfuck (technically we have 60+ services, but I only have to run half...) that is held together by hopes and prayers, takes literal hours of actual effort to bring everything up to date on master, and has become a fractured hellscape where people are afraid to leave their tightly constrained silos of service combinations.
Long story short... I will take a bad monorepo over bad multirepo any day of the week.
Yeah, compared with the maintenance effort of running around to 100 different repos, keeping everything in sync and the deps all updated, I'd always take the pain of 50GB checkouts.
It looks like these days a combination of git-scalar, git-lfs, and bazel will work for a small 50GB monorepo like that. The largest cost would be porting over the existing build system to bazel and then integrating it with CI. Once that is done, though, the monorepo will just scale, and all the constant ongoing cost of multirepo sync will be gone.
gwking · · focus · HN ↗
My question is, why is this taken as a given? Is it so hard to have docs and code live together in version control after a certain scale? If so, what is the specific problem and what is the cause?
I ask because I've never been that satisfied with the various ways I've tried to organize projects in git. Recently I've been trying to keep the source, tests and docs together in the same tree so that changes are more localized. It seems to be helping me keep track of things, especially with coding agents so eager to make changes all over the place. I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
bluGill · · focus · HN ↗
Most people advocating a monorepo have never worked on a project large enough to see the issues with a monorepo and so are arguing for a monorepo without understanding the problems with them. For most people a monorepo is the correct answer because their project is small.
dualvariable · · focus · HN ↗
bluGill · · focus · HN ↗
sshine · · focus · HN ↗
By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.
So far, I’ve started a new job a handful of times by collapsing a premature polyrepo strategy: people were not experienced enough to merge two git repos without a common root.
I’ve only once went the other way, and it incurred so much overhead, it decreased developer productivity by some small but not insignificant percentage.
To be clear: I’m not a maximalist. All of my open-source work is exceedingly compartmentalised. My DNS library is separate from my external-dns webhook is separate from my fork of external-dns. They could all live in one repo. But FOSS encourages reusability, commercial software encourages clumping and vendoring.
gilfaethwy · · focus · HN ↗
Conversely to your experience, I have worked at a handful of places who have a monorepo that has been creaking under its own weight for years, but its structure as a monorepo now underpins the business, and so migration to a polyrepo simply never happens, and developers are now checking out a 50GB repo in its entirety periodically.
horsawlarway · · focus · HN ↗
Instead I deal with a 30+ repo clusterfuck (technically we have 60+ services, but I only have to run half...) that is held together by hopes and prayers, takes literal hours of actual effort to bring everything up to date on master, and has become a fractured hellscape where people are afraid to leave their tightly constrained silos of service combinations.
Long story short... I will take a bad monorepo over bad multirepo any day of the week.
dualvariable · · focus · HN ↗
dualvariable · · focus · HN ↗