Thread

Fakecloud: Local AWS cloud emulator for integration tests

153 points · 76 comments · theanonymousone

Loading the complete thread in the background. This saved snapshot is available now. Refresh

igetspam · 1790516038 · focus · HN ↗
But how does it compare to flock, which already picked up with localstack left off?
1. duesabati · 1790516375 · focus · HN ↗
  you mean floci? <a href="https://floci.io/floci/" rel="nofollow">https://floci.io/floci/
  1. igetspam · 1790517567 · focus · HN ↗
    Yeah. Stupid really didn’t want to type that. :)
2. WalterGR · 1790522032 · focus · HN ↗
  Floci happened to be discussed yesterday, with 47 comments: <a href="https://news.ycombinator.com/item?id=49854416">https://news.ycombinator.com/item?id=49854416
bilekas · 1790516512 · focus · HN ↗
There are always nice, but there are a lot of scaffolding work involved with larger systems. It would be nice to be able to provide for example the terraform of the infrastructure and let these tool create their own. Might be a nice feature to add because I haven't seen it done yet.
1. igetspam · 1790517623 · focus · HN ↗
  I use tf with floci and floci-gcp, weekly…
  1. _joel · 1790518965 · focus · HN ↗
    Oh nice, not aware of this. We're opentofu on GCP, so could be a good addition to ci
2. lucas_vieira · 1790561502 · focus · HN ↗
  [dead]
gyre007 · 1790518789 · focus · HN ↗
I kinda like most of these emulators but why is there none targeting GCP/Azure
1. figmert · 1790519203 · focus · HN ↗
  AWS is still way ahead in cloud usage, that's probably why AWS is the primary target. Anyway, floci has GCP and Azure versions, how mature they are, I'm unsure. But it does exist.
  1. oso2k · 1790525118 · focus · HN ↗
    Right. Floci got posted yesterday here on HN.
    <a href="https://news.ycombinator.com/item?id=49854416">https://news.ycombinator.com/item?id=49854416
2. [deleted] · 1790519534 · focus · HN ↗
  [deleted]
3. tombuildsstuff · 1790522476 · focus · HN ↗
  I'm launching something in that space this week - if you're interested in trying one of those out, feel free drop me a message via my website (in bio) :)
4. nullbio · 1790522830 · focus · HN ↗
  Agreed. Azure would be nice. Something that can work with Terraform too.
  1. igetspam · 1790525785 · focus · HN ↗
    Check out floci-az. I haven’t touched it but I know it’s there.
5. Hikikomori · 1790525638 · focus · HN ↗
  Hard to replicate all the bugs in azure.
6. dmacvicar · 1790797759 · focus · HN ↗
  LocalStack has an Azure emulator in preview:
  <a href="https://www.localstack.cloud/localstack-for-azure" rel="nofollow">https://www.localstack.cloud/localstack-for-azure
  Disclaimer: I work for LocalStack
Lucasoato · 1790520422 · focus · HN ↗
I always welcome warmly projects like these, the only problem is that unless there’s a strong community behind, founders tend to build companies around them, raise capital, then put useful functionalities behind paywalls (see localstack).
When I see that, it always reminds me of Steve Jobs’s quote: ”It’s not a product, it’s a feature”.
Aren’t cloud companies like AWS, Azure, GCP the first to be interested in providing their users a meaningful way to do test integrations of their products? Shouldn’t they provide them in the first place?
1. igetspam · 1790525726 · focus · HN ↗
  You’d think that but it doesn’t seem to be the case. I do a lot of offline development and the only way it used to be possible was localstack but that’s AWS only. The open source community is the only interested party, it seems. Thankfully, there’s a lot of contribution happening in this space. Because I have need, I make a lot of my own contributions. I’m doing full stack cloud development against GCP without ever leaving my device.
2. stackskipton · 1790528788 · focus · HN ↗
  It depends on the company. Azure provides local emulators for some of their products like CosmosDB and Azure Storage.
  Depending on what your size and other things, sometimes it's just easier to have dev instances you test against. Like S3 emulator, you could download one of many and test it but unless your test suite is extensive, the costs against S3 test bucket would probably cost you .03 cents for assurances that everything will work against AWS. If that's breaking the bank at the company, you probably have bigger problems.
  1. hvb2 · 1790530850 · focus · HN ↗
    > Like S3 emulator, you could download one of many and test it but unless your test suite is extensive, the costs against S3 test bucket would probably cost you .03 cents
    Not sure if your serious.
    You picked the simplest service that isn't expensive at all to run. Have you tried with an RDS cluster for example, how does that work?
    Does your integration test create a new cluster to use? That's slow to set up.
    Does it use an existing one? Now you might have failures due to noisy neighbors and your cost will not be small.
    And that's just RDS, as soon as it's not serverless, costs aren't trivial anymore.
    stackskipton · 1790533028 · focus · HN ↗
    I assume for RDS for integration testing, you could just use publicly available software. So stand up MySQL container if your RDS is MySQL. This is integration testing, not load testing so shared RDS in lowers would be fine as well. Noisy neighbors
    Again, while local cloud emulator is great for cost sensitive customers, at some point, you should test Dev against almost exact environment you will be running Prod on. If not, you are asking for eventual outage in Prod when some slightly different AWS thing bites you.
    hvb2 · 1790594404 · focus · HN ↗
    Your first paragraph "just use a MySQL container" and the second "test against what prod runs on", seem to contradict?
    > If not, you are asking for eventual outage in Prod when some slightly different AWS thing bites you.
    Exactly...
    stackskipton · 1790622602 · focus · HN ↗
    It depends on your tests and environments. MySQL container should be used for integration and CI/CD testing if you are cost sensitive. Dev or Staging should be end to end testing against AWS RDS.
    Whole point is this stuff is for those who think running Dev RDS is too expensive.
otterley · 1790521883 · focus · HN ↗
There’s also MiniStack, which forked from LocalStack after they started breaking developer workflows: <a href="https://ministack.org/" rel="nofollow">https://ministack.org/
Both fakecloud and its website look sloppily vibe-coded, and its “authors” are anonymous. It’s going to take a while for it to earn trust. I’d treat it with suspicion. (Curl-to-shell pipe to install? Ugh.)
1. straygarr · 1790522617 · focus · HN ↗
  Agreed with everything up to:
  "curl-to-shell pipe to install" - what's the problem here? that's pretty common on linux systems and something the AWS CLI uses.
  Or is the problem the fact that this dev is untrusted and is executing a possibly malicious script on your machine?
  1. MisterMunchkin · 1790523114 · focus · HN ↗
    Running arbitrary code directly in your terminal is very dangerous
    luma · 1790526881 · focus · HN ↗
    Any code I didn't write is arbitrary code. At some point I'm left to trust someone or run no software at all.
    mulmen · 1790527183 · focus · HN ↗
    [delayed]
    mynameisvlad · 1790531175 · focus · HN ↗
    So then... Just do that and it's no longer reckless.
    If someone wants to be reckless they can be. If someone doesn't, they also have that ability.
    bornfreddy · 1790531627 · focus · HN ↗
    Actually... Server can detect if you are piping or not and serve a modified version for inspection.
    But really, there is no reason not to use prebuilt packages for distribution. Curlpiping needs to die.
    otterley · 1790532481 · focus · HN ↗
    How can the server detect what the user agent is doing to do?
    otterley · 1790533270 · focus · HN ↗
    At first I thought “this sounds like bullshit” but then found <a href="https://web.archive.org/web/20230408195648/https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side" rel="nofollow">https://web.archive.org/web/20230408195648/https://www.idont...
    mulmen · 1790547629 · focus · HN ↗
    [delayed]
    mulmen · 1790553145 · focus · HN ↗
    [delayed]
    john01dav · 1790554411 · focus · HN ↗
    If you're about to install an upstream binary with this and you're not reviewing this then it's purely (irrational) vibes to be concerned with the also unreviewed shell script from the same source.
    NewJazz · 1790564526 · focus · HN ↗
    [delayed]
    darkwater · 1790581060 · focus · HN ↗
    If you don't really trust the author, what's a checksum going to provide you? That the maybe malicious code released on GH has not been tampered with? Not so helpful.
    NewJazz · 1790608400 · focus · HN ↗
    [delayed]
  2. sdcfgy · 1790524530 · focus · HN ↗
    I'm worried that this behaviour has to be defended.
  3. jasongi · 1790524639 · focus · HN ↗
    For a mock server? Surely a versioned, standalone executable, library/package or docker image makes more sense. Integration tests generally need to be portable and running on CI, you don't wanna be shell-piping whatever exists in the moment.
  4. x3n0ph3n3 · 1790528279 · focus · HN ↗
    curl-to-shell is a terrible installation mechanism because it's not easily reversible and I can't tell if any of the assets are signed, or integrity checked, or not.
  5. ssl-3 · 1790531353 · focus · HN ↗
    The problem, for me, is that self-running installers can create a mess that's hard to keep track of.
    I've run Linux without meaningful package management, as that was kind of the style of the time 30 years ago with Slackware. It can quickly become untenable.
    There's no real difference between an uninspected script that gets piped straight from the URL into the shell, or a similarly-uninspected make&&sudo make install routine from a tarball. They can both execute code that does bad things (whether unintentionally or deliberately), and they can both leave a mess that is hard to cleaned up.
    I've found that it is better to just avoid going down that road to begin with. Whether distro-specific packages, Docker containers, flatpaks, or whatever: All of these make housekeeping easier.
    giantrobot · 1790561914 · focus · HN ↗
    > The problem, for me, is that self-running installers can create a mess that's hard to keep track of.
    For reasons my machine with a beefy GPU is stuck on an older set of Nvidia drivers. I've got them pinned with apt. The ollama installer fucked everything up by updating stuff that apparently wasn't pinned. After I had fun cleaning up that fucking mess I found it overwrote my custom systemd service file so I had to go in and fix that.
    Curl-to-shell is a bullshit antipattern. I have no interest in going back to the dark days of expanding tarballs to / and hoping for the best.
    ssl-3 · 1790600299 · focus · HN ↗
    [delayed]
2. tingletech · 1790525313 · focus · HN ↗
  > and its “authors” are anonymous.
  The blog posts are all attributed to "Lucas Vieira" and the dev group <a href="https://faisca.dev" rel="nofollow">https://faisca.dev that is attributed as the author has 2 other projects. Lucas comes up in LinkedIn and looks like an actual person working in San Francisco.
  Re: curl; this seems to work:
  cargo install fakecloud
  1. losteric · 1790529736 · focus · HN ↗
    faisca.dev is also vibed - perhaps it's a real human using AI to build an online fascade
    SomeUserName432 · 1790534445 · focus · HN ↗
    I don't think it's being questioned that at some point there is human behind this. Though it could always be that spider.
    _zoltan_ · 1790536843 · focus · HN ↗
    why does it matter that it's a vibe coded website?
    otterley · 1790541703 · focus · HN ↗
    Because taste, creativity, and differentiation matter to a human audience. Perhaps not you personally, but to most people. Ask any experienced marketer.
    fragmede · 1790552321 · focus · HN ↗
    I don't know about your experienced marketers, but mine are all using AI in some way shape or form!
    otterley · 1790554023 · focus · HN ↗
    It’s not really a question of whether they’re using it or not (most of them are); it’s a question of whether they’re using it to generate work product intended to be consumed by their customers. Most know better than that. Some might use it for that, too; but you know what they say about half of the population being below average.
3. iLoveOncall · 1790546322 · focus · HN ↗
  > Both fakecloud and its website look sloppily vibe-coded
  As if the MiniStack website wasn't also obviously vibe-coded lol.
  1. otterley · 1790552096 · focus · HN ↗
    It probably is, and that’s not great, either.
  2. nnucera · 1790585620 · focus · HN ↗
    Yes, the prompt was: "Make a Ministack website. Make no mistakes" Does that make it less useful?
    You can check who’s using Ministack on public GitHub repos, NVIDIA, Square, NASA, the U.S. gov, the UK gov, and many others. There are plenty of examples demonstrating that we’re doing something good
    If you still want to use COBOL because it makes you feel better or smarter, good for you. That doesn’t mean Ministack lacks quality
    otterley · 1790587699 · focus · HN ↗
    Your software can be great and your website can be mid at the same time. Perhaps the latter isn’t the highest priority for you, but criticism of it isn’t invalid.
4. upg1979 · 1790557577 · focus · HN ↗
  Happy living with Moto <a href="https://docs.getmoto.org/en/latest/index.html" rel="nofollow">https://docs.getmoto.org/en/latest/index.html
5. lucas_vieira · 1790560906 · focus · HN ↗
  [dead]
6. lucas_vieira · 1790561198 · focus · HN ↗
  [dead]
7. dmacvicar · 1790797944 · focus · HN ↗
  MiniStack is not (as I remember), a fork of LocalStack.
  Disclaimer: I work for LocalStack
shermantanktop · 1790524735 · focus · HN ↗
There are so many of these, but none of them simulate the surprise-billing experience.
1. igetspam · 1790525827 · focus · HN ↗
  File a feature request! ;)
jeduardo · 1790525240 · focus · HN ↗
There is also localemu, which I understand to be another localstack fork: <a href="https://localemu.cloud/" rel="nofollow">https://localemu.cloud/
arpinum · 1790528924 · focus · HN ↗
"46 at true 100% conformance". This is factually incorrect. Inspected their DynamoDB service and it comes nowhere close to implementing the full api.
1. arpinum · 1790529441 · focus · HN ↗
  Here are some bugs: - `ADD m.a.c :inc` update expression wrote a literal top-level m.a.c attribute but didn't change the nested number. - TransactGetItems returned fields not requested. - update expression can update the same key multiple times.
  In total my conformance suite found 583 failures out of 1279 tests.
  1. equinumerous · 1790538586 · focus · HN ↗
    Wow! This is not production-ready at all. DynamoDB would be my main use case. I'll stick with DynamoDB local.
    arpinum · 1790543826 · focus · HN ↗
    Funny fact, DynamoDB local fails about 100 of my conformance tests. Mainly around error message formatting and text. Just be careful about parsing those.
    nnucera · 1790546510 · focus · HN ↗
    <a href="https://paritysuite.org/" rel="nofollow">https://paritysuite.org/ You can check this, awesome work they have to check ddb vs real (local ddb also has a lot of diff with real ddb)
    jamesfinlayson · 1790578361 · focus · HN ↗
    Interesting - I've only ever used it for unit tests and the only issue I had was (unsurprisingly) restoring from a back-up not being supported.
  2. lucas_vieira · 1790560282 · focus · HN ↗
    Thank you. I'm going through the DynamoDB issues now, starting with the three you listed. The Parity Suite is great. I'm wiring it into fakecloud's CI.
michaelastreiko · 1790531304 · focus · HN ↗
Local stubs like this are a lifesaver for one-person webhook testing without burning cloud credits every morning.
karpetrosyan · 1790540813 · focus · HN ↗
Interesting thing, looking forward to testing it with my Smithy-generated Python AWS SDK <a href="https://github.com/kap-sh/capo" rel="nofollow">https://github.com/kap-sh/capo
matt3210 · 1790543324 · focus · HN ↗
Why not just offer an AWS compatible cloud service at this point ... can you copyright an API?
goostavos · 1790548148 · focus · HN ↗
These miss the point of integration testing (in my Humble opinion). Integration tests are valuable because they get around the problem of mocking. They let you reach out into the foreign thing you use and see if it behaves the way you expect it to. If you replace the real thing with a fake thing, you are, well, by definition, not learning anything about the real thing. It's the subtle behaviors and quirks of real services that often cause the most problems.
My $0.02: just use the AWS SDK to spin up real infrastructure on demand. You can provision SQS in < 10ms. Buckets in <100ms. DDB can take upwards of 30 seconds to a minute if you're adding GSIs, but it's a very small tax in exchange for the confidence it brings. (People often balk at this idea when they first hear it, but give it a shot! I have stamped out many a bug thanks to being able to bounce ideas off real infra.)
1. joshribakoff · 1790553664 · focus · HN ↗
  You seem to be mixing up integration tests with end to end tests.
2. lucas_vieira · 1790560007 · focus · HN ↗
  [dead]
3. najmbajwa123 · 1790591913 · focus · HN ↗
  I think what actually pushes people back is cleanup not speed. Real infra can give you things you actually want to test.
  1. goostavos · 1790738498 · focus · HN ↗
    Cleanup is indeed an art when you go with this approach. Namespacing helps a lot, as well as little things like randomizing individual queue names such that test run B never knows that infra from run A (that may still be hanging around from a failed cleanup) exists.
joshribakoff · 1790553866 · focus · HN ↗
Instead of a simulator that runs in its own process, I would recommend mocking out inline in the tests. Only then are your tests entirely “self contained”.
Otherwise your test implicitly make assumptions like “User A is always off-line” (or bucket A is always empty), add a reader of the test would have to go find where that assumption is hardcoded in the simulator and verify it separately from the test itself.
To someone reading that test it is not intuitive that user A is always off-line or bucket A is empty.
If you mock out in line, you can have tests that mock user a / bucket a — to simulate any state for any user or bucket. Perhaps multiple states in one test.
Also think about scenarios like the bucket is initially populated, but then on a subsequent request, it’s empty.
With the simulator approach, you’d have an endless number of permutations of scenarios to add — each of these scenarios would be highly coupled to the test it belongs to, so why not just put it in line in the test itself?
Any reader of the test that uses inline mocking would clearly see which scenario is being tested instead of having to dive into a separate simulator to see what special treatment is hardcoded for User a / bucket a (or maintaining potentially hundreds or thousands of simulated buckets with ad hoc names like “empty-on-second-request” or “bucket–that-responds-slow-and-times-out-but-succeeds-on-retry”.
1. hk1337 · 1790555210 · focus · HN ↗
  And if you want to see how they all interact together in AWS, that's why you have a development sandbox AWS account.
2. bushbaba · 1790567695 · focus · HN ↗
  Better yet, use dependency injection and use fakes. Makes it very easy to test out logic
3. regularfry · 1790580422 · focus · HN ↗
  That's fine if the interaction with the service is within the scope of your code. If you want to, say, mock out terraforming a bunch of IAM permissions to least-privilege a lambda such that you can check it actually works, you don't have a seam where any app-level tests can go in.
  That's where the difficulty is, where the round trip to testing this stuff on live services is painfully slow and error feedback is mixed at best.
zeronone · 1790554930 · focus · HN ↗
The support for SES looks really great, better than LocalStack.
- <a href="https://fakecloud.dev/docs/services/ses/" rel="nofollow">https://fakecloud.dev/docs/services/ses/
favori995749721 · 1790561256 · focus · HN ↗
[dead]
lucas_vieira · 1790562128 · focus · HN ↗
[dead]
waterTanuki · 1790581296 · focus · HN ↗
Alot of mock services have come out of the woodwork now due to genAI, but I don't think these are the solution from decoupling your services from cloud providers, because they have to keep up with schema updates for every single service which isn't practical even with genAI.
You should be writing everything in a sans-IO pattern: define your core, the interfaces, and your own types, (even if it seems redundant) and never, ever import a single dependency from a cloud provider in that core. Make two implementations of the interfaces: one in-memory that can be spun up quickly for testing and one that uses the actual cloud provider APIs. Now instead of hoping fakecloud or localstack update their mocks to be inline with AWS, all you have to do is update your dependencies and the implementation code instead of the core.
This is also why I avoid lambda and other "serverless" compute like the plague because it's designed from the ground up to lock you in.
1. utopiah · 1790582532 · focus · HN ↗
  > decoupling your services from cloud providers
  doesn't it boil down to not using unique services?
  If nobody provides an alternative today that could switch as quickly as changing an endpoint and token then you are coupled.
  It's not a technical decision as much admitting in the strategy that we are cutting corners because it's cheaper/faster today but some hypothetical day in the future when we'll magically have more resource we won't?

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.