Thread

1. mafro · 1789642301 · focus · HN ↗

   I've used OpenSpec extensively on two large pieces of software which were worked solo over 6-9 months.

   Recently, I've completely ditched the specification part. I found they just weren't useful over the longer term. I used an LLM to assess in both directions whether the code matched the specs and whether the specs matched the code. On both software projects this came out with huge divergence from spec to code.

   Basically the old theory is true - the code IS the specification.

   What I did find very useful and have retained is the process flow. Create a proposal, review the proposal, implement, review the code. Also useful was building and maintaining ADRs and invariant logs for where a unit test cannot be made to verify behaviour. The process and the ADRs, unit test, invariant log all help the software stay coherent as the LLM churns on it over many unconnected contexts.

   1. cloverich · 1789653920 · focus · HN ↗

      The reason the code is the specification is because people don't take specification seriously. Usually for good reason, but sometimes not. Real, long-lived RFC's can exist, and can have directional and corrective impacts on your LLM generated code. The trick is the RFC needs to be human written and maintained. The moment the org allows the LLM themselves to modify the spec... then yes the spec is no longer useful; or rather, "why" comments in code + a sliver of high level directional / summary content is probably all that is valuable.

      A fun example can be having the LLM implement a well known spec (which it cannot edit), and then to use the spec in review to find mistakes and corrections. That's helped it really click for me. All these tools that let LLMs generate spec, even with iterative planning... I haven't found it useful for very long. It's great for building something complex in the very short term (e.g. days). But i haven't found keeping them afterwards to provide any benefit.

Unofficial Hacker News client; not affiliated with Y Combinator.