‹ BackHN Continuity

Thread

The Legend of von Neumann (1973) [pdf]

311 points · 186 comments · suopspaces

  1. ricksunny · · focus · HN ↗
    Feels appropriate to mention here recent HN mentions about who deserves credit for 'von Neumann architecture"

    <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49870485">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49870485

    1. dandersch · · focus · HN ↗
      Could the controversy behind it be the reason that the author ommitted it? It was established terminology by 1973, as far as I can tell.
      1. ricksunny · · focus · HN ↗
        &gt; It was established terminology by 1973, as far as I can tell

        That&#x27;s a worthwhile observation. The 1973 article anyway doesn&#x27;t go into depth about comparative computer architecture. It&#x27;s also possible the term&#x27;s currency in computer science circles hadn&#x27;t yet crossed into Halmos&#x27; mathematics circles, but I wouldn&#x27;t know.

    2. adrian_b · · focus · HN ↗
      The conclusion is that von Neumann deserves the credit for the &quot;von Neumann architecture&quot;, despite any contrary claims of Eckert or Mauchly.

      Herman Goldstine, who distributed the document and John von Neumann, who wrote it, are the reason for the explosive growth of the computer industry during the following decades.

      The paper written by von Neumann was an exemplary model of clarity and good logical thinking. Nothing ever written by Eckert or Mauchly was at a comparable level.

      Everyone who read that paper understood immediately how to design and build an electronic automatic computer, and a great number of teams in many countries all over the world did precisely this, so a decade later there already existed experimental electronic computers in many countries and also commercial electronic computers in the UK and in USA.

      Some of the details from the paper must have been learned by von Neumann from discussions with the ENIAC team, but others were obviously von Neumann&#x27;s own ideas, e.g. the use for the main memory of an iconoscope tube with fast random access, i.e. a DRAM like today, instead of the slow serial delay lines chosen by Eckert and Mauchly for their following computers.

      Whatever von Neumann has learned from the ENIAC team, he obviously understood better than the people who taught him, allowing him to formulate general principles for the organization of an automatic computer.

      The publication of the von Neumann paper allowed the concurrent development of many computers in many places, and each of those projects found various improvements that were essential in making the electronic computers successful commercial products.

      If Eckert and Mauchly had succeeded to block any competitors, then the evolution of computers might have been delayed by up to 2 decades, until any patents would have expired, because Eckert and Mauchly have never done later any significant innovations and they would have never succeeded to develop better computers at the pace that happened due to the von Neumann paper.

      Moreover, Eckert and Mauchly had a history themselves of failing to mention their sources of inspiration, so they were not the people entitled to complain about someone &quot;stealing&quot; ideas from them.

      Before ENIAC, the first electronic computer in USA was the Atanasoff-Berry computer. That computer was a special-purpose computer, designed for solving systems of linear algebraic equations. However, ENIAC was a special-purpose computer too, unlike the relay-based computers that were older than it, like the Harvard Mark I computer, as even its name implies (Numerical Integrator and Computer).

      ENIAC was conceived as a replacement for the older mechanical &quot;differential analyzers&quot;, which were used to solve ordinary differential equations, e.g. for computing artillery tables, so its architecture mimicked the architecture of the mechanical differential analyzers, and it was reconfigured for new problems in a similar manner with those, by rewiring.

      John Vincent Atanasoff had written in 1940 a very high-quality document about the design of his computer: “Computing Machine for the Solution of Large Systems of Linear Algebraic Equations”. It is unknown if anyone of the ENIAC team had read it, but it is known that one of them had visited the designers of the Atanasoff-Berry computer, inquiring about the electronic circuits used by them to implement arithmetic operations and data storage. Later, during the design of ENIAC and after that, they never mentioned any connection with the earlier electronic computer.

      1. ricksunny · · focus · HN ↗
        Perhaps, and thanks for contributing that; and I&#x27;m not taking a position currently on this. But I&#x27;ve long been uncomfortable with the term &#x27;random access&#x27; for random access memory (RAM) as it implies that the concept of &#x27;randomness&#x27; is somehow a required and fundamental property of the memory technology described, when it&#x27;s not. &#x27;Arbitrary access&#x27; is a much more appropriately encapsulating description of the function of the memory at hand. Alas, the incumbency of terminology.
        1. adrian_b · · focus · HN ↗
          I agree with you about terminology.

          Nonetheless, the programs whose performance is limited by the speed of the main memory can be divided in programs whose performance is limited by the throughput of the memory interface and programs whose performance is limited by the latency of the memory accesses.

          The programs from the first class will have the same performance when using a serial memory or a RAM, if they are equally fast for sequential access, but for programs from the second class, it is important to have a RAM as the main memory, otherwise they would be very slow.

          In order to measure the performance achievable by programs from the second class on a given computer, it is necessary to run a benchmark where the addresses of the memory accesses are generated by a good random-number generator.

          The reason is that modern CPUs have a variety of hardware prefetchers that attempt to predict the next memory address, so for meaningful results that reflect the true performance of the memory you need truly random addresses that are impossible to guess by the hardware.

          Thus nowadays the latency of the memory accesses of a RAM cannot be measured otherwise than with really random accesses and a memory chip must be designed to have a low latency even when successive accesses happen at unpredictable random addresses, not only when the accesses are at arbitrary addresses, but those can be predicted in advance of the actual accesses.

          So there is an argument in favor of the term RAM, because random access is the worst possible case and the memory must still be able to handle it.

      2. HarHarVeryFunny · · focus · HN ↗
        Von Neumann&#x27;s architecture (i.e. report on the EDVAC) didn&#x27;t focus on random-access memory, and in fact his whole discussion on memory is rather rambling. The EDVAC itself, which the report is nominally describing, used serial-access mercury delay line memory. What really matters from an architectural POV is memory address-ability, not speed of access (random vs serial).

        The whole document is, as the title promises, really more of an engineering one - a report on the EDVAC - rather than some clean abstract treatise on computer architecture.

        <a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20130314123032&#x2F;http:&#x2F;&#x2F;qss.stanford.edu&#x2F;~godfrey&#x2F;vonNeumann&#x2F;vnedvac.pdf" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20130314123032&#x2F;http:&#x2F;&#x2F;qss.stanfo...

        1. somethinsfishy · · focus · HN ↗
          When the report was written there wasn&#x27;t any random access memory. The only memory suited to computers in 1946 was the ultrasonic delay line as implemented by Eckert. What makes it not-random access is that all the bits in the store are being constantly shifted and counted in a loop (about 1Mhz?), so you have to wait for the word you want to read&#x2F;write to fly past and the controller grabs it then.
          1. adrian_b · · focus · HN ↗
            The document written by Atanasoff in 1940, which I have mentioned in another comment, analyzed all the known devices suitable to be used as memories and it proposed the first kind of DRAM, made with discrete capacitors, as the best combination of low price and high speed.

            However, during the war this document was known by few.

            Delay lines were already used as analog memories in radars and from there Eckert and Mauchly took them and modified them to be used as digital memories.

            Von Neumann learned about the delay line memories from the ENIAC team, but he did not like the idea, because serial access seemed too slow, so he proposed to use instead of delay lines the second kind of DRAM, with a cathode-ray tube instead of discrete capacitors. Von Neumann took the idea of using a CRT as a memory from the tubes used in video cameras for television, which used an analog memory to store the image after exposure to light.

            So when the report was written, probably von Neumann was not aware of the earlier RAM used by Atanasoff in his computer, but he also proposed a RAM as being better suited for computers than delay lines. Von Neumann was proven right a few years later, when at Manchester Williams made such DRAMs with cathode-ray tubes, which were later used in many commercial computers, e.g. in the first generation of IBM computers, until the development of magnetic core memories made obsolete both the delay lines and the Williams tubes (until the magnetic cores were made themselves obsolete 20 years later, during the seventies, by the third kind of DRAM, with MOS integrated circuits).

            The most important users of delay line memories were the early UNIVAC computers, because they had bought the failed company of Eckert and Mauchly, for their IP.

            1. HarHarVeryFunny · · focus · HN ↗
              It&#x27;s interesting to read Von Neumann&#x27;s perspective on CRT (&quot;iconoscope&quot;) memory - saying how important it was not to lose the serial aspect of it since of course(!) you would not want to specify the address for every access, with most memory accesses being sequential (presumably meaning each bit of a 44-bit word), but it was advantageous to gain the speed of occasionally switching read location, which would be slower and more costly to do with delay lines (more, shorter, delay lines).

              Note that Atanasoff&#x27;s capacitor-based memory was based on rows of capacitors arranged around a mechanical rotating drum (a bit reminiscent of the later EDSAC&#x27;s &quot;initial orders&quot; module!), so it wasn&#x27;t true random access - there would be a rotational delay to get access to a given row. In a way it was as much a predecessor of rotating mass storage devices as it was of DRAM.

              I just missed the core memory era myself, at least as a user, learning to program (while in highschool) on an IBM mainframe in the mid-late 70&#x27;s, which was already using semiconductor memory. Still, years ago I bought a core memory module and a (very similar size) etched 6&quot; silicon wafer, intending to frame them side-by-side as the defining story of my era - the switch from physical to solid state integrated electronics.

              1. ricksunny · · focus · HN ↗
                finally, a bonafide non-LLLm (c. 2026) answer! :)
      3. tovej · · focus · HN ↗
        No, von Neumann does not deserve credit for stealing Eckerts and Mauchleys life work.

        von Neumann deserves credit for many things, but not this.

        He famously brought a lawyer with him to a meeting after they wanted to discuss the report and the rights of their invention. The whole ENIAC saga has made me reconsider von Neumann as a person. He was a talented mathematician, sure, but he seems to have also been a greedy scumbag.

        Eckert and Mauchley did not try to conceal their inspiration from Atanasoff, although I believe there was a court case on this topic.

        And it is completely ridiculous to claim that von Neumann would have been able to describe the computer better than its inventors.

        It&#x27;s eaually ridiculous to consider that they would not have published themselves. They were collaborating with von Neumann on the report after all.

        Goldstine and von Neumann stole the credit from Eckert and Mauchley, that is documented history.

      4. Animats · · focus · HN ↗
        As I point out occasionally, the bottleneck in early computing was memory. There were no good memory devices for quite a while. IBM had electronic arithmetic in test before WWII, and that eventually emerged as the IBM 603 Electronic Multiplier (1946). That was the first mass-produced electronic calculation device. From IBM&#x27;s perspective, they needed something reliable enough to install at customer sites that would run without repair techs on site. IBM&#x27;s electromechanical machines already worked that well, and businesses could get work done. So there was a lot of work that had to be done on tubes, sockets, wiring, soldering, and packaging before they could ship.

        Eckert and Mauchley came from a background of trying to get compute done with modified IBM hardware. They were painfully familiar with the lack-of-memory problem. Everybody involved with that era of machinery was. (I&#x27;ve used IBM plugboard wired tabulators, like the one at the Computer Museum in Mountain View. My high school had one.) The ENIAC had a workaround for lack of memory, huge rolling cabinets of manually set rotary switches. If something better had been available, it would have been used. You have to appreciate how much early architecture was restricted by available components.

        The first real hardware breakthrough was acoustic delay-line memory. First mercury tanks, then long coils of steel wire. Once that was available, it was clear that was the place to put the program. But it took a while to make that work reliably. Then there was Williams tube electrostatic storage, and high speed drums, which got things going. Sort of. Memory cost was an issue well into the 1980s. In the early 1970s, a megabyte cost a million dollars. Cheap memory didn&#x27;t appear until the 1990s.

        The big lack in Von Neumann&#x27;s architecture was index registers. He was into storing into the address part of instructions. The Manchester Mark I had the first index register (the &quot;B box&quot;), at which point programs could be read-only and still be able to index arrays. That completed the basic CPU architecture.

        1. ilamont · · focus · HN ↗
          &gt; I&#x27;ve used IBM plugboard wired tabulators

          This is a genuinely rare experience. Was it like old hardware on a shelf that no one knew what to do with and you tried it out? Or this was how lessons&#x2F;clubs&#x2F;assignments got things done, using an antique IBM tabulator?

          When I started public high school near Boston in the early 80s we didn&#x27;t have Apples or TRS 80s or PCs, it was terminals that used printed output ... can&#x27;t remember the details, maybe DECwriter to PDP 11.

          1. Animats · · focus · HN ↗
            It was obsolete even when I was in high school, but the school had the basic set of unit record equipment - tabulator, sorter, collator, keypunch. Computers hadn&#x27;t filtered down to high schools yet.
        2. ricksunny · · focus · HN ↗
          &gt;The big lack in Von Neumann&#x27;s architecture was index registers. He was into storing into the address part of instructions. The Manchester Mark I had the first index register (the &quot;B box&quot;), at which point programs could be read-only and still be able to index arrays. That completed the basic CPU architecture.

          strikes me as gilding the lily (I earlier even imagined possibly even a claude-esque stock &#x27;one caveat&#x27; flourish, but I know you wouldn&#x27;t do that given your background). Addressable memory, where any given addressed memory register can contain not just conventional data to be operated on but can also contain an address of some other memory register to jump to. is what strikes me as the highly-enriched uranium of computer architecture. indexing an array is a feature bump by comparison.

          1. adrian_b · · focus · HN ↗
            The early computers that did not have index registers realized the equivalent of indexed addressing by modifying the address fields of the instructions before they were executed, e.g. during a loop.

            In the early computers with non-pipelined execution and without cache memories, and also without security concerns, modifying the executable program during execution did not have disadvantages.

            Nonetheless, pipelined execution became widely used and also cache memories appeared soon, and such performance-enhancing techniques could not work well with self-modifying programs, so index registers became absolutely necessary in fast CPUs.

          2. Animats · · focus · HN ↗
            Being able to store into the program was once considered much more important than it is now. Today we almost always load code, lock it read-only, and run it. Even JIT compilers don&#x27;t patch much. They mostly generate code into empty memory. (There are exceptions. I think PyPy patches existing when a Python program replaces an existing function while running. You can do this in Python, but probably shouldn&#x27;t.) Von Neumann considered self-modifying code important for somewhat philosophical reasons related to his self-replicating automata ideas.

            Previous pre-computer programmable machines, back to the Jacquard loom, stored programs in a completely different medium than the data. Plugboards, chains of Jacquard cards, cams, etc. Von Neumann insisted that programs were just data and belonged in the same memory as the data. This was a conceptual advance. The hardware had to get there to make it work.

            The next big innovation was the stack, in the modern sense. This came later than one might expect. Note that there are at least two types of stack computers. There are ones with a return point and context stack, which is the C view of the world. And there are ones where working storage is on the stack, like Forth machines and early Burroughs machines. The first kind seems to have been invented for ALGOL in 1957, because ALGOL allowed recursion. Stacks made a lot of people nervous back then, because how much memory to allocate for the stack was unclear, and there wasn&#x27;t much memory. FORTRAN disallowed recursion and thus did not need stacks and stack frames.

            So storing into the program hung on in some architectures. UNIVAC 1107 and later mainframes had a Store Location Jump (SLJ) instruction. This stored the return address into the beginning of a subroutine. Not into a stack frame; there was no stack by default. Into the first word of the subroutine&#x27;s code. For non-recursive non-concurrent code, it worked.[1] But it was used less over time, with different constructs used instead.

            Program in the same memory as data, yes. Program can create more program, yes. Program modifies itself while running, not so good. Index registers eliminate most needs for the third case.

            [1] <a href="https:&#x2F;&#x2F;www.fourmilab.ch&#x2F;documents&#x2F;univac&#x2F;instructions.html" rel="nofollow">https:&#x2F;&#x2F;www.fourmilab.ch&#x2F;documents&#x2F;univac&#x2F;instructions.html

        3. adrian_b · · focus · HN ↗
          The Williams tube electrostatic storage was just the practical implementation of the proposal made by von Neumann in his report, where it was referred as an &quot;iconoscope tube memory&quot;.

          An iconoscope was a cathode-ray tube used in video cameras, where it allowed the reading of every pixel of an image, and where the value of the pixel consisted of the electric charge stored at that point (the amount of charge depended on the light exposure of a photoconductive plate). Williams did exactly what von Neumann had suggested, combining the reading of the memory like in an iconoscope tube with the writing of the memory like in a CRT display tube.

          The first commercial IBM fully-electronic computers, IBM 701 for scientific computing and IBM 702 for business data processing, used Williams tubes, according to the von Neumann proposal. The Williams tube memory of the IBM computers was an advantage over the Univac computers, which were still using the slow delay line memories invented by Eckert and Mauchly (which had been derived from the delay lines used as analog memories in radars, during the war).

          The next generation of IBM computers switched to the magnetic core memory invented at MIT, except for the cheapest computers, like IBM 650, which used magnetic drums for the main memory. IBM also invented soon the magnetic disks, as a memory tier intermediate between magnetic drums and magnetic tapes. The use of magnetic tapes had started in the first commercial electronic computer from USA, UNIVAC I.

          By the end of the fifties, some computers had 5 tiers of memory, from the fastest to the one with the greatest capacity: registers made with flip-flops, magnetic cores, magnetic drums, magnetic disks and magnetic tapes.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.