Google's newly announced Compute Engine is an infrastructure-as-service product that allows users to run hundreds of thousands of Linux Virtual Machines on Google's infrastructure.
Watch the sequence from Google I/O 2012 starting around 41:00 and particularly 45:00 to 47:00; Compute Engine head Urs Holzle describes how fast a computer can be when you spin up 600,000 cores to solve a problem from the Institute for Systems Biology. The Institute built a 1,000 core device to define relationships between genes in a genomic analysis.
(Google helps the Institute for Systems Biology)
SF writers love describing enormous computing systems; readers may recall the metal calculator planet from Clifford Simak's 1949 story Limiting Factor and the gigagnostotron from Stanislaw Lem's 1965 novel The Cyberiad: Fables for the Cybernetic Age. Also, the City Fathers from James Blish's 1957 novel Cities in Flight makes use of the notion of more than one hundred separate devices that work together.
The Machine itself stood on the leveled crest of a mountain.
It was a scintillating, silvery shaft rearing up into the sky...
People sometimes think that the electronic brain system of the Machine constitutes a development superior to that of man. They marvel at the Machine's capacity to handle twenty-five thousand individuals at once, but actually it can do so only because twenty-five thousand electronic brains were set up in intricate series for just that purpose... [my italics]
(Read more about van Vogt's games Machine)