“A new generation of sky surveys promises to catalog literally billions and billions of astronomical objects. Trouble is, there are not enough graduate students in the known universe to classify all of them. When the Large Synoptic Survey Telescope (LSST) in Cerro Pachón, Chile, aims its 3.2-
billion-pixel digital camera (the world’s largest) at the night sky in 2019, it will capture an area 49 times as large as the moon in each 15-second exposure, 2,000 times a night. Those snapshots will be stitched together over a decade to eventually form a motion picture of half the visible sky. The LSST, producing 30 terabytes of data nightly, will become the centerpiece of what some experts have dubbed the age of petascale astronomy—that’s 1015 bits (what Borne jokingly calls “a tonabytes”).
The data deluge is already overwhelming astronomers, who in the past endured fierce competition to get just a little observing time at a major observatory. “For the first time in history, we cannot examine all our data,” says George Djorgovski, an astronomy professor and codirector of the Center for Advanced Computing Research at Caltech. “It’s not just data volume. It’s also the quality and complexity. A major sky survey might detect millions or even billions of objects, and for each object we might measure thousands of attributes in a thousand dimensions. You can get a data-mining package off the shelf, but if you want to deal with a billion data vectors in a thousand dimensions, you’re out of luck even if you own the world’s biggest supercomputer. The challenge is to develop a new scientific methodology for the 21st century.” —When Astronomy Met Computer Science
The data deluge is already overwhelming astronomers, who in the past endured fierce competition to get just a little observing time at a major observatory. “For the first time in history, we cannot examine all our data,” says George Djorgovski, an astronomy professor and codirector of the Center for Advanced Computing Research at Caltech. “It’s not just data volume. It’s also the quality and complexity. A major sky survey might detect millions or even billions of objects, and for each object we might measure thousands of attributes in a thousand dimensions. You can get a data-mining package off the shelf, but if you want to deal with a billion data vectors in a thousand dimensions, you’re out of luck even if you own the world’s biggest supercomputer. The challenge is to develop a new scientific methodology for the 21st century.” —When Astronomy Met Computer Science
