“Billions and billions…”
It’s a phrase made famous by physicist Carl Sagan on his popular Cosmos TV show, and he was referring to the number of stars in our universe.
But it could also easily be applied to the bits of data NASA is collecting about that same universe.
It’s a daunting task — and with its dozens of missions and hundreds of scientists taken together, it may constitute the biggest big data project ever undertaken.
Source for picture: click here
NASA’s big data by the numbers
So how big is big? Just a few facts about the data NASA collects, stores, processes, and analyses today:
So how does it handle all that data? Here are some examples of what NASA is doing today.
Managing and Processing
NASA uses a system called the Mission Data Processing and Control System (MPCS) to manage and process data. It provides custom data visualizations that are used by the flight operations team, and does it all in real-time — a process that used to take hours, if not days to accomplish.
NASA has several storage centers for all this data, including the NASA Center for Climate Simulation (NCCS), which focuses on climate and weather data. It currently houses 32 petabytes of data, with a total capacity of 37 petabytes. It also has a one-of-a-kind 17-by-6-foot visualization wall, providing a single high-resolution surface for scientists to display data visualisations.
Archiving and Distribution
One example of how NASA processes and archives all this data is the Planetary Data System (PDS), focused solely on planetary science. It archives and distributes all data from NASA planetary missions, astronomical observations, and laboratory measurements into a single website and offers access to more than 100 TB of space images, telemetry, models, and anything else associated with planetary missions from the past 30 years.
Commercial cloud computing services
For its cloud computing, NASA is using commercial services just like any other company. For the recent Mars Science Laboratory mission, NASA migrated its legacy content management system and websites to Amazon Web Services, which needed to be able to deliver more than 150 Gigabits per second of traffic to a global team of operators, scientists, and the general public. As the data came in, every image from Mars was uploaded, processed, stored, and delivered from the cloud.
And all this is really the tip of the iceberg. Luckily, NASA understands the size and complexity of its probable future data needs and is already planning for a data-heavy future.