I have written something about big data in astronomy, but it has to do with how big the measurements are (in terms of miles) rather than the volume of data itself. Anyway, if interested, feel free to read my article "Can we use data science to measure distances to stars".
Thank you, I did read it.
I am interested to know of active projects in these science domains and the application of analytics there.
I am trying to understand if the analytics of big data is confined to the business & internet / social world only or if it has now spread across to other disciplines.
I see terms such as "bioinformatics" and "computational biologist", both of which did not exist before so was curious to the penetration of analytics into these domains.
1-enormous amount of data coming from the Large Hadron Collider
2-the analysis of large genomes
3-Earth observation platform focusing on earthquake and volcano research
Large Synoptic Survey Telescope (LSST).
"Over 30 thousand gigabytes (30TB) of images will be generated every night during the decade -long LSST sky survey. "
Fits very well the description of big data situation. A huge amount of data flowing in that has to be analyzed in near real time
to find what they are looking for, a super nova in progress so that they can zoom in on it for specific science exploration before it disappears.
I believe this group is using big data techniques also
"Scientists in the Broad community have developed many critical software tools for the analysis of increasingly large genome-related datasets, and they make these tools openly available to the scientific community. "
One cell. One division. Half a gig of data.
Many of the current telescopes today create many GB's of data per night. However, if you want to be overwhelmed with the numbers;
- ALMA Interferometer in Chile creates around 0.5-1 TB per day.
- LOFAR arrays creates around 500 TB per day.
- The ultimate-observing-machine, SKA (Square Kilometer Array) arrays and dishes will create around 700 TB per second when it is in full operation --probably later than 2025--. And imagine, it will observe the sky 24/7.