For the 16th meeting of the astronomy twitter journal club we were looking for ways to prevent us all drowning in data when upcoming instruments and surveys get going. The discussion was based on this paper, and you can also read our preview and the full transcript of the meeting.
Onto the review…
So, do you agree with the paper’s authors that we’re facing a ‘data tsunami’? – @astronomyjc
From what I’ve heard about LSST in particular: yes, a data tsunami might well be upcoming! – @MarcelAstroph
i think that the earthquake to trigger the tsunami hasn’t quite happened yet. things like WFIRST and LSST actually go on the sky.. actually I take that back. LOFAR, VISTA, etc are on the sky – @augustmuench
Yes data tsunami is at the doorstep. Probably each of us have couple of 10ths Gigs data on our own – @khanzadyan
But the increase from SDSS to LSST is slower than the (current) pace of Moore’s law…A quick plot w/ Python of the current data increase in optical surveys. The gray dashed line is Moore’s law. http://t.co/cHm1q3Ge – @GeertMcTwit
That looks convincing, but I’m still worried about the other wavelengths i.e radio, submm – @astronomyjc
I agree radio can produce a ridiculous amount of data if everything was recorded
Some day tho … – @GeertMcTwit
Are we making adequate preparations for it then? Will we be able to handle the data? – @astronomyjc
I think we mostly need new education and/or funding policies: either train or hire computer scientists. – @MarcelAstroph
I would say we need more pre-processing and final product. I’m sure these can be separated from the main archive – @khanzadyan
Training Computer Scientists with astro is fine by me
– @kashfarooq
Should we be investing more in projects like the Virtual Observatory which bring together multiple online datasets? – @astronomyjc
Most definitely yes! We need to know what data is out there to prevent multiple almost identical data-sets – @khanzadyan
Unified data archives can also be good for stimulating people to use archival data rather than expensive observatory time… – @MarcelAstroph
lots of people use 2MASS data. A good archive with documentation is key. – @johngizis
What’s the next step? Apply for funding? Introduce phd IT education? Buy more servers? ?? – @astronomyjc
would say collaborate in VO type of initiative. Would be more convincing for policy makers – @khanzadyan
I think training in astro is difficult enough. Why not team up w/ compsci? Let people do what they’re good at? Interdisciplinarity! – @MarcelAstroph
next steps: training? that requires buy in from advisors & students. what if students say, “we’ll figure it out on our own!” – @augustmuench
lots of developers are always looking for interesting open source projects to be involved with. – @kashfarooq
I’d say decent programming skills are as important a tool as maths – it’s time for Physics courses to include it! – @GeertMcTwit
The final word:
some would say & I agree that “computer science” != “scientific computing” we should teach the latter and partner with the former. – @augustmuench
