Big data is everywhere in science today, opening new insights in fields from astrophysics to zoology. For all its promise, though, big data also poses a real danger to academic science, warns Jake VanderPlas, a postdoc in astronomy and computer science at the University of Washington, in an illuminating essay on his Pythonic Perambulationsblog. Specifically, VanderPlas believes that universities have to change their reward system or risk losing the very people whose expertise makes big-data research possible.
"Where scientific research is concerned, this recently accelerated shift to data-centric science has a dark side, which boils down to this: the skills required to be a successful scientific researcher are increasingly indistinguishable from the skills required to be successful in industry," VanderPlas writes. "While academia, with typical inertia, gradually shifts to accommodate this, the rest of the world has already begun to embrace and reward these skills to a much greater degree. The unfortunate result is that some of the most promising upcoming researchers are finding no place for themselves in the academic community, while the for-profit world of industry stands by with deep pockets and open arms." (Note that the emphasis is in the original.)
Software and the ability to write, test, and maintain it are fundamental to the new style of research, VanderPlas explains, and "the new breed of scientist must be a broadly-trained expert in statistics, in computing, in algorithm-building, in software design, and (perhaps as an afterthought) in domain knowledge as well."