"Big Data" - a catchphrase for datasets that grow so large that they become awkward to work with. Big Data happens purposefully, when large data sets allow in-depth business analysis, but also due to the current ease in data collection. We collect vast stores of data simply because we can.
Aside from the technologies needed to process Big Data, technologies and processes are also needed to store and manage that data. Unfortunately, due to its size, Big Data management often fails to address many areas of information management - including security.
Gartner recently discussed this challenge, noting that:
"The business's demand for access to the vast resources of big data gives information managers an opportunity to alter the way the enterprise uses information. IT leaders must educate their business counterparts on the challenges while ensuring some degree of control and coordination so that the big-data opportunity doesn't become big-data chaos, which may raise compliance risks, increase costs and create yet more silos."
Is your organization using Big Data? If so, how are you managing these complexities?