There is some big news on Big Data from the National Institute of Standards and Technology (NIST).
NIST just published the final version of its new Big Data Framework.
What the NIST Big Data Framework accomplishes
Just like the NIST Cybersecurity Framework which is so prominent in cybersecurity, NIST spent years creating and refining this new framework. The end result is built with the help of 800 experts from industry, academia, and government.
Here is the power of the Big Data Interoperability Framework (NBDIF), according to NIST:
Filling nine volumes, the framework is intended to guide developers on how to deploy software tools that can analyze data using any type of computing platform, be it a single laptop or the most powerful cloud-based environment. Just as important, it can allow analysts to move their work from one platform to another and substitute a more advanced algorithm without retooling the computing environment.
"We want to enable data scientists to do effective work using whatever platform they choose or have available, and however their operation grows or changes," said Wo Chang, a NIST computer scientist and convener of one of the collaboration's working groups. "This framework is a reference for how to create an 'agnostic' environment for tool creation. If software vendors use the framework's guidelines when developing analytical tools, then analysts' results can flow uninterruptedly, even as their goals change and technology advances."
Security and privacy in NIST Big Data Framework
The focus of the Big Data Framework is helping data scientists deliver on the growing demands expected by their organizations.
However, it also covers topics highly valued by SecureWorld readers. Says NIST:
It also includes key requirements for data security and privacy protections that these tools should have. What is new in the final version is a reference architecture interface specification that will guide these tools' actual deployment.
This is fantastic, as we know that talking about security and privacy protection and implementing them are two completely different things.
And the NIST Security and Privacy Subgroup that tackled this part of the NBDIF also identified a number of ways that security and privacy in Big Data projects can be different from traditional implementations.
The number one difference they identified?
Big Data projects often encompass heterogeneous components in which a single security scheme has not been designed from the outset.
Speaking of NIST, we recently interviewed Tim Callahan, Global Chief Security Officer at Aflac. He's a huge fan of the NIST Cybersecurity Framework, and you can hear him explain why on the podcast:
Also, take a look at the new document itself: NIST Big Data Interoperability Framework.
