Big Data

Two of the biggest trends in the world of big data stand somewhat in opposition to each other: the proliferation of big data that informs smart technology, and also the growing movement for consumers to own and decide how their personal data is being used. Technologies like IoT, artificial intelligence, machine learning, and even customer relationship management (CRM) databases collect terabytes of data that contain highly sensitive personal information. This personal form of big data is valuable for enterprises that want to better cater their products and services to their audience, but it also means that all companies and third-party vendors are held responsible for the ethical use and management of personal data.

As big data and its enterprise use cases continue to grow, most organizations work hard to comply with consumer data laws and regulations, but their security holes leave data vulnerable to breach. Take a look at some of the top trends happening in the big data world, the important security points that many companies are missing, and some tips for getting big data security right.


Big data security is a constant concern because Big Data deployments are valuable targets to would-be intruders. A single ransomware attack might leave your big data deployment subject to ransom demands. Even worse, an unauthorized user may gain access to your big data to siphon off and sell valuable information. The losses can be severe. Your IP may be spread everywhere to unauthorized buyers, you may suffer fines and judgments from regulators, and you can be hindered by big reputational losses.


Stage 1: Data Sources. Big data sources come from a variety of sources and data types. User-generated data alone can include CRM or ERM data, transactional and database data, and vast amounts of unstructured data such as email messages or social media posts. In addition to this, you have the whole world of machine-generated data including logs and sensors. You need to secure this data in transit, from sources to the platform.

Stage 2: Stored Data. Protecting stored data takes mature security toolsets including encryption at rest, strong user authentication, and intrusion protection and planning. You will also need to run your security toolsets across a distributed cluster platform with many servers and nodes. In addition, your security tools must protect log files and analytics tools as they operate inside the platform.

Stage 3: Output Data. The entire reason for the complexity and expense of the big data platform is being able to run meaningful analytics across massive data volumes and different types of data. These analytics output results to applications, reports, and dashboards. This extremely valuable intelligence makes for a rich target for intrusion, and it is critical to encrypt output as well as ingress. Also, secure compliance at this stage: make certain that results going out to end-users do not contain regulated data.