Design Build and Management of Planned Big Data Environments

Big Data is no longer the sole domain of large organizations (e.g. media, broadcasters, CERN, NASA, etc.). As organizations of all size begin to discover the business value of their data, there is a growing need to develop architectures for planned Big Data that are based around mainstream products and standardized architectures, ensuring a reasonable cost and faster time to market than currently possible.

In his Knowledge Sharing article, Alan Senior examines unstructured, planned data environments (currently, custom-built systems) where an organization’s need for data storage is known and understood, but has previously been impossible to meet, mainly due to the requirement for large capacity and high performance within a reasonable budget.

Based on experience gathered in real-world examples, this article looks at the requirements for a use case for an unstructured, planned Big Data requirement, including:

• Requirements for hardware, software, and deployment

• Requirements for data in use vs. data at rest

• Ingest requirement—where does your data come from?

• Data access requirements—who gets access to data?

• Retrieval of planned Big Data—searching across petabytes

• Where Big Data meets the cloud—linking external storage to planned Big Data systems

Once the use case is established, Alan looks at the potential concept architecture that is built around the industry and use case, which will then be fleshed out further as a product set using the EMC product portfolio where applicable. Throughout, he highlights gaps and proposes options.

Finally, Alan looks at where the concept architecture built around existing use cases can be applied across industries—points of commonality which can be exploited to limit redevelopment time and effort, while still ensuring the architecture remains relevant in the new industry.

Read the full article.