Blog Date: 10/2/2012
Author: Ray Coulombe
Terabytes, petabytes, exabytes, zettabytes, brontobytes...what’s our security world coming to?
When you combine the trends of more surveillance cameras, higher bandwidth requirements, fatter bandwidth pipes and the continuing decline in storage costs, you get near-exponential growth in bytes stored. The question is, how do we manage this vast amount of video data? I’ve discussed this question with several large-scale storage vendors recently and learned some very interesting things.
First, there are well-developed content management systems available; however, managing “regular†data is different from managing video data — particularly streaming surveillance video. Whereas enterprise data consists mainly of small random blocks that are primarily reads rather than writes, video data are large blocks of data that are written far more than they are read. Further, you must organize the files and handle Metadata (tags) that go beyond date and time — a task that VMS and video storage vendors deal with on a regular basis.
This helps explain why those that can tag — such as analytics providers, and high volume storage providers — regularly seek to partner with VMS suppliers, who are the gateway between users and the external storage connected to the network (NAS, DAS, SAN). Innovation and implementation success occurs around these relationships. The manner in which the information is marked, cataloged, and shipped off to storage will directly affect the ease and speed of pulling it back out for review.
Most video is stored for a limited period (usually less than 30 days). It could be the user’s situation or needs do not demand anything longer, but the decision may also be driven by storage costs. In a prior product generation, the digital tape “jukebox†was an answer to long-term archival — now, tape may be back.
New entrants to the industry, such as Crossroads Systems, working with tape library provider Spectra Logic, promote a technology based on LTFS (Linear Tape File System) to create a storage center that sits on the network, looks like a network device, holds a ton of video, and costs about 80 percent less than hard drives, with better reliability, higher density, and less energy consumed. Tape capacity is similar to SATA drives, 1.5 or 2.5 TB.
When provisioning storage, take a long look at Total Cost of Ownership (TCO), as well as performance and redundancy. Consider not only acquisition costs, but electrical usage, heat generation, space requirements and expandability.A petabyte (PB) is 1000 trillion bytes (1000 TB) of data, and a storage system for this much data costs $750,000 to $2 million vs. $200,000 for a tape-based system. In case you were wondering, a brontobyte is a trillion PB (27 zeroes).
Link to Complete Article as it appeared in Security Technology Executive Magazine