XProtect VMS data vs. IT data

Before looking at the details of the Milestone XProtect VMS media database, storage architecture and storage system technologies, it is important to understand the nature of XProtect VMS media data in the form of video, audio or metadata being streamed, recorded, and stored in real-time.

At first glance, without knowing the details of how an XProtect VMS works, an IT system designer or administrator will often think of XProtect VMS media data as being the same as standard IT data like documents or picture files stored on a corporate network share - with the amount of data being the only difference.

If considering a static XProtect VMS that is not recording, this belief is correct. In that case the VMS media database is just files stored in folders.

However, an XProtect VMS is never static. Every single second, the XProtect VMS writes new media data retrieved from cameras and other devices to the storage system in real-time. Furthermore, media older than the set retention time is also constantly being archived to a new location or deleted.

Because of the real-time operation, the load on the storage system will be of a very different nature in an XProtect VMS compared to standard file shares in an IT system. If this is not understood and properly accounted for during the design of the storage system for the XProtect VMS, it will result in performance issues and loss of recorded media data.

Standard IT storage system usage

Looking at the storage part of a standard distributed IT system, it typically consists of several servers and/or NAS hosting file shares from an underlying storage system as well as users and other servers/services accessing (reading/writing/deleting) the files stored on the file shares.

When files in a standard IT system are written from, for example, a user’s PC to a file share, it is not especially critical how fast the underlying storage system is from a data viewpoint, or in other words, how much time it takes to write the data. The reason for this is that the data that should be written is buffered at the source (for instance a user’s PC) while being written to the storage system.

This means that data is not lost even if it takes some time to write it. Furthermore, the data is only transferred between source and destination when the destination is ready to receive more data. So even though a slow storage system can be annoying for users, data is never lost.

Furthermore, only a smaller number of files are typically being read or written on the storage system at the same time, making the read/write operation mostly sequential, which gives a better performance for most storage technologies.

XProtect VMS storage system usage

Writing XProtect VMS media data to a storage system is very different compared to writing files in a standard IT system. This is mainly because in the XProtect VMS, cameras and other devices constantly produce new video, audio, and metadata in real time, which needs to be sent to the XProtect VMS where it is recorded and viewed live.

Should the recording server and/or storage system be too slow to handle the media data in real-time, the standard IT transmission control and buffering mechanisms cannot be used to buffer, pause, or throttle the transfer of data from the cameras, as this would create gaps in the recordings and cause video and audio to be paused and delayed when viewed live in the XProtect VMS clients.

In addition to the media data being received in real-time and not being able to be buffered or paused, the recording server typically records media data from many sources at the same time. This means that the media data from all cameras, microphones, speakers, and metadata sources on the recording server is written in parallel to multiple files on the storage system.

Writing multiple files in parallel is called non-sequential or random writing and causes most standard storage systems to perform much slower compared to when writing data sequentially, which is common to standard IT systems where only one or a few files are read or written at a time.

Finally, in extension to the process of recording the media data, the media data is simultaneously deleted again after the set retention time, or alternatively archived to another storage system and then deleted after the set retention time.

Summary

The nature of live XProtect VMS data and IT data is very different, and thus the performance requirements are also very different which needs to be considered when choosing a storage system for the XProtect VMS.

If the XProtect VMS system designer does not take the real-time nature of the VMS data into account, but instead dimensions the storage system based on standard IT needs, it will result in issues in the XProtect VMS with missing recordings and slow playback performance.