Storage architecture

The Milestone XProtect VMS media database and storage architecture have been designed to provide the best possible performance, reliability, security, and flexibility. This gives the XProtect VMS designer or administrator the freedom to choose a storage technology and architecture that fits the specific system needs, should the needs be performance, reliability, maintainability, size, cost, or a combination of these.

Storage and Media Databases

In the XProtect VMS, the recording and archive parameters are configured as a part of an overall storage configuration, which specifies the retention time, size, encryption etc. for the recording database and any optional archive databases.

Each device assigned to use the defined storage will have its own database in it where the recorded media is stored. This means that there are as many media databases in the XProtect VMS as there are enabled devices.

The media is continuously streamed over the network from the device to the recording server where it is initially stored in the (optional) pre-buffer which, when enabled, provides the ability to record some time before a recording triggering event occurs. See the ‘Pre-buffer’ section for details.

Once recording is triggered, the recordings are moved from the pre-buffer and written in the recording database. If no recording is triggered, data that is older than the set pre-buffer time is deleted. Once the data stored in the recording database is as old as the set retention time it is deleted.

In extension to the recording database, the storage definition supports a function called ‘archiving’, which is the process of moving the recordings at a set interval from the recording database to an archive database which is often stored at a different location. The archiving process can be repeated if needed. The archiving process also supports reducing the framerate of the recordings. See the ‘Reduce Framerate’ section for details.

No matter if pre-buffering and/or archiving are used, playback of recordings is completely transparent for the users of the XProtect VMS. The desired recordings are simply read by the media database from the location they are currently stored in, should it be the recording database or an archive database.

Pre-buffer

The purpose of the pre-buffer is to allow the XProtect VMS to record video, audio and metadata for a period leading up to an event triggering actual recording.

When enabled, the pre-buffer can work in two modes:

• Memory based

• Disk based

With memory-based pre-buffering, the media data is stored in the server’s RAM and only written to the recording database on the storage system when recording is triggered. This reduces the load, wear, and tear on the storage system.

The reduction is inversely proportional to the percentage of media being recorded. If for example, the XProtect VMS is configured to record always, there isn’t any reduction as everything needs to be recorded. Whereas with VMD triggered recording and VMD 20% of the time, the load, wear, and tear on the storage systems is reduced by 80%.

Memory pre-buffering is the default and preferred option but is limited to maximum 15 seconds. Should a pre-buffer longer than 15 seconds be needed, it must be disk based.

With disk-based pre-buffering, all media data is written directly to the storage system where it is stored for the set pre-buffer time. Once the set pre-buffer time has been exceeded, the recordings are either kept or deleted depending on if any recordings have been triggered for the period that they cover.

Because all media is written to the storage system and potentially deleted again (in case no recording has been triggered), the load on the storage system is permanently high when using disk-based pre-buffering.

Recording database

The recording database is where recordings are initially written and stored. The term recording is used because the database files on the disk are open and actively being edited when the live feed from the devices is recorded.

The recorded media stays in the recording database until the retention time is reached and is then deleted – unless an archive has been defined.

If an archive has been defined, the recordings are not deleted when the retention time is reached, but instead remain in the recording database until the archive process is started. Because of this, the recording database will at times hold recordings older than the set retention time when archiving is enabled.

As an example, the recording database is configured to 7 days retention, and the archive is scheduled to run once a day. This means that when the archive process starts, the recording database will hold 8 days of recordings: 7 days that should remain in the recording database, and 1 day which is about to be archived.

As mentioned earlier, recording many devices in parallel and in real-time to the recording databases causes a lot of non-sequential writing on the disks and storage system. Designing a storage system that is fast enough to handle this and large enough to store the recordings for the required time can be expensive, which is where archiving can be used to reduce the cost without compromising on performance.

Archive database

Archiving is not a requirement, but it can be used to optimize the overall cost and performance of the storage system. This is done by allowing a multi-tier storage architecture where live media is initially recorded to the first tier, and then later moved to the second tier.

The first tier can use smaller, faster but per Gigabyte more expensive disks, optimized for the non-sequential recording of live streamed data. The second tier can use larger, slower and per Gigabyte cheaper disks, optimized for storing the recorded media for the retention time needed.

The reason why the disks used for the archive database can be slower is that the archive process that moves the recordings from the recording database to the archive database does this with only a few cameras at a time. This makes the disk access mostly sequential, in which case even cheaper, larger, and slower disks perform well.

As with the recording database, the recordings remain in the archive until the retention time is reached, where the recordings are then deleted – unless a second archive has been configured, in which case the recordings remain in the archive database until moved to the next archive.

Note: The functions to archive the recordings more than one time are only available in XProtect Expert and XProtect Corporate.