Archive

While a backup is a copy of the data, an archive is a selected record of the original, that is moved elsewhere from its initial location, for long-term storage and future reference. Archives contain static data that is no longer actively used. They are usually not meant for updating or deleting, thus are not involved in the medialoopster housekeeping as well. Since archives tend to pile up, searchability is more important than a speedy recovery. Just like for non-archives, medialoopster stores metadata about archived assets, indexes them in the database and allows searches via the browser user-supplied search-options.

Medialoopster uses 3 archive methods, also called engines:

• Filesystem

  • saves the archive on a local, share, mount or LAN folder

• LTFS

  • moves data to tape using the LTFS data format and file system 

• Archiware P5

  • moves data to disk and tape, including SAN and offsite storage

Add an archive connector

When you archive assets/projects, first make sure that you have at least one archive connector in the system. When you add a new archive connector give it a name, choose the engine and add JSON encoded positional and keyword arguments. The arguments differ for the 3 types or engines:

Local/"Filesystem" arguments

{
	"archive_path": "/path/to/archive/dir/",
	"debug": false
}

LTFS arguments

{
  "connection": {
    "srv": "127.0.0.1",
    "port": 13373,
    "tape_base": "/path/to/base/dir/",
    "staging_base": "/path/to/staging/dir/",
    "index_data_secure_size": 32,
    "exclude_tape_list": [],
  }
  "debug": true
}

Archiware P5 arguments

{
  "irs_connection": "server:port",
  "awps_cli": {
    "pscli_prog": "/path/to/nsdchat",
    "pscli_nsdchat_session_id": "Archiware_01",
    "pscli_connection": "user:pass@server:port",
    "pscli_relocation_path": "",
    "pscli_time_zone_out": "UTC",
    "pscli_asset_id_meta_key": "asset_id",
    "pscli_store_callback_script": "/path/to/remote/_aw_store_callback.py",
    "pscli_restore_callback_script": "/path/to/remote/_aw_restore_callback.py",
    "pscli_callback_data_dir_local": "/path/to/local/pscli_callback_data/dir",
    "pscli_callback_data_dir_remote": "/path/to/remote/pscli_callback_data/dir",
    "medialoopster_meta_keys": {
      "asset_path": [
        "path",
        "value"
      ],
      "asset_id": [
        "char",
        "value"
      ],
      "content_type_id": [
        "char",
        "value"
      ]
    }
  },
  "archiveindex": "Default-Archive",
  "archiveplan": "10001",
  "archiveclient": "localhost",
  "debug": true
}

Add an archive device

Secondly, make sure that the path of your achive connector is assigned to one device. Add a new device for your path /mnt/medialoopster/Watchfolder/test-archive, give it a name and a display name, set "Is archive" to trueand check "Allow direct writing". 

Assign the archive connector to devices

Thirdly, for each Device where you want to archive assets from , pick an archive connector at "Archive connector".

Send to and restore from archive

Only after the previous setups steps are done, the archive icons under each asset or project in the browser user interface will become visible and enabled for users with the archive_<type>_asset / restore_<type>_asset permissions:

• "Send to archive" for assets/projects that are not-archived and are "Ready for production" (online, "is in production" checked, assets have an essence). The store request goes to a waiting list and gets handled via periodic tasks only:
  → A copy of the assets' essence is sent to the archive
  → The assets/projects are marked "Stored in archive" and:
  • • "Ready for production": if "delete_afterwards" was true (see  below)
    • "Not ready for production": if "delete_afterwards" was false (and delete the assets's essence)
• "Restore from archive" for assets/projects that are archived and "Not ready for production" (offline, "is in production" not checked, assets do not have an essence). The restore executes immediately:
  → The essence is copied from the archive to the initial HighRes path
  → The asset remains "Stored in archive" and is marked "Ready for production"

Archiving a project involves sending/restoring from/to archive the assets from its Audio/Image/Video assets fields as well.

The "is in production" (corresponding to online) and "is in archive" fields should not be tampered manually. They are set automatically via the archive actions.

Add a periodic archiving task

To execute the archiving store requests from the waiting list, you must add a periodic task. Give it a name, choose the registered task archive.tasks.start_archiving, keep the checkbox "enabled" checked, give it an interval or a cron time format, and add the following JSON configuration:

{
	"archive_connector_name": "archive_name",
	"delete_afterwards": true
}

Records of archive activities

A record of all archive store activities are automatically created at "send to archive" requests in the administration and management interface . The activities are first set to "Waiting", then changed to "Successfully archived" after a periodic task takes them over. 

A record of all archive restore activities are automatically created at "restore from archive" requests in the administration interface only (not in the management interface).

In the administration interface the models should be used only for visualizing. 

Archives and housekeeping

After restoring from archive, the asset's and project's delete date is reset to today + the hold time in weeks, according to the rules from the medialoopster housekeeping.