File storage#

This section describes how the asset store abstraction layer stores the physical files underlying the ORM, and explains some of the considerations.

Component overview#

The assets module is composed of these major storage classes:

• File: This is the main DataObject that user code interacts with when working with files. This class has the following subclasses:

  • Folder: Logical folder which holds a set of files. These can be nested.
  • Image: Specialisation of File representing an image which can be resized. Note that this does not include non-resizable image files.
• DBFile: This is the DB field used by the File dataobject internally for storing references to physical files in the asset backend.
• FlysystemAssetStore: The default backend, provided by Flysystem, which Silverstripe CMS uses as an asset persistence layer.
• InterventionBackend: Default image resizing mechanism, provided by intervention image.

These interfaces are also provided to abstract certain behaviour:

• AssetContainer: Abstract interface for a file reference. Implemented by both File and DBFile. Declare API for reading to and writing an single file.
• AssetStore: Abstract interface for the backend store for the asset system. Implemented by FlysystemAssetStore. Declares API for reading and writing assets from and to the store.

Storage via database columns#

Asset storage is provided out of the box via a Flysystem backend store. However, any class that implements the AssetStore interface could be substituted to provide storage backends via other mechanisms.

Internally, files are stored as DBFile records on the rows of parent objects. These records are composite fields which contain sufficient information useful to the configured asset backend in order to store, manage, and publish files. By default this composite field behind this field stores the following details:

Note that the Hash and Filename always point to the original file, if a Variant is specified. It is up to the storage backend to determine how variants are managed.

Note that the storage backend used will not be automatically synchronised with the database. Only files which are loaded into the backend through the asset API will be available for use within a site.

Flysystem adapters#

The Flysystem backend includes several optional adapters for storing files in different ways. By default the local filesystem adapter is used, which stores files in your local filesystem in the path that the ASSETS_PATH constant resolves to (usually public/assets/ relative to your project root).

If you chose to use a different adapter for your project, note that there may be performance implications for doing so:

• We use glob pattern matching where possible to look for image variants. If the adapter you've chosen can support glob pattern matching, consider subclassing it and implementing the GlobContentLister interface. This dramatically improves performance. The alternative behaviour is to iterate through all files in a folder to check if they're a variant file or not.
• Flysystem doesn't implement a way to check if a folder is empty. Filesystem::isEmpty() tries to get around this by only checking the first item in a generator returned by the adapter, but some adapters will still fetch data about many more files than is necessary e.g. from a networked service.

Public file paths#

Public files are published either directly through the "Assets" CMS UI, or indirectly as part of a versioned ownership structure. They are stored as you'd expect on the filesystem: In their folder, by their file name.

Copyassets/
    my-public-folder/
        my-public-file.jpg

The URL for this file will match the physical location on disk: https://www.example.com/assets/my-public-folder/my-public-file.jpg.

Variant file paths#

Each file can have variants.

A variant is a manipulated version of the original file - for example if you resize an image or convert a file to another format, this will generate a variant (leaving the original file intact).

These can be generated by resizing an image in the CMS rich text editor, through template logic, or programmatically with PHP. They are stored in the same folder alongside the original file, but contain a special variant suffix.

Copyassets/
    my-public-folder/
        my-public-file.jpg
        my-public-file__FitWzYwLDYwXQ.jpg

The variant suffix is usually comprised of the name of the method that created it followed by a base64 encoding of settings arguments to that method. It's separated from the original file name with FileIDHelper::VARIANT_SEPARATOR unless a custom FileIDHelper implementation is used.

The URL for the variant file will match the physical location on disk: https://www.example.com/assets/my-public-folder/my-public-file__FitWzYwLDYwXQ.jpg.

Protected file paths#

Uploaded files are protected by default, which puts them in a "draft" mode that requires permissions to view them. Protected files can also be published but access restricted. In either case, they're stored in a special assets/.protected folder. In this case, they're stored in a folder matching the truncated hash of the file's content.

Copyassets/
    my-public-folder/
        my-public-file.jpg
        my-public-file__FitWzYwLDYwXQ.jpg
    .protected/
        my-protected-folder/
            b63923d8d4/
                my-protected-file.jpg
                my-protected-file__FitWzYwLDYwXQ.jpg

This corresponds to a file with the following properties:

• Filename: my-protected-folder/my-protected-file-hash/my-protected-file.jpg
• Hash: b63923d8d4089c9da16fbcbcdfef3e1b24806334 (trimmed to first 10 chars)
• Variant: FitWzYwLDYwXQ (corresponds to Fit[60,60])

The URL for this file will not match the physical location on disk. It leaves out the .protected/ folder, and leaves that to Silverstripe CMS's integrated routing: https://www.example.com/assets/my-protected-folder/b63923d8d4/my-protected-file.jpg.

For more information on how protected files are stored see the file security section. We recommend moving this folder outside of the webroot, see Server Requirements: Secure Assets.

Versioned file paths#

Older versions of file contents are kept in the .protected folder, following the same rules as protected file paths.

Copyassets/
    my-file.jpg <- current published file
    .protected/
        dec83f348d/ <- old content hash of replaced file version
            my-file.jpg
        b63923d8d4/ <- old content hash of replaced file version
            my-file.jpg

Versioned and archived files#

By default, when files are replaced or removed, their original file contents aren't retained in order to avoid bloat on the filesystem. Changes are only tracked for file metadata (e.g. the Title attribute).

You can opt-in to retaining the file content for replaced or removed files.

CopySilverStripe\Assets\File:
  keep_archived_assets: true

The filesystem structure follows the same rules as protected file paths:

Copyassets/
    my-file.jpg <- current published file
    .protected/
        dec83f348d/ <- old content hash of replaced file version
            my-file.jpg
        b63923d8d4/ <- old content hash of replaced file version
            my-file.jpg

Loading content into DBFile#

A file can be written to the backend from a file which exists on the local filesystem (but not necessarily within the assets folder).

For example, to load a temporary file into a DataObject you could use the below:

Copynamespace App\Model;

use SilverStripe\ORM\DataObject;

class Banner extends DataObject
{
    private static $db = [
        'Image' => 'DBFile',
    ];
}

Copyuse App\Model\Banner;

// Image could be assigned in other parts of the code using the below
$banner = Banner::create();
$banner->Image->setFromLocalFile($tempfile['path'], 'my-folder/my-file.jpg');