6.1.0 (unreleased)#

Overview#

• Features and enhancements

  • Optimisation of database queries for common CMS operations
  • Improved performance working with assets
  • Database query caching
  • Generated column support
  • Composite indexes for default_sort
  • Drop indexes in indexes configuration
  • New indexes for various database tables
  • Non-blocking session handlers
  • Password strength feedback
  • Other new features and enhancements
• Performance documentation
• API changes
• Bug fixes

Features and enhancements#

The primary focus of this release has been on improving performance of the CMS, which will result in a smoother and more responsive experience for users.

Optimisation of database queries for common CMS operations#

We've optimised several database queries used for common operations within the CMS.

They were done in a couple of ways:

• combining multiple WHERE "ID" = <id> style SQL statements into more efficient WHERE "ID" IN (<id>, <id>, <id>, ...) queries
• temporarily caching database results in memory for the current HTTP requests.

These optimisations can significantly reduce the number of database requests made, which will be particularly beneficial if your web-hosting setup has the webserver on a different physical server from the database, as there will be a degree of latency for every database request made, no matter the size of the actual query.

The following operations had database optimisation applied to them and should now perform better:

• Viewing files in asset-admin i.e. /admin/assets
• Viewing records in versioned-admin i.e. /admin/archive
• Viewing records in a GridField

Improved performance working with assets#

In projects with thousands of files in the same folder, especially with images, performance can be slow for some operations. That performance hit scales with the number of files in the folder you're dealing with. One reason for this is that Flysystem (the filesystem abstraction layer) can't assume the filesystem it's interfacing with has a way to discover if a folder is empty, or find files in a folder matching a certain pattern. This means that any time either of those operations was performed on a folder, Flysystem had to loop over every file it contained.

To remedy this, we've made the following changes:

• Reduced the number of times we check if a folder is empty by removing duplicate checks.
• Added Filesystem::isEmpty() to check if a folder is empty. With the default local filesystem adapter, this means we check exactly one file to know if the folder is empty or not. With other adapters it depends on the implementation (e.g. the AWS S3 adapter will likely still fetch a full page of up to 1000 files).
• Added GlobContentLister and GlobbableFileIDHelper interfaces to enable looking for image variants using a glob pattern. This dramatically improves performance with the default local filesystem adapter, but by default will have no effect for other adapters. If you are using a different adapter that might support globbing such as using FTP (for specific FTP servers), consider implementing the GlobContentLister interface. If you use your own implementation of FileIDHelper consider also implementing the GlobbableFileIDHelper interface.

Database query caching#

Queries made using a DataList or SQLSelect can now be cached. The cached query result lasts until the end of the request, unless invalidated during that request.

This allows you to cache the results of repeated database queries all over your application, even if they're custom queries you write yourself.

To enable caching on a DataList, call the DataList::setUseCache method.

Copyuse App\Model\MyDataObject;

$cachedList = MyDataObject::get()->setUseCache(true);

To enable caching on a SQLSelect, call the SQLSelect::setUseCache method.

Copyuse SilverStripe\ORM\Queries\SQLSelect;

$cachedQuery = SQLSelect::create(/*...*/)->setUseCache(true, 'some-namespace');

As part of this change, the DataObject::get_by_id(), DataObject::get_one(), and DataObject::delete_by_id() methods have been deprecated. Usage in supported modules of these methods have been replaced with using a cached DataList instead.

See caching database queries in the performance documentation for more details.

Generated column support#

Generated columns are database columns where the value is generated inside the database, rather than being set by a user. They're usually based on other columns in the database and can be either generated when the record is updated and stored, or generated when requested in which case they aren't stored.

Some good use cases for generated columns include:

• sorting in a gridfield on a complex summary field: It's common to use a getter method to get some value derived from your database fields (e.g. a discounted price), and use that in summary_fields. With a generated column, you can remove the getter method and the GridFieldSortableHeader component will be able to sort using the column.
• using functional indexes: generated columns can be included in indexes, allowing you to sort or filter by complex expressions in an efficient way.
• data integrity: unlike generating values inside an onBeforeWrite() method, generated column values will be correct even if you update the record with raw SQL. You also don't need to manually update values for historic records (e.g. when using versioning) even if you change the logic that determines the value.
• reduce repetition: instead of using a complex expression in multiple different places, you can just give the expression a name with a generated column.

See data types and casting for details about using generated columns.

Support in other SQL servers#

Generated column support has been added to the MySQL database connector in silverstripe/framework, but is not guaranteed to work for other database servers.

If you maintain a module that adds support for another database server, you'll need to implement the new DBSchemaManager::makeGenerated() and DBSchemaManager::needRebuildColumn() methods to support generated columns.

You likely also need to adjust your implementation of DBSchemaManager::alterTable() to drop and recreate columns (instead of just updating their schema in place) based on $advancedOptions['rebuildCols'], and DBSchemaManager::fieldList() to normalise the representation of generated columns.

Composite indexes for default_sort#

The ORM automatically creates an index for each column in the default_sort configuration for your DataObject subclasses. This index might only be useful for part of the sort, or in some cases ignored entirely if the database server decides it will be faster to just traverse the whole table itself.

To remedy this, a new DataObject.default_sort_index_mode configuration property has been added, which by default will tell the ORM to create a composite index (if you're sorting by multiple fields) in addition to the individual column indexes it was already making.

When created, the composite index will always be called default_sort_composite.

There are some caveats to this, along with some other options you might want to set, so check out the indexes documentation for more details.

Drop indexes in indexes configuration#

If you define some database index in the indexes configuration for a DataObject model and then remove that index from the configuration, the ORM won't drop that index for you. This means you can have indexes taking up space in your database that you don't want anymore.

Now you can set the value of the index to false instead of just removing it from the configuration, which will tell the ORM to drop the index. This can also be used to tell the ORM to not create indexes it would otherwise create by default (e.g. for relation joins). In general you should leave those alone unless you know what you're doing and intend to fine-tune your indexes to suit your specific project needs.

If you maintain an alternative database connector module (such as for postgresql) and it has a subclass of DBSchemaManager, you may need to update the alterTable() method to handle the case where the index spec is ['drop' => true].

If you are using the DataObjectSchema::databaseIndexes() method (or using its output downstream), note that the values in the array may now be false representing an index that should not be created. If you were relying on the keys in that array to tell you about the names of existing indexes, you will need to filter out the false values first.

See the indexes documentation for more details.

New indexes for various database tables#

A number of new indexes are added as part of this release. Along with the composite indexes for default_sort mentioned above, the following indexes have also been added:

• If a default_sort has been set on the join table for a many_many relation (as mentioned in relations between records), those columns will be added to appropriate indexes where possible.
• AutoLoginHash, AutoLoginTempHash, and TempIDHash on the Member table. These are used for the "keep me signed in" and CMS re-authentication features.
• Name on the TaxonomyTerm and TaxonomyType tables.
• Token on the ShareToken table.

Note that building indexes for large tables may result in a noticable increase in the time it takes to run sake db:build the first time after upgrading your project.

Non-blocking session handlers#

The default file-based session handler for PHP holds a lock on the session file while the session is open. This means that multiple concurrent requests from the same user have to wait for one another to finish processing after a session has been started. This includes AJAX requests.

To resolve this problem, we have provided three new session save handlers. Each of these are non-blocking which means that multiple concurrent requests from the same user don't have to wait for one another to finish.

You can choose how sessions are handled by setting the Session.save_handler configuration property or the SS_SESSION_SAVE_HANDLER_CLASS environment variable to the FQCN of your preferred save handler that implements SessionHandlerInterface.

Note that in edge case scenarios, for example if your application wants to modify a session value based on the value that is already set and must do so for each request, non-blocking sessions may cause unexpected results.

If you want to use the default blocking PHP file session handler instead, you can set the Session.save_handler configuration to null.

CopySilverStripe\Control\Session:
  save_handler: null

See the save handler documentation for more information about the new session save handlers and how to use the new configuration.

Note that if you are using silverstripe/hybridsessions or silverstripe/dynamodb, you can continue to use these with no change, provided you update them to the latest version.

If you want this functionality in your CMS 5 projects, we have created a new silverstripe/non-blocking-sessions module that backports the file-based non-blocking session save handler to CMS 5.

Password strength feedback#

The ConfirmedPasswordField now provides real-time feedback on password strength as users type. You'll see this in action on the member edit form, when setting a new password during the "lost password" process, and for any usages of ConfirmedPasswordField in your own forms if requireStrongPassword is toggled on.

In Member edit forms and the "lost password" process, this feature is active only when an EntropyPasswordValidator is in use for password validation. If you're using the alternative RulesPasswordValidator or another password validator, this strength indicator won't appear. The strength itself is assessed by Symfony's PasswordStrength validation constraint.

As part of this work, ChangePasswordForm now uses a ConfirmedPasswordField instead of separate PasswordField instances. Consequently, the ChangePasswordHandler that processes form submissions has been updated to expect a different data structure. If your site includes any custom modifications to the change password flow, please be aware of this change. The same is also true for the reset account form created and processed in the silverstripe/mfa module in SecurityExtension.

This functionality will not work in your front-end forms out of the box as the required JavaScript will not be available on the frontend theme. If your site does not have the silverstripe/login-forms module installed, then this functionality will not work for the "Lost password" flow out of the box as the required JavaScript will not be available on the frontend theme. This JavaScript can be found in the login-forms module and can be manually added to your frontend theme.

Other new features and enhancements#

• New DataList::filterByList() and DataList::excludeByList() methods allow filtering by or excluding based on the results of another list. See filterByList() and excludeByList() for details.
• New DataObjectSchema::tablesAreReadyForClass() method added for checking if the database is ready for queries for a given DataObject subclass.
• New FileIDHelper::VARIANT_SEPARATOR improves discoverability of the string that separates variant names from original file names in the asset system.
• When executing commands via the Sake CLI application, both the Sake instance and the Symfony\Component\Console\Command\Command instance are added to the dependency injector. See accessing sake from outside a command for more details.
• The File class now has a IsFolder generated column which is included in a new index. This makes sorting files in the asset admin faster.
• Previously there were a large number of unnecessary AJAX requests made to fetch the form schema for the search form for sections of the CMS that are searchable such as the site tree i.e. the list of pages on /admin/page. This has been fixed so these requests are only made when the filter button is clicked. Note this enhancement was originally released as a patch for CMS 5.4.
• There have been a number of other smaller performance enhancements that have been included in this release. Some of these enhacements were also released as patch releases for CMS 5.4.
• Inside your cache directory, sub-directories now include the PHP version, e.g. my-user-8.3.23/. Previously the PHP version was not included. If you do not have a silverstripe-cache/ directory in your project root, the parent cache directory that gets created no longer contains the PHP version (e.g. it will be /tmp/silverstripe-cache-var-www-html/ where it used to be /tmp/silverstripe-cache-php8.3.23-var-www-html/).

Performance documentation#

The performance documentation section has been updated, with some new sections and additional tips that might help make your project more performant.

API changes#

Deprecated API#

The following API has been deprecated in this release, and will be removed or replaced in a future major release.

It's best practice to avoid using deprecated code where possible, but sometimes it's unavoidable. This API will all continue to be available at least until the next major release.

• The Session.session_store_path configuration property has been deprecated. Use session.save_path in ini configuration instead.
• The Session.sessionCacheLimiter has been deprecated and will be removed without equivalent functionality to replace it in a future major release.
• The DataObject::get_by_id() method has been deprecated. Use DataObject::get($className)->setUseCache(true)->byID($id) instead.
• The DataObject::get_one() method has been deprecated. Use DataObject::get($className)->setUseCache(true)->first() instead.
• The DataObject::delete_by_id() method has been deprecated. Use DataObject::get($className)->setUseCache(true)->byID($id)->delete() instead.

Bug fixes#