Updating the Piano Sonata Ensemble

I have been using the Piano Sonata ensemble for my personal blog. It needs updating: the site is still running on Symphony 2.0.6 and the content hasn’t been updated for quite a while.

Updating the Symphony Core

I copied the files locally to ~/Sites/domains/stephenbau/ and updated the .htaccess and manifest/config.php files, pointing to the MySQL database I created on my local MySQL server.

This will be my legacy copy that I will use to compare working code and functionality as I inevitably break the updated copy. One goal I have in updating the site is to have an ensemble that uses the most recent core and remove dependencies on extensions other than the core extensions, if possible.

I will do my experimenting on another install. So I copied the site to ~/Sites/domains/lab/ and updated the .htaccess to use a different root URL.

              RewriteBase /domains/lab/

I have set up my localhost using the default Apache server on Mac OS X with an additional virtualhost setting for my ~/Sites directory. By default, this would point to http://localhost/~stephen/. Instead, I prefer to simply go to http://home/.

In /etc/apache2/users/, I have modified the stephen.conf file to include the following directives:

              NameVirtualHost *:80
NameVirtualHost 127.0.0.1

<Directory "/Users/stephen/Sites">
    Options Indexes FollowSymLinks MultiViews
    AllowOverride All
    Order allow,deny
    Allow from all
    DirectoryIndex index.html index.php
</Directory>

<VirtualHost 127.0.0.1>
    ServerName home
    DocumentRoot /Users/stephen/Sites
</VirtualHost>

The /etc/hosts file also includes a reference to the virtual host:

              #### Sites
127.0.0.1       home

After restarting Apache

              sudo apachectl graceful

I am now able to access my ~/Sites directory in a browser by going to http://home/.

Now, with the site that I will be updating in the ~/Sites/domains/lab/ directory, I can navigate to http:/home/domains/lab/ to view my site, because the RewriteBase directive in the site’s .htaccess file is set to /domains/lab/.

Distributed Version Control System

I am using Git as my version control system, following the convention established by the Symphony development team with the release of Symphony 2 Beta Revision 5, establishing Symphony as an open source project, published under the MIT licence.

With the beta releases of Symphony 2, the development team had created an extension called Export Ensemble to enable the packaging of the core system files and core extensions as a complete installer package with a one page installation process. This extension also provided a means to create your own modifications and easily package a customized installer of your own. This had the potential to create a themes ecosystem, similar to those offered by WordPress sites. However, Symphony sites tend to be customized for the specific requirements of each site, so the idea of themes did not work as well for Symphony, because that would assume that the database structure should be identical from one theme to another. For most Symphony developers, this was too limiting, and theming has never been one of the major attractions of Symphony in comparison to the high level of customization that is possible.

Still, I have always had in mind the possibility of creating a themes ecosystem for Symphony. The dream is to recreate the user interface of WordPress on the front end of Symphony and create a database structure similar to that of WordPress as a starting point for those who might want to migrate from WordPress to Symphony.

However, let’s take this one step at a time. I need to get back up to speed with Symphony development by getting used to working with the latest and greatest version that we have now: Symphony 2.3.

Writing in Markdown

Content creation, for me, takes place in an editor created for the purpose of creating text in the Markdown syntax. I have explored many systems for writing and have decided to take a Bit Literate approach by keeping things as simple as possible and distributed as possible.

Extension Compatibility

The Symphony core is really easy to update. Check out the latest core files with Git, navigate to the http://example.com/install/ and click the update button. That’s pretty much it. The challenge comes when there are installed extensions. Let’s say you have only the core extensions installed. This could actually be relatively simple, then.

But first, check the remote repository.

              cd /path/to/symphony/install/
git remote -v

Add a remote if it is missing for the symphony core repository. In my case, I usually change the origin remote to that of the ensemble repository:

              git remote add sym git://github.com/symphonycms/symphony-2.git

Switch to the master branch:

              git checkout master

Copy the master branch to a new branch called update:

              git checkout -b update

Then pull in the latest core files:

              git pull sym master

The core extensions are:

  • Debug DevKit
  • Export Ensemble
  • JIT Image Manipulation
  • Maintenance Mode
  • Markdown
  • Profile DevKit
  • Selectbox Link Field
  • XSS Filter

And update the core extensions. The easiest way to update the extensions is to make sure that the .git/config file has all references to the symphony and pointybeard remotes changed to symphonycms as documented by the update instructions.

              git submodule update --init --recursive

However, once the update is complete, it is clear that there are incompatibilities between the updated system files and the outdated or deprecated extensions. Taking a look at the list, it is clear that there are extensions that are going to be a problem:

  • Calendar Overlay
  • Mediathek
  • Multiselect to Checkboxes
  • Static Section
  • Subsection Field
  • Typogrify
  • Unique Upload Field

Check compatibility on Symphony Extensions.

  • Calendar Overlay extension no longer exists as a repository. It has been replaced by Nils Hoermann’s Date and Time extension.
  • Mediathek has been replaced by the Subsection Manager extension.
  • Multiselect to Checkboxes has not been actively supported.
  • Static Section was updated two months ago and is compatible with Symphony 2.3.
  • Subsection Field was last updated about a year ago, so it is doubtful that it is compatible.
  • Typogrify is deprecated and has been replaced by symphony-typography, which cannot be found.
  • Unique Upload Field was last updated 4 months ago.

So, the challenge, then, is to remove extensions that have been deprecated and find something to replace them. That may not be possible after the update, as the old extensions tend to break the front end and back end of the site.

At this point, the only option is to pull out the backups and revert files and database to the original, in order to remove the offending extensions so they won’t completely break the site and render it inaccessible.

To get back to where I was, I just checkout the original branch.

So, the update was fraught with difficulties because of unsupported or deprecated extensions. Usually, these extensions are minor additions and are most easily removed from the section fields and uninstalled before updating to prevent any unnecessary issues.

I kept running into a fatal error related to the Static Section extension when trying to view sections. Unable to uninstall the field, I manually altered the database to remove the extension.

The front end of the site was complaining that the Login event was missing. The event was removed from the core and replaced with the Logged in Author data source according to this thread on the Symphony discussion forum. So, I manually removed login from the events column of the sym_pages database table.

So far, I have a working site, updated to Symphony 2.3. Whew!

I am not sure how this came to be, but I was having issues updating extensions. There were three extensions that were displaying old version numbers and notifying to “Enable to update to version n”. But enabling was not updating the version number. I tracked the problem down to a strange table column attribute, where the version column was set to DOUBLE SIGNED instead of VARCHAR(20). After I altered the table column, the extensions updated just fine.

Text Formatting

The Markdown extension had been updated a little since the release of the original ensemble, so the sections and data sources needed to be updated accordingly. The Text Formatter options for each textarea field in the section fields needed to be updated. I selected Markdown Extra (With SmartyPants). The data sources now have the option of outputting unformatted text and formatted HTML to the XML data.

Replacing Extensions

A couple extensions that need to be replaced to restore the ensemble to the same basic functionality as before are the Mediathek and Static Section extensions. The extensions that were built to replace them are the Subsection Manager and Limit Section Entries extensions, respectively.

Once these two extensions were added and the original relationships manually restored between the Articles and Images section entries, everything was working pretty much as before. The only significant differences are largely cosmetic. The Calendar Overlay and Multiselect to Checkboxes extensions are still waiting for updates to be compatible with Symphony 2.3. These extensions provided UI enhancements to display a calendar to select dates and to use checkboxes instead of the multiple select box form element. That may be the next task to take on: Extension modification and development, so I can send some pull requests. But first things first, I still have some pull requests for my own extensions to attend to.

Update the Piano Sonata Ensemble

Fazal Khan released the Piano Sonata ensemble with Symphony 2.0.6 three years ago. I had updated the Piano Sonata Ensemble to Symphony 2.1 a couple years ago. The latest stable release is Symphony 2.3.

When I updated the ensemble, I decided to include as many extensions as I could in the same way the core extensions are included with the Symphony core files: as git submodules. This method relies on the assumption that the extension repositories exist at the URL references used by each submodule. If not, it’s a problem. This method is by far the easiest method to maintain updated extensions from one version to another, but it can be very brittle in the cases where extensions are deprecated or are not continuously supported by the extension authors to maintain compatibility with each successive version of Symphony.

It turns out that I cannot actually install the ensemble by initializing and checking out extensions through a git submodule update because there are extensions that have been removed from GitHub. This effectively breaks the submodule update process. At this point, it is easier to copy files from a working install or unpack a ZIP file. I actually ended up having to rsync files from a remote backup.

The first step in the update process is to ensure compatibility of the extensions with the latest version of Symphony. The core extensions are always updated with each release of Symphony. However, the same is not necessarily true for other extensions. The best place to research extension compatibility is to visit Nick Dunn’s Symphony Extensions site.

It turns out that all extensions beyond the core extensions are no longer supported or have been deprecated. Some have been replaced by new extensions. I came to the conclusion that the best way to update the Piano Sonata ensemble would be to remove any dependency on any but the core extensions. The only exception appeared to be the Unique File Upload extension, maintained by Michael Eichelsdoerfer. However, it appears that the updates to the core File Upload field in Symphony 2.3.1 RC1 renders the functionality offered by the extension redundant, but this may be debatable.

  • Remove Article Images Mediathek field from the Articles section
  • Remove Article Images Sub Section Child field from the Images section
  • Remove the Static Section field from the About section
  • Remove External Links Sub Section field from the About section
  • Remove External Link Sub Section Child field from the Elsewhere section

Once these fields have been removed from the sections, it is safe to uninstall the related extensions.

  • Calendar Overlay
  • Sub Section Field
  • Mediathek
  • Multiselect to Checkboxes
  • Static Section
  • Typogrify Text Formatters

Create a new branch based on the existing branch.

              git checkout -b sym-2.3

Then remove the extension directories from the extensions directory:

              rm -rf calendaroverlay mediathek multiselect_to_checkboxes static_section subsectionfield typogrify

Remove the submodule references for the deleted extensions from the .gitmodules file. And commit the changes to the new branch. Stage updated files to be committed.

              git add -u
git commit -m "Remove deprecated and incompatible extensions"

The next step is to update the core files. Start by adding the Symphony GitHub repository as a remote.

              git remote add sym git://github.com/symphonycms/symphony-2.git

Pull the latest files for the Symphony core into the repository:

              git pull sym master

There are conflicts with the .gitmodules and install/includes/install.sql files. I resolve the conflicts by reverting the install.sql file to that in the Symphony repository (to be updated after completing the ensemble). For the .gitmodules file, keep the core extensions and the Unique File Upload field submodule references. Stage the modified files to be committed and commit them.

              git add .
git commit -m "Update Symphony to version 2.3 for the Piano Sonata ensemble"

Initialize and update the extensions:

              git submodule update --init
cd extensions/uniqueuploadfield
git checkout master
git pull origin master

Modify the permissions for the install directory:

              chmod -R 777 install manifest

Navigate to http://home/sym/ensembles/piano-sonata-2.3/install/. Click the Update Symphony button.

On the frontend, there is an error:

              Fatal error: Call to a member function isLoggedIn() on a non-object in /Users/stephen/Sites/sym/ensembles/piano-sonata-2.3/workspace/events/event.login.php on line 44

The Login Info event has been deprecated in favour of the Logged in Author data source. So the Login Info event must be removed from every page on which it is attached. Copy the Logged in Author data source from the default workspace repository.

Alternatively, create a data source called Logged in Author using Authors as the source and filter by Username with the enumerated value {$cookie-username:0} and add username to the Included Elements in the Output Options.

For every page where the Login Info event is used, remove the event and add the Logged in Author data source.

In the workspace/master.xsl file, change the variable:

              <xsl:variable name="is-logged-in" select="/data/events/login-info/@logged-in"/>

to this:

              <xsl:variable name="is-logged-in" select="/data/logged-in-author/author" />

And modify the tests for the logged in author in the utilities/navigation.xsl and utilities/get-article.xsl files by replacing this:

              <xsl:if test="/data/events/login-info/@logged-in = 'true'">

with this:

              <xsl:if test="/data/logged-in-author/author">

Remove the workspace directory from the .gitignore file. And commit the modified utilities files. Delete the Login Info event from the Events list.

Update the Unique Upload Field extension submodule reference:

              git add extensions
git commit -m "Update Unique Upload Field extension"

Update and Install Extensions

Install the core XSS Filter extension to increase security for the Symphony install.

Before I go much further, I should navigate to the Extensions page and enable the extensions to update them. It is possible to select multiple extensions by clicking on the first extension in the list and using the control key (on the Mac) to click on the last item to select multiple extensions. Then select Enable/Install from the With Selected... menu and click the Apply button.

Checkbox Fields and Text Formatting

The Checkbox field was modified by adding the ability to create a handle for the XML output. But the longer label shows up in the entries table headings, so these need to be shortened. While I’m at it, I might as well update the Text Formatter for Textarea fields, as these have reverted back to None because of changes to the names of the Markdown extension options. So, I need to reselect Markdown Extra (With SmartyPants) for each of the Textarea fields where the field has been reverted.

Replacing Mediathek with Subsection Manager

Nils Hoerrmann has been a consistently active extension develop, having created the Subsection Manager extension as a successor to his popular Mediathek extension to manage relationships between sections. There was no easy upgrade path from Mediathek to Subsection Manager, because of all the breaking changes required to improve the extension. Since there were so few entries, the easiest path to upgrading the extension was to simply remove it before the update and add it back later, restoring the relationships between the sections after installing the Subsection Manager extension.

Find the Subsection Manager extension from the Symphony Extensions site and find the command to add the extension as a submodule.

              git submodule add git://github.com/hananils/subsectionmanager.git extensions/subsectionmanager --recursive

Install the Subsection Manager extension and add the field to the sections to restore the fields removed to allow the update to Symphony 2.3. Add article images back to the individual articles. Then, update the Article and Homepage Articles data sources to include the article-images: image and article-images: description elements to the XML output.

Create entries for the External Links Subsection field in the About entry. Add the external-links: title and external-links: link elements to the XML output of the About Me data source. The Subsection Manager extension outputs item elements for each entry, so the XSLT needs to be updated. The sidebar.xsl file contains a named template for external-links, with the following xsl:for-each statement:

              <xsl:for-each select="//external-links/entry">

This needs to be changed to:

              <xsl:for-each select="//external-links/item">

Add Limit Subsection Entries Extension

The Limit Subsection Entries extension replaces the Static Section extenion to limit the number of entries that can be created for the About section. When the limit for the section is set to 1, the About section menu will navigate directly to the edit page for this single entry.

Export Ensemble

To export the ensemble, simply click the Save Install Files button on the Preferences page under Export Ensemble. This will save three files:

  • install/includes/config_default.php
  • install/includes/install.sql
  • workspace/install.sql

Or, if you just want to save the entire repository to a ZIP file, excluding all the git files, click on the Download ZIP button instead.

I added these files to the repo and pushed them to the sym-2.3 branch of my GitHub repository.

Pull Request

Fazal Khan created the Piano Sonata ensemble. It’s been a while since he has updated this repository, and that’s understandable, since he is busy helping out with the Soario effort. So, I try to help out where I can by working on things that I have expertise in, like building and updating ensembles.

So, I sent a pull request:

Thanks for the Piano Sonata ensemble. I spent some time updating the ensemble so I could continue using the latest version of Symphony for my blog.

If you are still interested in maintaining this ensemble, I have a pull request to update the ensemble to Symphony 2.3. The downside is that the update was not straightforward. There are deprecated and incompatible extensions that needed to be trimmed from the ensemble and replaced with more recent versions.

Removed

  • Calendar Overlay
  • Mediathek
  • Multiselect to Checkboxes
  • Static Section
  • Subsection Field
  • Typogrify

Added

  • Limit Section Entries
  • Subsection Manager

To match the functionality of the old ensemble, it would be necessary to update the Calendar Overlay and Multiselect to Checkboxes extensions. For now, the ensemble functions in much the same way as before, except without these particular UI enhancements. These can certainly be added back into the ensemble when (or if) these extensions are updated for Symphony 2.3.