Git Remote Updater, the Personal Remote Management System

The need or desire to use a personal remote management service makes a few assumptions.

  1. It assumes that the developer has personal or client projects that are under version control on a system like GitHub, Bitbucket, GitLab, or some other system.
  2. It assumes that these projects are utilized on multiple sites.
  3. Lastly, it assumes that the developer would like a simple method of pushing updates to these multiple sites.

GitHub Updater allows for updating of plugins and themes that are developed outside of the official WordPress repository. Often these projects reside in GitHub, Bitbucket, or GitLab. Updating these repositories to a handful of sites usually involves visiting that site’s dashboard and updating the plugin or theme in the usual manner. The difficulty arises when you’re managing the same plugin(s) or theme(s) on many, many sites.

Current State of Updating

You don’t want to spend time visiting each site to update these repositories. It takes too long and quickly becomes inconvenient. It’s much simpler to use a remote management system and push updates to multiple sites. There are several commercial plugins and services to accomplish this. Among them are iThemes Sync, ManageWP, MainWP, and InfiniteWP.

The problem is that these services were designed only to work with the official WordPress repository and not with GitHub, etc. These plugins all work by showing you that an update is pending and allow you to update all the plugins for a site en masse or all of a particular plugin on multiple sites. Set up of these plugins requires a variable degree of effort and occasional errors can be difficult to troubleshoot.

GitHub Updater has had the ability to utilize many of these remote management services for a couple of years but it’s always been a bit unstable and difficult to test. Recently much of this instability has been removed by allowing for GitHub Updater’s API calls regardless of the web site visitor’s privileges. However, it still requires someone visit the site to ensure that WP-Cron runs and GitHub Updater maintains current update information.

Webhook Updating

GitHub Updater has also been able to utilize a webhook to update repositories too. This method doesn’t require WP-Cron or anyone to even visit the site. Send the appropriate webhook and the repository will update. Personally, I use a webhook to update GitHub Updater on the develop branch on my test site every time a GitHub push is identified. This way I can ensure I’m always testing with the most recent commits.

Updating via a webhook is a more of a push than a pull. For reference, my idea of a pull would be the site shows that an update is pending and pulls the update from the remote source. A push would be similar to a re-installation of the plugin from the current source. In GitHub Updater’s parlance, it’s exactly like branch switching. If you switch to the same branch it’s simply re-installing with the latest code. There is no version checking in this process.

REST API Endpoint Updating

GitHub Updater previously used a admin-ajax.php request to provide for webhook updating. I have finally re-written this to an actual REST API endpoint and it has made the Git Remote Updater possible.

Currently there are 3 endpoints.

  1. repos – returns a listing of the sites GitHub Updater enabled plugins and themes along with their current branch.
  2. update – processes the actual webhook request.
  3. test – Yeah, I just left it in. 🙃

Git Remote Updater

Recently I’ve written a plugin to facilitate your own personal remote management system, Git Remote Updater. Git Remote Updater will allow you to update individual repositories on multiple sites or all the repositories in a single site with a single button click.

How It Works

Functionally, the Git Remote Updater Plugin simply sends a webhook as an HTTP GET request to the site. This allows GitHub Updater to update the repository. The best part is that Git Remote Updater is able to send multiple webhook requests with a single click.

Git Remote Updater only needs the site’s URL and API key from GitHub Updater to enter into the Git Remote Updater Settings tab.

GitHub Updater Site Data
GitHub Updater Site Data
Git Remote Updater Settings
Git Remote Updater Settings

This site specific data is all that is necessary to validate with the site and update the repositories. This data only needs to be updated if the site’s Remote Management API key changes.

The API key is easily reset and only functions on the update endpoint to authorize an update on the site and one the repos endpoint to collect repository data.

There is a REST API endpoint that returns the site’s GitHub Updater enabled plugins and themes along with data regarding the installed branch. The second REST API endpoint is for updating. It utilizes data from the JSON file to form a correct webhook for updating. Both of these endpoints require an API key for authentication.

In order to ensure that this remote updating works, updating must work on the site’s dashboard too. This means that any access tokens or username/password data must be correctly saved on the remote site. I have found that utilizing the Git Remote Updater works best on a local development environment.

Update by Repository or by Site

There are 2 ways to update in Git Remote Updater. You may update individual repositories on multiple sites or you may update multiple repositories on an individual site. A dropdown menu conveniently switches between the two options.

Git Remote Updater by Repository
Git Remote Updater by Repository
Git Remote Updater by Site
Git Remote Updater by Site

Feedback

An example of the type of feedback supplied is below.

Git Remote Updater Feedback
Git Remote Updater Feedback

Functional Differences

Git Remote Updater doesn’t seek to provide feedback about the update status of a particular plugin or theme. It simply forces an update based on the installed branch of the remote repository. This is functionally equivalent to a reinstallation or branch switch to the same installed branch of the repository.

WP Core Development with Local Lightning

Firstly, this is only meant as a how I do it. I’m using MacOS.

core.git.wordpress.org

It is based on using git://core.git.wordpress.org/. This would be similar to WP core development on a regular install using the WordPress Beta Tester plugin set for bleeding edge nightlies.

  • Create a new site in Local Lightning.
  • From Local Lightning Open Site Shell
  • You should cd into /app.
  • Move the wp-config.php and delete /public.
  • Then clone core.git.wordpress.org to /public.
cd .. mv ./public/wp-config.php . rm -rf ./public git clone git://core.git.wordpress.org/ public

Create symlink of wp-config.php into /public

ln -sv $PWD/wp-config.php $PWD/public/wp-config.php

The following can be used to keep your clone less cluttered. Add ./app/public/.gitignore with the following data.

Add the above from my gist to the /app directory with the following command.

curl -o ./public/.gitignore https://gist.githubusercontent.com/afragen/43dfff563e942353d866c81904498cb2/raw/.gitignore

Add setup-phpunit.sh script for testing.

curl -o setup-phpunit.sh https://raw.githubusercontent.com/afragen/setup-phpunit/master/setup-phpunit.sh

I’ve written a script to aid in applying patches or changesets from core.trac.wordpress.org. It is added to the commands as well.

curl -o apply-trac-patch.sh https://gist.githubusercontent.com/afragen/977d765414189d5f5fae42215fe92a27/raw/apply-trac-patch.sh

Run setup-phpunit.sh script.

Update trunk via git pull from /app/public using Open Site Shell

Below are the list of sequential commands after Open Site Shell

Install core.git.wordpress.org in Local

Someway to automate this setup in a one-click install or an advanced setting on the install would be tremendous. see https://localbyflywheel.com/community/t/feature-request-add-simple-install-of-a-wp-core-dev-environment/12985/3

develop.git.wordpress.org

A separate one-click install using git://develop.git.wordpress.org/ would also be great but that would also likely require installing npm and setting the database to display /build as the home URL endpoint.

As above you will need to create a new site in Local Lightning and then Open Site Shell from Local Lightning.

To make this function, before running the commands you must ensure that your local environment has wget and npm installed. If you’re on a Mac I highly recommend using Homebrew and brew install wget. Installing npm using Homebrew can be done but isn’t necessarily the recommended method.

I have to give lots of credit to Sal Ferrarello for his post WordPress Core Development on Local by Flywheel and to Kees Meijer for his script setup-phpunit.sh

I had to modify the setup-phpunit.sh script to work with Local Lightning. It is heavily biased towards using MacOS. I’m hoping I can get a little help making it more universal. My version is on GitHub

Here’s the sequential commands I’ve adapted from Sal’s post to use the git://develop.git.wordpress.org repository.

Install develop.git.wordpress.org in Local

The preceding bases your WordPress installation on develop.git.wordpess.org. You will open your site from a URL similar to mylocalsite.local/build/ and the dashboard is mylocalsite.local/build/wp-admin/.

One-click Install

For a one-click install you can use the following commands.

# Setup environment from core.git.wordpress.org sh -c "$(curl -fsSL https://gist.github.com/afragen/e1aa3ffccf1a73618ee6e756bd95d297/raw/core-git-wp.sh)";cd . # Setup environment from develop.git.wordpress.org sh -c "$(curl -fsSL https://gist.github.com/afragen/e1aa3ffccf1a73618ee6e756bd95d297/raw/develop-git-wp.sh)";cd .

Bringing WordPress Core to PHP 5.6 and Beyond

In the 2018 State of the Word, Matt told us the plan was to move the minimum PHP version for WordPress Core to 5.6 in April 2019 and to PHP 7 in December 2019. I won’t discuss the irony of WordPress 5.2 being the update that kills support for PHP 5.2, but the coincidence is remarkable.

Every version of PHP from 7.0 and below has been designated end of life (EOL). Currently, WordPress’ minimum PHP requirement is 5.2.7 which was EOL’d over 8 years ago.

In the 2018 State of the Word Matt said we would be moving to PHP 5.6 as a minimum requirement in April, 2019 and increasing the minimum to PHP 7.0 by the end of 2019.

This presentation will attempt to describe the safeguards put in place to avoid breaking the internet. Much of this emanated from a single conceptual Trac ticket.

Coding for WP Core is different.

Overview of Servehappy

It’s no secret that WordPress has stayed on PHP 5.2 long after it’s been cold, dead, and buried. The reasoning was simple. WordPress is used by a third of the internet and even though fewer and fewer sites are using these EOL’d versions of PHP no one wants to break the internet for these people, or anyone else for that matter.

There has been a concerted effort to work with hosting companies to move the needle and it has worked.

Servehappy is the code name for the Site Health project. The goal of this project is to put safeguards in place to protect as many users as possible during this transition. Nothing is perfect and the numerical combinations of WordPress versions, PHP versions, plugins, and themes is astronomical.

WordPress PHP versions

There have been several methods at play.

In WordPress 5.1 the following was added.

  • The dashboard call out to update PHP.
  • Protection from installing plugins whose requirements are higher than the site can provide.

In WordPress 5.2 the following are scheduled to be added.

  • White Screen of Death (WSOD) protection.
  • Protection from updating plugins whose requirements are higher than the site can provide.
  • Protection from activation of plugins whose requirements are higher than the site can provide.

Many wonderful people have been involved in creating these solutions and they are led by Alain Schlessera and Felix Arntz. I made a conscious decision to participate in this project. Yes, code was involved but like most things in life. You have to show up.

“Decisions are made by those who show up.” – Aaron Sorkin, The West Wing

In this case it meant making time to show up and participate in #core-php and #core Slack meetings.

Sometimes it meant creating solutions/patches in Trac. Sometimes it meant testing other patches. Sometimes it meant reporting issues or problems.

There are many more concerns when coding for WordPress Core than when coding for your own projects. It may be having a higher priority for accessibility or translation readiness, but it shouldn’t.

What I learned was mostly it’s about having to support new functions and filters with an obsession towards backwards compatibility. What this means is that a hard-coded solution is more likely to be acceptable than a more versatile modular solution.

Of course that shouldn’t stop you from creating a solution that utilizes modern techniques but the solution must work within the minimum PHP requirements of Core.

Update PHP Callout

One of the first things created for the Servehappy project was a dashboard callout to update your site to a current PHP version.

dashboard callout
Dashboard callout

Along with this callout is the Update PHP page containing reasons why you should update PHP as well as information on how to get help from your web host on actually updating your PHP version.

In WordPress 5.1 code was introduced to provide a check against a Plugin’s reported compatibility with either WordPress Core or PHP. The plugin developer would declare these minimum requirements of both WordPress and PHP in the plugin readme.txt file.

Plugin Installation

The code check will disable the Install button in the plugin card in the plugin search window and provide information as to why the the plugin cannot be installed.

install screen plugin cards
Plugin Install Screen
Install screen view details
Plugin Card View Details iFrame

Plugin Updates

On schedule for inclusion in WordPress 5.2 is the automatic disabling of plugin updates for plugins that don’t meet the WordPress Core or PHP version requirements as listed in the plugin’s `readme.txt` file.

Plugin updates can occur in two locations: the `plugins.php` page and the `update-core.php` page. Disabling updates from both of these locations was introduced in separate Trac tickets, the plugins screen and the updates screen.

Plugins page
Plugins Page
Updates page
Updates Page
Plugin Card View Details iFrame
Plugin Card View Details iFrame

Plugin Activation

One of the final pieces was to disable activation of a plugin if it didn’t meet the WordPress or PHP compatibility requirements. With some help from others I was able to use the `get_file_data()` to parse the plugin’s readme.txt file headers.

If a plugin doesn’t meet the minimum compatibility requirements a WP_Error is generated, the plugin is not activated, and the user simply needs to use the browser’s back button to return to their site.

Part of the original patch was also adding 2 additional plugin file headers, Requires WP and Requires PHP so that plugins that exist outside of dot org and don’t have a properly formatted readme.txt file could still designate their plugin requirements. This was removed in the commit, opened in a separate Trac ticket.

Activation Error
Activation Error

WSOD Protection

One of the primary focuses of Servehappy was White Screen of Death (WSOD) protection. It was thought that after an update it would be a huge benefit to be able to create a sandbox for the site so that if a plugin or theme caused a PHP Fatal to occur error it would be a simpler process to access the backend of the site and either effect a change to the plugin or to disable the plugin entirely.

This patch was initially committed for WordPress 5.1 but do to late identification of potential security issues the commit was reverted. A different idea on implementing this is being developed for WordPress 5.2. This new method mitigates the security issues raised in the initial patch. Here is the official post on WSOD protection.

WSOD Error
WSOD Error
WSOD email
WSOD email
WSOD dashboard notice
WSOD dashboard notice
WSOD disabled plugin
WSOD disabled plugin

Perfection??

I don’t really think anything is perfect, but I believe the safeguards created and included in WordPress 5.1 and 5.2 take us most of the way there.

Building a better mousetrap can’t prevent a more determined mouse from success, or in this case failure.

WordPress Debugging

It is inevitable. At some point when running a WordPress site you will have a conflict, an error, or worst case – a PHP Fatal leading to a WSOD (White Screen of Death).

My goal is to provide the means with which you should be able to view and hopefully understand, to some degree, the errors so that the most appropriate person can provide a solution.

Why is it Inevitable?

By virtue of the shear number of different WordPress plugins, themes, and PHP versions, there are bound to be interactions that cause issues. Hopefully these issues don’t bring down your site. But some will.

Types of Errors

There are a few basic types of errors common to WordPress sites. Primarily all are PHP errors. There are 3 primary types of PHP errors: PHP Fatal, PHP Warning, and PHP Notice.

Under most circumstances you might not even be aware of either the PHP Warning or PHP Notice errors as they commonly only display in your PHP error log. A PHP Fatal error is the most common cause of the WSOD, but again you won’t see the actual error outside of an error log.

By default WordPress doesn’t display these errors to the user. You can adjust certain settings within wp-config.php to bring these errors to display and/or log them to a WordPress specific debug.log.

For many, modifying the wp-config.php file is a daunting task that in and of itself, can bring your site down. I’ve tried to simplify this with the creation of my WP Debugging plugin.

My plugin will add settings to wp-config.php. More specifically setting WP_DEBUG to true and setting WP_DEBUG_LOG to true. There are a number of additional settings that can also set to assist in debugging.

xDebug Isn’t the Only Way

Tom McFarlin has written extensively about coding and debugging.

In this member’s only post, Tom explains many of the individual settings that can assist in debugging a WordPress site using only native WordPress functions. These constants are also described in Debugging in WordPress.

Automate All the Things

WP Debugging is a plugin I wrote to automatically add many of WordPress’ built-in settings on plugin activation and remove them on plugin deactivation. The plugin uses the WP-CLI command to add and remove constants from the wp-config.php file. WP Debugging should be available in the Plugins Repository soon.

There are two optional plugin dependencies that request to be installed, Query Monitor by John Billion and Debug Bar. The notice for these dismisses for 45 days.

Query Monitor is an established development plugin that provides a wealth of information for debugging. Debug Bar is another excellent development plugin by Automattic.

Debug Quick Look by Andrew Norcross is a wonderful plugin whose sole function is to display the debug.log that WordPress writes debugging errors to when WP_DEBUG_LOG is set to true. I have included a modified version of this plugin.

Looking in the Logs

Viewing the debug.log will allow you to gain insight into the cause of the error. Often these errors will provide a stack trace pointing to exactly the file, function, or line of the error. They will definitely aid the developer.

Debugging is a art. One that you will only gain proficiency in through practice. It is my goal to help bring this information closer to you as simply as possible via the WP Debugging plugin.

You can read more about the specifics of what the WP Debugging plugin does on GitHub and, as always, PRs are happily considered on the develop branch.

GitHub Updater & GitLab

GitLab Support

I’ve finally been able to add support for GitLab in the GitHub Updater v4.5.0. Additionally, support for GitLab CE and GitLab Enterprise are also included. All that’s required is an additional header with the base URI for the GitLab server. As an example, GitLab CE: https://gitlab.example.com or GitLab Enterprise: https://gitlab.example.com.

Extras for GitHub

Support for GitHub Enterprise is also included using a similar header, GitHub Enterprise: https://github.example.com.
Support for updating from GitHub assets is also included if an asset exists for a tagged release. It will be preferentially used for the update. Using an asset’s URI for the remote installation of a plugin or theme will also function as expected.

Under the Hood

A bit of refactoring has been done as well. An abstract class API has been created to simplify the structure of all the git server API classes. Additionally a class Messages has been created to hold admin notices.
The class Base has been refactored to hold information regarding the added APIs in static arrays for use throughout the codebase.
I hope everyone likes the changes and updates. As always, if there are problems or improvements, please create an issue on GitHub.
Thanks.