So Many Blog Posts ...

I still find it amazing that there are 2,644 blog posts on my personal site. If one includes all the other blogs that I have written for (on 10C and off), then there would be over 3000 posts written since October 2006. While this isn't anywhere near 1 post per day, it's a heck of a lot more than most other people I know.

Mind you, I probably would have written more posts by now if I still used WordPress. I created Noteworthy (and later 10C) because I was tired of dealing with WP issues. What resulted was me investing thousands of hours into a personal project that has taken time away from blogging 🙄

Of course, one of the bigger reasons I created Noteworthy was because I wanted to have a better way to write blog posts. Evernote was a pretty decent tool back in 2010~2012, and their iOS client rocked. So I used to write blog posts in Evernote on iOS (an iPod Touch) while out and about for the day job. When I got home, I'd copy from Evernote into WordPress, add some tags, then hit publish. Noteworthy cut that process out by syncing with the Evernote API and pulling in any new notes that were in a given notebook (or removing posts that were removed from the notebook).

Those were simpler times …

WordPress DB Tuning

Last weekend a regular client of mine asked me to take a look at their website and try to improve its load times. Like a lot of places on the web, it was powered by WordPress and had a number of caching plugins installed in an effort to reduce the amount of time visitors have to wait before they can read and move about the page. Unfortunately, despite their best efforts, they could not get everything to finish loaded in less than 3 seconds during the two busy periods of the day. After a couple of minutes of research, the problem was pretty apparent.

First a little background. The site is a mobile game review site based out of Vancouver, with readers aged anywhere from 13 to 30 … give or take. Amazon's EC2 platform is used, with the database being hosted on a dedicated c4.xlarge instance with two t2.large web servers reading from this database. There are typically 200,000 visitors a day to the site and most traffic happens around lunchtime during the week and between 4:00pm and 10:00pm every day. This is not anything too crazy, though the EC2 instances are much larger than the ones I use for 10Centuries1. Because of all the caching plugins in use, the main site itself usually loads in about 1.5 seconds. The problem came down to a pair of custom-made plugins that were not particularly efficient … and a default database configuration.

Tuning a MySQL 5.7 Database for WordPress

Note: This not for the faint of heart. If you're not comfortable getting into the nitty-gritty of configuration files and occasionally seeing that MySQL refuses to restart because of something that's not quite right in my.cnf, then you do not want to do this. You can either continue to run stock, or you can hire someone to do this for you.

Note 2: The settings that I'll outline here worked for my customer's use case. Do not simply copy/paste and expect the same results. Tuning requires planning.

The two custom plugins that were causing problems had inefficient queries that needed some optimization and a bit of a rewrite to avoid using a FILESORT, but this was just a small part of the performance issues. The big one was a buffer pool that was simply too small. A c4.xlarge instance has 7.5GB of memory, but most of it was sitting unused. The WordPress database was about 2GB in total, so it wouldn't be an issue to have it all in memory ready to be read. I set the buffer pool to 3GB.

innodb_buffer_pool_size = 3G

I then set up a tmpfs partition in memory for MySQL to use for temporary tables, and ensured the InnoDB database engine was properly configured to use it:

innodb_temp_data_file_path = ../../mysqltmp/ibtmp1:128M:autoextend:max:3G

I then went through and ensured some of the other InnoDB settings were set for maximum efficiency. Because ACID compliance is not 100% critical with this website, I could get away with using different log flushing times.

innodb_flush_method = O_DIRECT
innodb_flush_log_at_trx_commit = 0
sync_binlog = 0
innodb_flush_log_at_trx_commit = 0
innodb_buffer_pool_instances = 8
innodb_thread_concurrency = 8
query_cache_size = 0
query_cache_type = OFF

With this done, I restarted MySQL. Page load times dropped from a "good" 1.5 seconds mid-day to 0.2 seconds, and 0.5 seconds during peaks. Not too bad, if you ask me.

There are a lot of InnoDB settings that can be configured, tweaked, and explored. While the default configuration is "good enough" to get people going with a WordPress site, it's really important that some research and time be spent getting the database performance up to something respectable. This isn't something that plugins alone can fix.


  1. which, I will admit, does not run WordPress.

What (Real) Problem Am I Trying to Solve?

Over the last year or so, I've invested a good amount of time into learning about blockchain, JSON, encryption, and a myriad of other tools that could be used in the next big version of 10C. After a great deal of research, I've come to the conclusion that blockchain is not what I'm looking for as a means of message validation and will instead fall back on other methods that employ technologies that are much easier to explain and verify. In addition to this, I've been looking at a number of other projects that are quite active across the web such as IndieWeb and JSON Feed1 to make the next version of the platform something people might actually want to use themselves. Yet, despite the effort going into the research and pre-development, a lingering question remains: what real problem am I trying to solve here?

There are already a number of open-source blogging tools that are admittedly much better in terms of UI and web standards than 10C. Why am I not simply using one of these as part of the larger goal? The same is said for social networking, photo sharing, notes, todos, and just about everything else I've been slowly building into the 10Centuries platform. What could possibly make anything I make better than setting up a NextCloud instance with a bunch of plugins to fill in the gaps?

It's a problem I struggle with because, while I am very much interested in helping people keep the words and images they want to share with the world online for a millennia, do I need to do it with a software platform that I build rather than one assembled from various open projects around the web? What could possibly make 10C better than WordPress with a myriad of plugins? Despite what people might want from the 10C platform, it is a silo. Even in v5, which involves a globally distributed system of servers operated by anybody who might want to participate, the system is a silo. A silo that anybody could operate, but a silo nonetheless.

Is this what people actually want? Would I be better off investing my time contributing to open projects that already have large, vibrant communities and encouraging adoption of ideas rather than of software?

One of the reasons these thoughts have been rolling around in my head is because I'm downright exhausted, and any amount of free time I might have enjoyed in the past is all but gone as a result of expectations elsewhere. This tiny blog post right here required 8 separate attempts across three devices and 30 hours to complete. This right here. Which is the length of something I used to bang out on an iPod Touch with Evernote while on the train to work. Despite all of my best efforts I just feel as though I'm letting people down as a result of diverted attention, and I really dislike letting people down.

So what is the real problem I am trying to solve with future versions of the 10C platform? Automatic distribution of encrypted content across servers to act as a backup for friends/family when systems go down? Self-hosted community creation on minimal hardware? An API-driven system that is open enough for people to easily create their own tools that interact with the system? Yes on all counts. But does it need to be something that I've written top to bottom? Is absolute control over the software stack really that important to me? Is it important to others? This is the question I need to answer ….


  1. this one will make its appearance in the existing version of 10C quite soon

No WordPress Here

A lot of pixels have been used by people to convey the frustration they’ve had with WordPress over the years, and I am no different. One of the more interesting patterns that I’ve seen since leaving the most popular platform on the web is how often bots will look for very specific files associated with WordPress exploits and try to log in to its web dashboard. There is no wp-login.php file anywhere on my servers, but it doesn’t stop simple programs running on web servers across the planet from trying to find it. Over the last year, something interesting has happened, though. Rather than try to log in with default usernames and simple passwords, they’re trying real-looking details.

Internet security is no laughing matter, yet the majority of us look at the subject with such ambivalence that we may as well have no opinion at all. Security is important, but it’s somebody else’s problem to solve. How very strange.

There are a number of services online that offer two-factor authentication, with many of the big companies housing our personal data including the functionality right out of the gate. Wikipedia has a good list of online services that support the added levels of protection. 10Centuries is not on the list1, but two-factor authorisation is on the roadmap for later this year. One can only hope that we’ll soon be able to protect our self-hosted sites this way with minimal effort.

Better Is Not Always Best

A client of mine recently had some trouble with their WordPress installation, and it really couldn't have come at a worse time. You see, this person is about to launch a pretty big online campaign which means his web presence needs to be rock solid and dependable. As one would expect before a big launch, my client was doing a bunch of testing and making minor modifications to his content … so many so that it triggered a plugin to take some rather drastic action with unforeseen consequences.

My client was using a WordPress plugin called Better WP Security, a tool I had never heard of or seen the need for. There are better ways to do what this plugin offers, and it doesn't affect system performance in the least. That said, I digress.

This plugin comes with many interesting features for the web administrator who doesn't want to administrate a web server, including:

  • Remove the meta "Generator" tag
  • Change the urls for WordPress dashboard including login, admin, and more
  • Completely turn off the ability to login for a given time period (away mode)
  • Rename "admin" account
  • Change the ID on the user with ID 1
  • Change the WordPress database table prefix
  • Change wp-content path
  • Display a random version number to non administrative users anywhere version is used

A lot of these things can be done without a plugin, but that's neither here nor there. What triggered today's problem was a feature listed under the "Protect" section:

  • Ban troublesome bots and other hosts

My client has all traffic routed through a load balancer. For most web servers this means that all traffic looks like it's coming from the same IP which may look like a DDoS attack if the traffic volume is too high. To resolve the issue Better WP Security made a quick update to the .htaccess file banning any and all traffic from the load balancer.

Whoops.

Only way to fix this is to manually log into the server and edit .htaccess.

Image from Felix

Why Not Use WordPress?

Earlier on App.Net I stumbled across a conversation where some people were discussing what purpose another blogging platform could serve when there are a number of existing options freely available for download and use. As someone who has made not one but two blogging platforms over the years, I feel that I can answer this question from a number of perspectives with objectivity. First I'll answer the question from the point of view of a technically-inclined person, then from the point of view of someone who does not live in a "First World" nation.

From My Point of View …

People who have followed me on social platforms such as Twitter and ADN will know that I have railed against the coding practices employed by the core WordPress developers for years. Functions are inconsistently defined, excessive use of global variables wreak havoc when trying to to some custom development with easily understood variable names, an excessive amount of "hard code" in places where easily-customisable variables can be found, and too many unnecessary libraries are loaded for each and every operation. The result of this hodgepodge of PHP is a Content Management System that requires far too many resources to do something that is insanely simple; show website visitors the content they came to see.

I've helped people tweak as much performance as possible so they can use WordPress on web servers that few web developers under the age of 30 would dare touch. More often than not, I've helped people who want to have a self-hosted website on Amazon's free EC2 server tier, which works out to be a 1.2GHz-powered virtual machine with 613MB RAM. Most of the cell phones that people have bought in the last two years ship with more computational power and memory than the t1.micro web servers Amazon hands out for one year. Five years ago this would have been plenty for a WordPress installation. Today, however, these tiny machines can run out of memory and crash at the most inopportune times1.

How often have you visited a site that's proudly powered by WordPress only to wait more than 30 seconds for the 300KB page to load and render? How can anybody fool themselves into thinking this is "good enough"?

No. Considering how mature the LAMP stack is, and how much experience web developers have had with scripting languages like PHP, this should be an embarrassment. This is one of the major items I tried to tackle when writing Noteworthy, which I've often bragged can run on a Pentium III 300MHz server with 128MB RAM and handle several thousand visitors a day. How do I know? Because this site was running on a Pentium III 300MHz server with 128MB RAM and a 12GB hard disk for the better part of a year while I built the system. 10Centuries is a fork of Noteworthy and runs just over 70 sites on a single installation, but it still operates on a single server with less-than-stellar hardware2. I'll be the first to admit that 10Centuries is nowhere near as robust as WordPress, but it operates very well for what I need it to do and has seen zero downtime in over a year as a result.

I can't say the same for my WordPress installation that ran on a better server two years ago.

So, as a geek, I can safely say that WordPress is a poor solution for someone like me who wants a simple blogging platform that enables me to write and share with the world while staying out of the way. I don't want to use a web interface to post articles. I don't want to remember to log in every few days to see if there are updates to download. I don't want to worry that some plugin someone wrote is siphoning data off my site and sending it elsewhere. I want to control the whole stack, and there are many people like me who want the same.

This is why I made Noteworthy and 10Centuries. This is why others will make their own blogging platforms and extend them over time. Some will reach some modicum of success while the vast majority will be obscure tools used by a handful of people, or just the developer. And why shouldn't we do this? The best way to appreciate the intricacies and complexities that arise from building a CMS or blogging tool can never be known unless you plough head first into the task of creating your own.

From Another Point of View

Earlier this year I was contacted by someone in a developing nation that was having trouble with a WordPress installation. She had managed to get her hands on an old PC that had been discarded but was still functional and wanted to make a blog that would be used by people in her school. WordPress had been installed and was properly configured to work as a blog network, but page loads were insanely slow. This server is only accessible on a network that rarely has more than 5 computers connected to it … much slower than the websites that loaded through the shared Internet connection on the same network. What could be the problem?

I helped this person work through all sorts of technical problems, from Apache settings to DNS routing to resource offloading. Nothing helped. The server just couldn't load WordPress' administration panel in any reasonable amount of time, nor would the finished pages. I asked her to put some static HTML pages on the server that included links to CSS and JavaScript files, and those would load in seconds on the school computers. Only WordPress was causing the problem.

Originally she had tried to get help through the WordPress forums, but found most of the answers to be anything but. Most people told her to use a WordPress.com blog, which she didn't want to do. Others blamed her for using "ancient hardware". A few did try to help, but nobody could really tell her what the problem was or how to fix it.

At the end of the ordeal, this young lady decided that WordPress was not the right tool and she installed a tiny blogging package from GitHub. Sadly it was not mine, nor was it as full-featured as Automattic's writing solution, but it did the job … quickly. She didn't have the money for the more powerful servers that so many of us take for granted, nor should she need to when the discarded computer she acquired should have been more than sufficient3 to serve blog posts to a maximum of five computers on the same network. Could the installation have been misconfigured? Absolutely. But this isn't the point. The fact of the matter is that WordPress has grown to become a very robust, very complicated software package that requires far more resources than should be necessary. She needed something simple and tiny … something that many mature software packages can no longer provide.

Can't Escape Complexity

Over the coming years I fully expect Noteworthy will lose the ability to run on a computer built in 1999. There are a number of features I have already started writing which will require something with a little more oomph, and this will make my software solution ineffective for people who may not have the same access to technology that I have. When that time comes, we will undoubtedly see another round of blogging tools enter the fray. There is no stopping it, nor should we mock those who want to build their own tool. Heck, considering just how many people there are in the world, it's surprising just how few options there are out there.

What works for me may not work for you and vice versa. Let's encourage the people who want to provide new takes on existing tools to do so. Who knows … it might just result in something really great.

Noteworthy Now Supports ADN

Today was one of those days where the stars aligned in just the right way to give me the four hours necessary to push out a couple of important bug fixes to Noteworthy and a bit of new functionality while sitting idle at the office. Some of the bugs that have been laid to rest include some of the automated data collection as well as search result parsing. There's also some better ENML->HTML scrubbers in place and, perhaps most importantly, the ability to play nice with ADN posts in a similar fashion to Tweets. It's this last bit of functionality that I am particularly happy with.

One of the primary goals of Noteworthy is very similar to ADN. People should have a way to keep all of the data they put online in one central location … forever. I've been incredibly impressed with the people on ADN as well as the potential power of the entire ADN platform since joining the service earlier this month, and there's no doubt in my mind that I'll be building some tools that play nicely with the service later this summer.

Any archiving system worth its weight in salt will need to be able to search its database quickly and reliably to find the data people are requesting and display it in a way that makes sense. Up until now, the search queries have only returned results from Twitter and blog posts. That changes today with the introduction of ADN post results. Not only that, but the yellow search text highlighting is back! There are still a few glitches, as we can see with some of the blog post results, but these problems will be worked out in the next few releases.

search_results

A few people have asked me whether I would be making a plugin for WordPress that can import ADN posts, Evernote notes, Tweets, and a host of other interesting items. Truth be told, I've thought long and hard about it. WordPress has a huge user base and I would be better off building tools that will appeal to a broad audience. That said, it's not something I plan on doing in the near future1.

This weekend will see quite a number of nice updates to Noteworthy as well as its sister project, 10Centuries. The roll-out date for the latter is drawing near, and I look forward to seeing what sort of reception it receives over the coming months. All I'm looking for is a moderate success and, from the feedback I've received thus far, it might just happen.

Proudly Powered By WordPress

It's no secret that I despise WordPress, the world's most popular website software1. It's a resource-hungry mess of spaghetti code that was in desperate need of an overhaul three years ago. I was very happy to ditch it in November of 2011 to replace it with something of my own creation. Now when there is a problem, I have nobody else to blame but myself, as darn near every line of code in Noteworthy was hand-written by the guy I see in the mirror every morning. That said, I do enjoy visiting sites that are proudly powered by Automattic's behemoth software, and I enjoy seeing messed up designs that nobody seems to care about even more.

Earlier today I visited a link to Android Dissected to take a look at Dash, a new ADN client. While I am quite accustomed to seeing Android-related sites running some very dark site designs with a lime green android shoved into a corner, I was not quite expecting what I ran into today.

narrow

Whoops! Looks like the theme is just a tad messed up. I've seen this quite a bit when trying to render on mobile sites, and it's usually due to one or two lines of CSS getting in the way of a nicer layout. A double-tap of the content area will zoom the screen to something better, but I wanted to see if I could find the culprit for this 45%-width layout.

I didn't have to look far:

narrow_reason

The ad is a bit too wide for the column and … look at that! Jeff Trocchio's bio does not resize itself for the content box it's sitting in. In fact, it looks to be set to a very specific width in pixels rather than percent, which would explain why the content has been squished like this. Well, no harm done. I'll just find the contact form and let the site owners know about the issue …

narrow_wordpress

Proudly powered, huh? I crack up every time I see this line. I've met very few people who use WordPress with pride. If anything, WordPress is the defacto standard because it's just so easy to bend and warp it into darn near any shape … even when that shape is bad. It takes nothing to set up for a semi-experienced person, and the money people can charge for tech support is little more than intentional highway robbery and more like a blatant scam2.

I really hope that more people move over to SquareSpace3 in the near future so that we can start to move away from the dearth of slow-loading, plugin-heavy, server-taxing websites like the ones we see today. Sure, the design problem is not the fault of WordPress, but it seems that many of the worst-performing sites are all proudly powered by a poisonous mixture of this software, a few dozen poorly-written plugins, and a heaping teaspoon of misinformation.

Right Message, Wrong Platform?

Johnathan Wold wrote a pretty decent article on Smashing Magazine on being a top WordPress developer and I agree with almost every point. There's just one little item that bugs me. People who want to become one of the top developers for any project need to be willing to invest a great deal of time, effort, and dedication to the project. He suggests we do at least an hour of reading every day on various aspects of the system. Talk to the right people and become part of the right circles. Learn PHP and MySQL inside-out. Contribute with new themes, plugins, bug fixes, and a plethora of options. All noble tasks, and absolutely necessary to be considered a "top" anything. The problem that I have with Mr. Wold's article, though, is that people may be barking up the wrong tree when it comes to the platform. WordPress is not where it's at.

Being a top WordPress developer means that you have to kiss an awful lot of ass to get noticed. Kissing ass is actually more important than knowing how to code or how to write optimal SQL queries. You need to slog through an awful lot of forum posts and tell n00bs to read the documents, say how great the existing code is, and pat people on the back despite their not having written a significant line of code in 3 years. Then, if you have a few plugins available for download, you can almost consider yourself to be part of the circle … but never the top circle.

Never, ever, ever.

Rant aside, I do believe that it's incredibly useful for people who want to contribute to an open source project to choose one that they're incredibly interested in supporting. Not only is this a great way to improve our skills and interact with intelligent problem solvers, it's also a great way to see other coding styles in action. There is more than one way to code a loop, after all.

One extra item that I would add to Johnathan's article would be the following:

After You've Learned the Ropes, Roll Your Own

Seriously. If someone wants to be the best anything, they need to have the ability to make their own competing system, too. The reason is twofold. By creating our own system we can get a better appreciation for the difficulties that are inherent with long-term development of any project. It's really, really hard to not have portions of "bad code"1 in a project that's seen active development for more than six months. By seeing some of our own poor or incomplete decisions come back to bite us, we can achieve the second reason we should make our own project … which is to gain a better appreciation for the people who worked so hard on the initial project in the first place.

I've re-coded darn near every function and feature in WordPress from versions 2.0.22 all the way up to the current release3 depending on what I or other people needed. While I may not agree with how … inconsistent … a lot of the functions in WordPress are, I can certainly understand why they've been coded that way. Hopefully 4.0 will see a rather large swath of the wasteful code brushed away and replaced with optimized functions. People shouldn't need anything larger than a t1.micro to host a simple blog, after all.

WordPress Redirects

Over the last few years the WordPress project has gone from being the blogging platform of choice to being synonymous with partially-functional, resource-intensive spaghetti code. I used the software for years and have encountered hard near every problem that a person might actually face with the software. From upgrades that destroy the database to ridiculous resource requirements, I've had to find ways to solve the problems without the help of the clueless WordPress developers who typically ignore trouble tickets or blame people for "breaking" the software by installing X plugin. It was actually because of WordPress' terrible overall experience that I decided to create Noteworthy, and I haven't looked back since. That said, I still manage a number of WordPress-powered sites1 and am regularly called upon to solve some of the more complex problems people face.

Yesterday I received an email from a great client who had run into some trouble setting up his WordPress multi-site network to accept new user registrations. The problem is actually built right into WordPress, and there is very little that the average person can do to prevent it. The issue is not caused by a plugin. It's not caused by a theme file. It's not caused by a configuration "error". It was created by a WordPress coder (or coders) who did not fully think through some of their code.

Description of the Problem

The site is running the latest version of WordPress, and it's been configured to run in a multi-site configuration across multiple domains. By default, the site was not allowing new user registrations, but a new blog has been added which will allow for them. When a person tries to create a new account, though, the page doesn't load and the browser eventually returns a 310 "Too Many Redirects" error.

Quite annoying.

According to numerous WordPress developers answering forum posts on this subject, people need to muck about their wp-config.php file, or .htaccess file, or add a /mu-plugins directory to their installation and write a custom plugin that disables the 404 redirects function built into WordPress. Some people even resort to completely starting over with a fresh installation of WordPress and re-importing all of their data … a process that can take an entire weekend for some.

But none of this is required. Here's the actual problem:

Let's look inside wp-signup.php at lines 24 to 27:

if ( !is_multisite() ) {

wp_redirect( site_url('wp-login.php?action=register') );

die();

}

So if the site is not multi-site, redirect to wp-login.php … gotcha. But the site is multisite and we are being redirected. Let's take a look at that wp_redirect function:

function is_multisite() {

if ( defined( 'MULTISITE' ) )

return MULTISITE;

if ( defined( 'SUBDOMAIN_INSTALL' ) || defined( 'VHOST' ) || defined( 'SUNRISE' ) )

return true;

return false;

}

Alright … this looks simple enough. Not sure what 'SUNRISE' is, nor do I care, but this should grab the values I specified in wp-config.php and return a happy boolean value. After all, these are the values in my wp-config.php file:

define('MULTISITE', true);

define('SUBDOMAIN_INSTALL', true);

Doesn't get much simpler than that. A value of true is returned, which is then converted to false, and the redirect is not called. But, wait a minute … it is being called. That's why I dug this deep. Is wp-config.php not being loaded when we try to load wp-signup.php? Let's find out …

If we call wp-signup.php this is what happens:

  1. Load wp-load.php
  2. Check if /wp-config.php exists and load it if it does
  3. Load the blog header if it has not been loaded before
  4. Load the theme and lay everything out nicely … even though it's not required in the least for wp-signup.php 99.999% of the time

Very strange. MULTISITE and SUBDOMAIN_INSTALL are defined before we check to see if this site is running multi-site. So why the heck isn't the code firing? Is it because the WordPress code regularly alternates between open and closed tags? Is it because of a secondary or tertiary block of code that is being called inbetween a definition?

You know what? Who cares. Let's just shut this crap off. The problem is that wp-signup.php is saying to go wp-login.php, and wp-login.php is redirecting to wp-signup.php. This endless loop never gives up and it sucks up server resources like you wouldn't believe2

Opening wp-login.php we see the following code in lines 486 to 490:

if ( is_multisite() ) {

// Multisite uses wp-signup.php

wp_redirect( apply_filters( 'wp_signup_location', site_url('wp-signup.php') ) );

exit;

}

Comment that out. The whole block. Heck, delete it if you'd like. It doesn't matter. Does multi-site actually use wp-signup.php? Well, if it does, it sure as heck doesn't want to do it.

Disabling the function means that people will not be able to create their own sub-domain, but this may be a good thing. Unless you are running a blog network on WordPress, there really isn't much need for this type of functionality. People will be able to sign up as contributors, readers, or whatever level you have already specified. It's not a perfect solution, but it's a heck of a lot better than the 310 Infinite Loop …