An Update on Nozomi

The last few days have been pretty hard on Nozomi. After spending the night at the vet's, she came home and almost instantly crawled into bed, coming out only when her stomach is cramping up, when she needs the bathroom, or when I coax her out to try and take some medicine or eat some food. Last night she had quite a lot of blood in her stool, so another trip to the vet was in order today. In addition to 75cc of medicine delivered through an IV, she was given three needles: two for her bleeding and one for her pain.

Hopefully this will be the last time she sees the vet for at least four months.

I would like to think that she's making a slow recovery, though she still has zero appetite and won't take her medicine. To make matters worse, she barely drinks any water. There's a selection of foods around her in the event she wants to try something, such as jelly, bread, and some expensive kibbles from the vet. Her water dish has also been moved right close to her bed so that she doesn't need to traverse a chilly floor to get a drink. Unfortunately this is all I can really do for her.

Last year at this time she was likely fast asleep on my lap. The same could be said about the year before, the year before that, and every year since 2010. There's still time for some lap-naps, and I'll gladly make space for her when she's feeling better.

Real-Time Search

This morning, while staring at the day-job workload waiting for me in the form of GitHub tickets, I started thinking about a problem on 10Centuries that I have long wanted to solve. It's a topic that has come up again and again over the years, and I think it's almost "solved". The problem of course, is search.

Search on a website is theoretically pretty simple. People enter some words in a field, those words are checked against the content in the database, and results are returned. In its simplest form, only the exact search term is sought. This means that if I were to ask for all posts containing the words bright and yellow, I would see only this result as there is an exact match for "bright yellow". But if I were to ask for yellow and bright, nothing would come back. This is clearly suboptimal, so it's better to have all of the words split apart, with results that include all posts with the words bright or yellow, ideally scoring the posts in such a way that the above-referenced post is at the top of the list. A lot of effective software uses this weighted search result method to return relevant results, but I wanted to do something different still.

I wanted people to see something instantaneously.

The New Search Box

One of the tricky parts of instantaneous results is dealing with network latency, server load, and all sorts of less-than-desirable problems that can make a theoretically semi-decent idea practically untenable. More than this is the general response times of the service. Most people can type several characters per second. Sending multiple calls to the API just for the illusion of supplying decent results in realtime seems silly. So I decided to go about solving the problem a little differently: a subset of every post is loaded into memory and called when requested.

This blog post is number 2,428 in publication order — so long as I haven't back-dated anything since this post went live — and the average size of each post is roughly 618 words. That's 1.5-million words. A crazy number one might say, but then I have been blogging for almost a decade. 150,000 words a year is nothing compared to the number of words that have been published on various social networks, forums, and IRC channels over the years. Loading all of these words into a browser would be absolute overkill so, instead, I am loading just a subset of the words that constitute a post. As people type their search query into the box, the browser scans through the data stored in memory, finds matches, scores them, and then updates the results. People with relatively recent hardware will see that the operations are pretty much smooth and responsive. People with hardware as old as this blog … will unfortunately suffer some stuttering. People searching other 10C-powered sites will likely not notice a hiccup at all.

The browser is working with a subset of the posts, though. What's not included? The content.

For the moment, search will pull from titles, URLs, tags, and author names. Future updates will include the content of the pages and posts. Yet before it can happen, two things must first take place.

  1. I need to see that people are able to use the search in any browser on any platform. This is still in testing.
  2. I need to create a cached result for every post that contains just a single copy of every word in the article, excluding certain common words in various languages.

Once these two things are done, then I can build on the existing search tool in order to provide much better, more specific results.

In the meantime, people using the default blog theme on 10Centuries will see an "Archives" link in their navigation bar. Every post will be listed in reverse chronological order, and the search bar up top can be used to quickly find published items. If you don't see this link, it's because the cache for your site has not been refreshed. Simply write a new post (or update an existing one) to force the system to regenerate your website.

This isn't a perfect solution by any stretch of the imagination, but it solves a number of problems that I've been thinking about for quite some time, and it does it in the browser rather than taxing my own servers with Google-like search speeds. Hopefully this same search method will be employed in every theme going forward.

Wait ... What?

I make a habit of registering every piece of hardware I buy. Not only for the spam that often accompanies the transmission of a valid email address over the Internet, but also for important pieces of information such as driver updates and promotions.  Since registering my (retired) HP iPaq 211 PDA, I've received a grand total of 5 updated driver notifications, but none of them had a subject line like this:

HP's Routine Alert Emails

Maybe it's just me, but I really don't like the sound of a "Routine Alert". It makes the product seem unreliable.  Aside from the initial freezing problems I had with the unit, which were fixed by a comprehensive set of software updates, the PDA worked remarkably well right up until February when the screen became much less precise. HP's iPaq line is really quite good.

Hopefully someone in marketing sees this and opens a can of Whoop-Ass on the person who thought up the subject line of this email.  Something along the lines of "Driver Update for Your iPaq PDA" would have gone over much better.

QuickStudy Get A Little Better

After using the program for three days, I've found quite a few areas that needed work and made the changes with all haste. So far, the initial version of the application has been downloaded a total of 22 times (much more than I had expected for an Alpha release), but that's all I really know. I do not collect information about who downloads the application or whether it's being used or not, since it's really none of my business if the application is free to everyone.

Ah, but I'm getting off track!

QuickStudy is now available for download to all Windows Mobile 5 and 6-based devices and comes with quite a few small updates to the application. Fixes include:

  • new Zoom-Level function added to Quiz screen to make reading easier

  • new Database Updating software installed for seamless upgrades of all future QuickStudy releases

  • faster load time of Quiz information on front page

  • faster Quiz Imports

  • faster Question/Answer loading in Study screens

  • elimination of pesky glitch when pressing "Review" button in Study screen before showing answer

  • elimination of (most) character-based errors when importing Quizzes

  • better random question engine

  • Study screen remembers personal settings

This is pretty much the gist of what I've done to the application since I started using it full time on my own PDA two days ago. There's still lots of little tweaks and whatnot that need to be put into this application, and I'm always open to suggestions.

One small thing that I should mention is that if you've installed the first version of the application, you'll need to eliminate the initial database, then install this version. The database is contained in the benkyo.sdf file found in the root directory of your PDA and, once you delete it, QuickStudy will rebuild it the next time you start the application. This should be the first and only time you'll ever need to do this, as the next releases will make use of a proper upgrading mechanism.

If theres something you feel is missing from QuickStudy, feel free to contact me at jason[at], and I'll be sure to get back to you.

You can download the latest version of QuickStudy here.