The First Bots

Many years ago, when App.net was a reluctant social network1, people had a tough time finding new members to welcome them to the community. There were a lot of accounts being created every day, but most of them were completely automated and quite literally spammed the global timeline, rendering the stream of posts all but useless. People lobbied the creators of the platform to do something about this, but the requests fell on deaf ears. It was at this point that I decided to take matters into my own hands and created Nice.Social, a web client for App.net, along with an API that would read the global timeline in an effort to determine which accounts were powered by a person and which ones were just automatons. The API was called NiceRank and, in the space of four months, it was able to accurately identify the viability of an account with a high degree of accuracy. By the time the service shut down in 2017, a total of 87 spamming techniques were identified in total across every published post to the network. Not everyone liked the system, but it did make it easier to use the global timeline to find real people — which is why the thing was created in the first place.

Given my rather public stance on spam2, it may be a bit unexpected to hear that I plan on creating not one, but two bots for the 10Cv5 platform. These will operate independently of the main source code, but they'll perform the following functions:

  1. To send a welcome message in the Global timeline to let people know there's a new member, and to give new accounts something to see in their Mentions
  2. To share earthquake data

What makes these bots different from a blind spammer, though, is the interactions that will be possible. Much like the bots on IRC, people will be able to ask questions and get an answer back. For me this will be most useful with the earthquake bot, as there is a lot of seismic activity in Japan. The initial data integrations will have just information on quakes in and around Japan while other regions are added later. What I'd like to be able to do is @mention the earthquake bot and ask a question like "@bot last earthquake near Toronto" and have it return a simple message with the data and map outlining the epicentre and maybe intensity rings. This sort of tool is not at all unique or new. Like I mentioned earlier, this is something that people have been doing on IRC for a quarter century through the use of eggdrop bots. What I would like to test is whether something like this would make sense on a service like 10Centuries.

There are some features that need to be restored and rolled out before these tools come online, though. Right now everyone who uses the service is generally limited to three views: the Global Timeline, Mentions, and Interactions. If there's an automated poster in the Global timeline, then there's a good chance that people will see the account as spam, which defeats the purpose of a person-readable Global timeline. In order to keep this clean, bots will be using Personas that are clearly marked as such. If a person wants to interact with a bot, they'll need to change their visibility filter settings to include bot messages, then — if they want to see the posts from the bot regularly — subscribe to updates.

App.net was theoretically supposed to work in a similar fashion. As per the terms of service, a broadcast account was supposed to have their type set as "Bot" when the account was created. More than 90% of the accounts filtered out by NiceRank were marked as human. To ensure a similar thing does not happen on 10C, the code for NiceRank has been "dusted off"3 and would be much more useful as part of a core API than operating as an auxiliary service. Should there be a deluge of spam (or RSS feeds), then the API can be expanded to include NiceRank to first flag, then re-assign personas that are incorrectly labelled.

Unlike the previous version, there would be no visible score. That was perhaps the biggest mistake I made when the tool was on App.net, as a number of people misunderstood the intent behind the service and demanded they be de-indexed. It was unfortunate and probably preventable had I better communicated why there was a score and how it was calculated.

The short term goal is to have these bots set up and ready to join the network next week. There are a number of features that need to be tested more and deployed before any bots can go live. Hopefully the interactive tools will be seen in a more positive light than the blind broadcasters who just shout mindlessly into global timelines everywhere.


  1. I don't know how else to describe it, given that the creators of the platform were hoping the system would turn out to be a unifying API that could be used for a whole host of things, not just a social network. That said, people got hooked on the idea of a "Twitter replacement" … including me.

  2. I am not a fan.

  3. Just a git clone nicerank.git sort of thing so that it's immediately available. The last update was in January 2017, and the code is … disappointingly ugly.

Anti-Negative

Why am I so negative when doing things for the day job? This is a question that I've been struggling with for quite some time and there has yet to be a good answer beyond "Maybe I'm just a bit too selfish". Looking at the myriad of posts that I've started this month and not published, there is a clear trend towards complaining, which is the main reason the posts were abandoned and left to collect digital dust in Evernote1. If people wanted to read such tripe, there's better-written stuff to be found on the popular social networks. What I really want to do is use the daily writing exercise to focus on something that is decidedly not negative. While this isn't always the case, it's something I am for.

There is a lot to be happy about, too. The boy managed to get accepted into a good kindergarten not too far from here2, Nozomi is in good health, Reiko and I are managing our jobs as well as the household responsibilities. Several years ago everything around me today would have seemed all but impossible. So where is this negative bias coming from?

As I've already alluded to, I might be just a bit too selfish. What I want I cannot have just yet3, but what I have is better than most. It's an absurd situation and I know it. More than this, though, is a feeling of uselessness at the day job. The tasks I'm doing are important, but they're not interesting. All the interesting work has gone to other people and I have missed the boat. This shouldn't be seen as a negative, yet the boredom of writing, testing, and verifying data migration scripts day after day does wear on a person who wants to create something new. This is clearly selfish and fuelled by an ego that is probably three sizes too large. Rationally, I recognize this.

Yet for a creative outlet there are not one, not two, but three other projects that I get to work on. The first is 10Centuries, which has seen a number of pretty nice updates to the API over the last few weeks to really speed things up. There is still a boatload of work that needs to be done to make it worth using, but progress is being made. The second is an inventory management system for a client in the US. I've written a bunch of software for him over the years and this new project is different enough from everything else I'm doing to encourage some additional learning. The third is a textbook management API that will be used internally at the day job to handle the rather large library of materials that are used across the globe. It's not a particularly large project, but it's one that I am quite capable of handling. So, again, why the negativity?

There are better ways to spend one's time than being upset.


  1. Has it been two weeks already? Where does the time go?

  2. The competition to get kids into the limited kindergarten spaces is quite intense at times. That said, the boy has a number of advantages that will help him through to university … if he chooses to capitalize on them.

  3. To be self-employed, doing things that make a positive difference in the lives of some people, and earning a fair amount as a result. I don't need to be rich so long as the bills can be paid and retirement can be saved for.

18 Degrees

A couple days ago Randolph had some drama on Twitter after writing a few words on seeing the difference between 4K and 1080p, which brought to mind the discussions I've had with people who are genuinely happy using a 15.6" display running at 1366x768 pixels; a screen resolution that gives me eyestrain within minutes. While WXGA1 does not work for me, it clearly gives millions of people just what they need. Ever the walking edge case, what I need is smooth characters on properly-lit screens. My 13" notebook's primary display runs at 1680x1050 while the 24" 4K Dell P2415Q next to it runs at 2304x1296. Would I want to run these screens at their native resolutions if I could?

No, of course not. That would be silly.

Display Resolutions that Work (for me)

Like Randolph said in their post, this topic started with a (now deleted) Tweet thread from SwiftOnSecurity that suggested that notebooks with hi-DPI screens be set to run at 1080p by default to improve performance and conserve battery power. This is a perfectly logical statement given that a lot of people today are clearly still happy with 1366x768. Going up to 1920x1080 with the next notebook — if there will even be another notebook purchase2 — would already be a nice improvement. Proper scaling would ensure that characters are smooth and pleasing to the eye while also saving on power and heat … which is one of the reasons I am not running my displays at higher resolutions.

When I first started using the 4K Dell, I wanted to run it as close to its native resolution as I could. Sure, the text was small, but the screen real estate was great when working with long SQL queries or web designs while the screen was turned to portrait mode. Even at 3008x1692, the highest resolution macOS supports on the Dell, the letters were smooth and easy to read. There was just one problem: heat.

Because the machine had to work much harder to push 6,853,536 pixels3 dozens of times a second, the fan would run constantly and the processor was 18˚C warmer when idle, which is the same amount of temperature rise seen when I'm compiling complex applications or using an Excel file that contains Asian characters. Was the extra screen real estate worth the noise and narrower thermal envelope? Would the faster battery drain be worth the price of pushing those pixels? Not in the least, hence the 2K scaling.

I agree that for most people, 1080p would be an excellent screen resolution to start with. People who want more or less working space can change the display to something they're more comfortable with. If someone can't see the difference between a 4K or 1080p resolution unless staring at text, that's great. If someone can, that's great too. Computers allow us to configure preferences for a reason.


  1. WXGA is 1366 pixels wide by 768 pixels high. Many cell phones ship with higher resolution screens now.

  2. I really, strongly feel that many people would be better served by a tablet with a detachable keyboard. Whether it's Android, iPadOS, or Windows doesn't matter. The portability and flexibility suits a lot of people's lifestyles.

  3. The combined number of pixels for both screens.

Five Things (I Would Photograph If I Were Any Good)

The first digital camera that I owned was an HP Photosmart 635 camera back in the early 2000s. It ran on a pair of AA batteries, took SD cards no larger than 128MB, and started disintegrating after the first six months of rather light use1. The experience was not particularly great, so I generally didn't want to use the camera very often. Later, when cameras started appearing in phones, the image quality was so poor that I often refused to take a photo because the pixellation would "ruin" my memory of the moment. In 2006, when I first visited Japan, I borrowed a friend's Olympus digital camera and was quite impressed with the results. Later that year I bought a Canon A95 and used that to take about 2,200 photos2 over the span of 7 years. It wasn't until Nozomi joined the family that I actively wanted to take pictures of the people, puppy, and places around me … which meant using a 4th Generation iPod Touch more often than not.

Over the intervening years, I've taken far more photos than I can keep track of. iCloud tells me that there are 24,818 pictures in my library, but these are just the ones I've kept, which is not the same as the number I've taken. The images that are blurry beyond recognition, horrendously out of focus, or just plain awful seldom see the next day. When "the cloud is the limit" to the number of pictures a person can keep, it's more important to ensure that every photo we keep has value in order to reduce the amount of visual checking and rechecking that goes on later. One thing that I can say for certain is that since 2011, my pictures have become noticeably better. Subjects are in better focus. The rule of thirds is observed a lot more. I try to frame static subjects when the opportunity arises. The use of focus to have the subject appear sharp while the background is blurry has also proven quite enjoyable and is easy as heck with the prosumer Canon DSLR we picked up before The Boy joined the family. However, this is pretty much the limit of what I can do to make a good picture. 90% of the photos I take are simply not worth sharing.

So, being Sunday, today's Five Things will involve five subjects that are not family and that I would like to learn how to photograph properly.

The Night Sky

Night photography has proven quite tricky with the DSLR and all but impossible with the phone. When the moon is shining bright, illuminating some fluffy while clouds from above while a city illuminates them from below, I am often awestruck by the beauty of it all. Light against the darkness. The contrast is easy to appreciate. Unfortunately my knowledge of ISO levels, exposure times, and other night photography techniques is insufficient to adequately capture the fleeting moment where the distant city, the fluffy clouds, and the incredibly bright moon are balanced "just right".

Parks from 15cm (Above the Ground)

Many years ago I wondered what the world looked like from Nozomi's point of view, as her eyes are generally just 15cm from the ground when we're out for a walk. So I started taking pictures with my phone at her level to reveal a world of giants. Trees that are impossibly tall. People that are skyscrapers in an of themselves. Grass that is tall enough to obscure a discarded bicycle just metres away. Some of the pictures did turn out quite nicely, but a lot had the wrong focal points. With a bit more practice and perhaps a better use of a mini-tripod, I'd love to create a series of images that can be shared with the world.

Historic Locations Permeated By Modern Tech

When Reiko and I take the boy to castles and other historic places around the area to introduce him to the history and culture of the country, I am often amazed by how many people are experiencing the location through their phones and/or tablets. The contrast intrigues me. To capture this in an image — or series of images — would be incredibly interesting.

Janitors

One of the many things that visitors to Japan often comment on is the lack of litter in the streets despite the ever-present dearth of garbage cans. While many people will hold onto their garbage until they can find an appropriate place to dispose of it, some people will "accidentally" drop things on the ground and just keep walking. When this happens, a janitor will usually be along in a matter of minutes, find the refuse, and take care of it. These people can also be seen sweeping stairs, cleaning public washrooms, organizing magazine racks, and just about anything else that would involve keeping a place tidy and organized. A lot of Japanese people that I talk to barely notice the janitors who keep our public areas clean for barely minimum wage, but I tend to see them everywhere I go and occasionally stop to thank them3. I would love to capture some of these people doing the things they quietly do in a well-framed image that conveys not the action being performed, but the human behind the broom. The images I've captured thus far have been fraught with lighting errors and focus problems.

Distant Worlds

What science-loving geek wouldn't want to take pictures of distant planets? The better the resolution, the bigger the smile. There's no denying that the earth is pretty amazing and has a lot of subjects to photograph, but the universe is a big place. There's a lot more we can all see and share with each other.


  1. Being single and a workaholic, I didn't really have many opportunities to take photos outside from one or two road trips every year.

  2. 2,200 photos according to the internal counter, which I have never reset.

  3. Most janitors are not accustomed to being thanked, which is a shame.

Dishonest Rage

Not too long ago I was an avid reader of The Guardian, a British news agency that dubs itself "independent and impartial". For years I would read dozens of articles on the site every day, visiting in the morning, after lunch, and again before bed, to keep my fingers on the pulse of the world … so to speak. However, a few months back I came to the conclusion that The Guardian — and many sites like it — was openly lying in order to push its own agenda. A news organization cannot claim to be independent, impartial, or even trustworthy if they are distorting the facts of news stories on a regular basis1. That said, I find myself going back to the site once a week or so in order to look at the articles with a new perspective: what are they lying about this time?

Unfortunately, it didn't take long to find something.

The Last Supper - Leonardo da Vinci

An article written by "nobody"2 outlines the reasons behind a Chilean bishop's resignation just weeks after being honoured with the title by Pope Francis. The article implies that bishop Carlos Eugenio Irarrazaval believes there is no place for women at The Last Supper and that they "like to be in the back room".

As one would expect, these comments were sliced, diced, and shared out of context for "other people to decide" on. People got angry. Really, really angry. And bishop Irarrazaval tendered his resignation in an attempt to reduce the attacks against the Catholic Church, an institution with no shortage of blood on its hands for other sins. Here is an exact extract from the Guardian outlining the "suggestion" that there's a reason for no women to be seated at The Last Supper:

The Guardian's Take

The problem with this article is that it contains just 30 words, carefully chosen, from an 18-minute conversation with a CNN Chile interviewer. There is no context behind these words. There is no lead up to see how the words came into being. There is even a lovely ellipsis in the middle of the single longest sentence, where context is conveniently removed from what the former auxiliary bishop had to say.

In the May 23rd interview, the priest was asked about the role of women in the Church, to which he responded:

We all have to ensure that they can do what they may want to do. Obviously, Jesus Christ marked out for us certain guidelines, and if we want to be the Church of Jesus Christ, we have to be faithful to Jesus Christ. Jewish culture is a male dominated culture to this day. If you see a Jew walking down the street, the woman goes ten steps behind. But Jesus Christ breaks with that pattern. Jesus Christ converses with women, converses with the adulteress, with the Samaritan woman. Jesus Christ let women care for him.

It is true that at the Last Supper there was no woman seated at the table, and we also have to respect that. Jesus Christ made choices and he didn’t do it ideologically.

Context is important. Letting a person speak is important. If I were asked why there were no women steated in The Last Supper, I would probably say something along the lines of:

Who am I to suggest what Jesus should have done?

The Guardian, and sites just like it, are intentionally spinning fabrications and willfully misrepresenting people and institutions in an effort to make them appear as some form of -ist or some form of -phobic regardless of the truth. For some people — whether it was justified or not — this has resulted in a completely destroyed career. For others it has resulted in an endless stream of abuse being hurled their way online and in public. And for what? To discredit "the patriarchy"? To "raise awareness" of perceived injustices in the world?

There are a lot of real injustices that need our attention but it seems that, in much of the English-speaking world3, the primary goal of our modern-day soapbox preachers is to supplant rational and contextual discussion with collectivist dogma that offers a person nothing of value beyond cognitive subjugation. We've seen how this plays out on a number of occasions in our history. It's often messy. It's often wasteful. It's often completely preventable. But only if people maintain their right to individualism and critical thinking, both of which have been under attack for as long as I've been alive.


  1. Everything on The Guardian involving Trump and the current American government I consider to be severely distorted to the point where it cannot be relied upon. Even when the writers are telling the truth, the historical dishonesty of the publication makes me suspect of everything they have to say. What's their agenda? What's being taken out of context? What's being twisted? Is this article different from the willfully warped ones?

  2. Generally, when an article is attributed to Reuters or some other non-individual, the content of the message is highly suspect and likely taken out of context. If nobody is willing to have their name next to an article, then it's perfectly reasonable to consider it "Fake News" … a term I never thought I would ever utter.

  3. I can only speak about the stuff I read in English, Japanese, and Korean. Asia has it's fair share of problems, but the attacks on individuals are generally done based on societal taboo breaking, not taking a handful of words out of context to generate an alternate narrative.

Outgrowing Hardware

There is a small collection of tools that I depend on every day to help pay the bills and keep food on the table, three of which are in the photo below. A MacBook Pro that I've been using day in and day out1 for just over three years, a 24" 4K Dell P2415Q monitor, and an iPad Air 2. The monitor and iPad are just fine for what I ask of them. The MacBook, however, is starting to struggle with the workload.

The Workspace

This is the 17th computer that I've owned since 19942, and it's the one that has been with me since the promotion out of the classroom and into a development role at the day job. The keyboard — which has very obvious signs of wear — is quite comfortable and the battery is still good for 5+ hours when plugged into the 4K monitor3 and a full workday if it's running standalone. The screen has some occasionally noticeable issues, but is still better than 99% of notebooks I see people using when out and about. The CPU4 can generally keep up with what I'm doing, too, as my job basically involves working with text. Lots and lots of text. What I need — and I understand this is a common gripe I've had with my recent machines — is more RAM and a faster storage device.

Because this is a modern MacBook Pro, there is no chance of upgrading the RAM. It already has the maximum 16GB installed and there's no way to add more, even if I were to try and get creative with a soldering iron. At the day job I've had to start working with Java, and this is just using all of the memory I have available on the notebook. 24GB would be a safe number for now, and 32 would offer that extra bit of flexibility when it comes time to run some heavier workloads locally. As this is not at all possible without replacing the machine for a recent 15" model, I can't even entertain the thought.

The storage card, however, is a different story. I've recently learned that it's possible to swap out the custom solid state drive for an NVMe device, which would just about double my read and write speeds. One of the things I really liked about the Lenovo X1 Carbon that I was using with Linux5 was its incredibly fast NVMe drive. It could write data at 1,200MB/sec and read at just over 1,400MB/sec. With these kinds of speeds I was consistently impressed with how quickly I/O-heavy database work was completed. The MacBook Pro cannot get the same speeds from an NVMe card, as the older notebook is limited to just two PCIe channels rather than the 4 available on many newer computers, but double the current speed would be a definite improvement. It would also be nice to go from 256GB of storage to a full terabyte. The cost of this upgrade would be about $250 all told … which may be worth it.

Will this buy the machine more time, though?

It's a strange feeling to "outgrow a tool". While there can be frustrations when working with a device that is clearly struggling to keep up with you, where the fans are spinning as fast as they can go and the case is going from "noticeably warm" to "I'd better stop and let the computer cool down a bit", there is also a great deal of sentiment embedded in the machine. This is the computer I used to create one of the best pieces of software I've ever written. This is the computer I used to write the vast majority of 10Cv4 and 10Cv5. This is the computer that has been used to clean up and edit a whole host of photos of the boy and Nozomi. I've been angry at the machine, and through the machine. I've also been incredibly happy, thoughtful, captivated, and reflective while using the machine. One can easily think about replacing a computer … but then what do you do with it? I am not one to let a computer sit around collecting dust when it can be useful in some capacity.

The bulk of my issues could be eliminated if it were possible to upgrade the RAM. Unfortunately this is simply not feasible. The remaining options are to upgrade the SSD to an NVMe drive to reduce the amount of time waiting while the system swaps memory from RAM to the storage medium, replacing the machine entirely6, or doing nothing and sticking with what I've used almost daily for 1000 days. The machine is still "young" by modern standards. If it weren't for the current workload at the day job, this machine would still be sufficient. There must be another option that I'm not seeing.


  1. Aside from a 4-month stint with a Lenovo where I tried very hard to be 100% Ubuntu. I gave this up almost a month ago due to the excessive amount of friction that I was facing when working with colleagues who, for the most part, use Windows. Ubuntu is a pretty decent operating system with a lot of software available for people to get work done, and I love the freedom that comes with running a proper Linux-based desktop. That said … the friction was just too much.

  2. There was a time not too long ago when it was normal for a geek to replace their computer every 12 ~ 18 months, either through upgrades or a direct replacement. Fortunately this is no longer necessary.

  3. I generally try to run off battery every couple of days to ensure the cell cycle. Maybe this isn't necessary, but I have an aversion to leaving a mobile computer — that needs to be mobile — plugged in all the time.

  4. The processor is a 5th generation Core i5 5257U running at about 2.7GHz.

  5. Aside from the keyboard. The ThinkPads still have one of the best notebook keyboards on the market.

  6. This would be tricky and expensive. I would likely aim for a 27" iMac, as that has expandable RAM.

Unpublishable

There's a topic that I've wanted to write about for years in order to better formulate my thoughts. Any time I've tried to have a conversation with people on the matter the end result has never been very satisfying. By writing about the subject, I'm forced to slow down and really think about what's being written and how things are being laid out. Do the points build on each other? Is the central theme followed the entire time? Are digressions and tangents kept to a minimum? By following the basic principles of writing, it's possible to put together an argument on just about any topic.

Yesterday I finished writing one of the longest pieces I've considered for this site. It weighed in at 7,218 words and covered quite a bit of ground. A number of memories were shared, conversations with people summarized, and a couple of religious texts were even referenced, all culminating in the 5-word central thesis of the piece. After reading through the essay twice, I decided that the item was ultimately unpublishable as it could result in some potential problems if my life insurance provider ever stopped by to read the article.

This got me thinking about what sorts of topics would be considered taboo to write about for a person who happens to find themselves gainfully employed with a family, a mortgage, car payments, various insurance policies, and a slow-but-steady freelance client base. To say that "nothing is taboo on the Internet" would be demonstrably false, yet there does seem to be a bit more freedom for a person to discuss ideas that might be unpopular if spoken aloud, particularly if something is published under a pseudonym or anonymously1. There are obvious taboos on posts that would promote violence against others, such as lynching, rape, and murder. There might even be taboos on posts that would intentionally mislead people, which are perhaps best exemplified by "Flat Earthers". That said, there are a lot of topics between claiming the world is a disc and pictures of kittens — the safest topic in the known universe.

Over the years I've written a number of posts that were originally intended for publication on this or another blog only to keep them offline for fear of reprisal at some point in the future. None were quite as long as the essay from yesterday, though some could have certainly resulted in some heated words2. Is it right to keep these offline? Or would this be an ideal use case for password-protected posts?


  1. Real anonymity on the Internet is very hard to find. It's often best to understand that there is no anonymity online without putting in a lot of work to hide who you are and where you're connecting from.

  2. Some of the opinion pieces I've written about the territorial disputes concerning islands around Japan have certainly resulted in some hate from drive-by commenters.

Apricot Cranberry Oatmeal Cookies

Earlier today posted a picture of some oatmeal cookies his daughter made and it brought back a bunch of memories of being in the kitchen with my mother a quarter century ago, making different kinds of cookies and seeing which ones people enjoyed the most1. Several years later, when I was living in Vancouver, I was out with friends and a cookie shop beckoned me to spend money. Inside there was the standard fare such as chocolate chip, double-chocolate chip, and even a double-double chocolate chip2. What caught my attention, though, was an oatmeal cookie that had cranberries and apricot. In Vancouver, a treat like this is called a "breakfast cookie".

As one would expect, I picked up several.

Apricot Cranberry Oatmeal Cookies

A couple years ago when I started compiling recipes from family to use in my own kitchen, I remembered this cookie and went looking for a comparable recipe online. The recipes I tried weren't quite the same but, with some experimentation, I think I landed on a pretty close facsimile.

Ingredients:

  • 110g unsalted butter, softened (1/2 cup)
  • ⅔ cup brown sugar, packed
  • 1 egg
  • ½ tsp vanilla extract
  • ¾ cup plain flour
  • ½ tsp baking soda
  • ½ tsp cinnamon
  • ¼ - ½ tsp sea salt, to taste
  • 1½ cups rolled oats
  • 1 cup dried apricots, diced
  • ¾ cup dried cranberries

Directions:

  • Line a pair of trays with baking paper
  • Using an electric mixer, beat the butter and brown sugar until combined
  • Add the egg and vanilla and beat until smooth
  • Turn the mixer onto a low speed, and add the flour, baking soda, cinnamon, salt and oats, just until everything sticks together
  • Stir in the apricots and cranberries
  • Use a tablespoon to scoop TimBit®-sized3 balls of dough on the prepared trays
  • Flatten the dough just a bit
  • Refrigerate the trays for about 30 minutes, which ensures the cookies bake up thick
  • Several minutes into the chilling time, preheat the oven to 180˚C (350˚F)
  • Bake the cookies for about 10 minutes, until the edges are golden and the centres still soft

This will make about 14 decent-sized cookies and the house will smell absolutely delectable. The last time I made these — about a year ago — the neighbours across the street came over to see what was in the oven. They were surprised to find that I was the one baking the cookies and they were more surprised when they had one. Oatmeal is not very popular in Japan for some reason, but the batch of treats I made didn't make it to the end of the next day.

I may just need to set a reminder to make these in the autumn when cranberries are not priced into the stratosphere.


  1. Everyone seemed to love the sugar cookies the most, though I was always a fan of the crunchy peanut butter cookies. Mum put a stop to these after a while, though, because peanut butter started getting quite expensive in the mid-90s.

  2. This was named after the excessively sweet, excessively creamy Tim Horton's coffee order that was so popular at the time. It was a milk chocolate chip cookie made with coffee. At around $3 per cookie, I never tried it.

  3. A TimBit® would be the "donut holes" sold at Tim Horton's. These would be about 3cm in diameter, give or take.

Reassurance

While I find it a little hard to believe, the boy's nightly routine has clearly made me more approachable at the day job. The implementation differs, but the similarities I see between putting a two year-old boy to sleep is not much different from working with middle managers. This isn't to say that those unfortunate souls sandwiched between C-Level1 managers and local managers act like children2, or that my son acts like a middle-aged adult. They're quite different in a number of regards. What the two have in common, though, is the requirement for specific questions and regular reassurance.

When the boy goes to bed, I generally stay with him for about half an hour so that he can settle down and get ready for sleep. We read some books. He signs a song or two. I tuck him in and arrange his toys in a manner so that he can reach them without becoming untucked. Questions are asked with every activity. "Which book would you like to read first?", "Do you like the bear in this story?", "What song are you signing?", "Where would you like your raccoon3?". The questions encourage dialog and give him the illusion of control. Once everything is set up and done, I'll rub his chest and wish him good night before heading downstairs to continue working.

Every so often, though, he'll start to cry shortly after because … who knows? At this point I need to go upstairs, ask how he's doing4, then calm him down by saying everything's okay and that he's safe. Sometimes I'll need to tuck him in again. Sometimes he'll ask for something and, so long as it's within reason, I'll get it for him. This process might repeat three, four, or five times on any given night depending on how tired the boy is. The more tired, the longer it takes for him to sleep.

Patience is an absolute must when working with children. The world is full of unknowns and they lack the knowledge, skills, and experience to make sense of it all. It's hardly any wonder that kids need consistent feedback and reassurance.

One thing that I've noticed over the last couple of months is that a lot of the management team at the day job also needs consistent feedback and reassurance. With the global project being in full swing and rapidly approaching the next critical phase, people are understandably anxious and nervous. People's weekends are at stake if anything goes wrong. People's bonuses, too. An endless stream of emails hit the inbox daily from people who have a lot of vague, superfluously-worded questions that boil down to the same underlying query: Are we okay?

Just as with the boy, I've found myself responding to messages like this by first clearly outlining the current status of my team's tasks, then asking some specific yes/no questions that are related to the reader, followed immediately by some follow-up information or an answer. In one of today's emails, this looked like:

… Did you have a chance to read the weekly summary report that {person} sent out Monday evening? She's about three days ahead of schedule on her end, so we should be ready to start testing the new data before the weekend. If that's too early for you, then we can stick with the current timeline and begin testing next Tuesday.

A year or two ago I would have likely been much more direct and to the point, answering just the questions I was asked without thinking about the underlying reason for those questions. What's worse is that I would tend to give answers in context, providing details that would have an email stretch on for several paragraphs beyond the point a person might stop. The context I provided was only for the answer, but people aren't interested in the answer. They're interested in the answer to the question they didn't specifically ask: Is everything okay?

This is probably something that "everybody" discovers at a young age, however, I've not been very successful in learning how to better interact with people until the last couple of years and there's still a lot I don't understand. Fortunately there is still time to catch up.


  1. CEO, CIO, COO, etc.

  2. Well …

  3. He has a stuffed toy raccoon, not a real animal. He's given this character the name "Chilli", but I can't for the life of me figure out why.

  4. It's a rhetorical question, but it encourages him to try and voice why he's upset.

From Data to Information

For a lot of people it seems there is no difference between data and information. In reality, the two could not be more different. Data is primarily unstructured or semi-structured elements that exist without meaning or context. Information is structured data that has context and can be used to make decisions. One of the first questions I ask when I begin to design systems that will store data is what sort of information a person wants to get out of the tool later. The answer will fundamentally guide the sorts of data that is collected to ensure that the desired information can be returned and, more importantly, be expanded upon as time goes on. This second part is just as crucial as the first.

Earlier today I found myself reverse-engineering a system designed by a vendor in order to solve a rather serious business problem created by the very same system. The problem amounted to an API that was designed to present a great deal of data rather than actionable information. As a result, teachers in the classroom have found it incredibly difficult to deliver their lessons. What struck me as odd about the software is that an API is generally used to present data in a structure, thereby ensuring it is parsed as information. This particular tool, however, appeared to have a consistent structure but wound up being little more than a data dump of keys and values. It was up to the JavaScript that read the data to determine the context and convert the data into information. Unfortunately, the implementation resulted in a website that would crash older tablets or present just a partial subset of information, which put the onus of "filling in the gaps" onto the teacher who had neither the time nor the resources to do anything of the sort.

So, being the corporate fool, I quietly waded into the mess and started reverse-engineering this system in order to extract the data to populate a database of my own design, then structure the data in a logical manner for the business need, then present it to the teacher in a format they can use. Given that this is a system designed to show textbooks, the lack of structure and clarity in the vendor's system has me questioning whether they understood the actual problem the business needed to solve in the first place.

In the space of six hours, I managed to reverse out the entire system and copy the bulk of the data from the vendor's system into my own, then build a preliminary API structure to return to a browser. Tomorrow's task will be to take the information and turn it into a textbook with the same formatting and features, plus a bunch of other details that should have existed from the first day the system went live. Two days of work on my part and a brand new system can replace nearly two years of development from a high-priced vendor. This sort of turn-around for problem-solving solutions is probably why a lot of senior managers at the day job allow me to break rules from time to time.

While I can generally turn around and solve problems like this through sheer force of will1, how can others avoid making the mistake of leaving data bloated and without form?

It comes down to understanding what a person wants out of the system, that early question I ask before writing the first line of code.

For this example, the goal of the project was to have an API return enough information to dynamically construct a textbook. Leaving the front-end code out of this, what would an effective structure be for a textbook, or a group of textbooks? Let's break down what sort of data makes up the information that is a textbook.

At a minimum we would need:

  • title ⇢ the title of the book
  • chapters ⇢ the sections of the book, allowing for a table of contents to be built
  • pages ⇢ the pages associated with the book, and possibly a chapter object

There is a whole host of meta data that could be included, such as a cover image, authors, publisher, ISBN numbers, MSRP, inventory on hand, search keywords, access permissions, and the like. The sky is really the limit when it comes to metadata, but the receiving software needn't be overloaded with data it never reads. If an API is going to return structured data, most of it should be used. If a complete dataset is only sometimes required, then an API filter show allow an application to request a limited amount of data or the whole shebang. What's nice about going this route is that websites that call the API will not be returning large amounts of data to discard or uselessly store. The less data there is to transfer, the faster everything can operate.

The original API decided to include everything about a digital textbook, including elements that would never be read by the front-end code. Details relating to the source system with index keys and when the chapter or page was last edited in that tertiary system. Details outlining the amount of storage space remaining on the API server, which is of no value unless regularly uploading. Details that appeared to be just random numbers thrown into an array. Details that included the address and contact information for the publisher of the book … which was attached to every page object, resulting in 477 sets of duplicated publisher information for one common textbook. The entire package was 6.68 MB to download, which took an average of 4.1 seconds.

Not cool.

My solution, which is probably not the best solution, stripped a lot of this information out. I put the title, chapters, and pages into their own objects and ensured the basic metadata was in place to show ISBN numbers, and similar details. The entire package now weighs in at 682 KB and can be downloaded in under a quarter second. With some compression on the server, the JSON object can be reduced even further and expanded at the browser. The next step is to replicate the front end with less code and more functionality to aid the teacher in the classroom.

How did this happen?

The people who made the current system are not stupid. I've worked with them on a number of occasions in the past and know the main developers are doing the best they can within the bounds of the client-vendor relationship. One of the problems that I've seen time and again, though, is that people often fail to ask about the ultimate goal of any system. This one started out with a colleague saying "We need a digital textbook system" and then answering a hundred questions around the idea. Looking at the early notes from the project2. Not once did the question of "What does the teacher see?" get asked. Heck, from the meeting notes, that question wasn't asked until 7 months into the project! Well after the database and API were designed.

I'll admit that I tend to look at business problems from the point of view of the person who'll be stuck using the things I create rather than the managers authorizing my wage. This often means that I may not create something that leaders ask for and instead provide the solution their people want, which involves quickly turning data into information and getting the heck out of the way. Being an internal resource means I have a lot more flexibility and access to these people than a vendor might, which gives me an unfair advantage. Fortunately it's one that the right people have appreciated a few times in the past.

When it comes time to solve a business problem, one of the very first questions needs to be "what do you want out of the solution?" Everything else is just window dressing.


  1. Sheer force of will … and a quarter-century of experience writing software. I've made every mistake in the book, plus a bunch that have never been documented. It's important to remember past mistakes and their solutions so that future endeavours can be more successful from the start.

  2. Everything is recorded in JIRA … which is both a good and sad thing. Good because documentation is key. Sad because someone had to put all of this stuff into JIRA.