Matigo dot See, eh?

Jussi recently wrote a bit about competency and it got me thinking in a similar vein, though not so much about skillsets. In his article, Jussi explores some of his competencies and how they've evolved over the course of his professional career. There have been victories. There have been struggles. There have been moments of doubt as Imposter Syndrome kicks in. All of these are familiar territory for the vast majority of people who aim to take their responsibilities seriously, something many of us can relate with. For me, though, I've been less concerned with the question of What am I good at? and more focussed on the question of What drives me?, because this is the font for where learning begins.

Much of the past few years has been rough for a lot of us. Professionally speaking, however, I feel that there was almost nothing of note that I accomplished between 2019 and mid-2021. Yeah, I hammered out a lot of code, I solved a lot of complex problems, and I managed to create some interesting things … but the vast majority of it has been discarded or drastically modified by colleagues for one reason or another. The fact that people discard or modify my work is fine, as this is the nature of software development, but the political reasons behind those decisions are what truly disappoint me and lead to the conclusion that two and a half years of effort were ultimately worth zilch. Now I find myself working on projects for a marketing department and Human Resources, two of the least interesting departments in any organisation¹, yet the work is almost invigorating. I actually look forward to sitting down at the keyboard every morning.

But why?

Problems exist, and they seek an answer. This is what drives me. A lot of creative people are likely engaged by the same sort of concept. When I try to solve a problem, I look at it from the perspective of someone who really doesn't like computers. Someone who, when asked about various aspects of software, will shrug their shoulders and say "Whatever. I don't care." only to later complain to someone else that digital tools are horrible substitutes for paper forms … or something thereabouts. When I approach a problem, I look at how it could be simplified from the perspective of the person forced to use the system. Is data entry kept to a minimum? Are required fields pre-populated whenever possible? Is there anything that can be removed from the main page without sacrificing something that a manager somewhere will insist is "absolutely essential"? All of these things are considered, plus a hundred more. Data is collected from a myriad of systems to reduce the amount of effort a person needs to expend. Items that can be inferred are presented in a list saying "Would you like to do X to all of these?". Every aspect of the UI is considered to ensure it works as seamlessly on a tablet as it might on a 40" projector.

This is what drives me: I embellish myself with the complexity of making something simple. When nothing more can be removed from a page, that is when I am complete. Do I always succeed? No. Do people notice that the systems I produce are generally faster and contain less friction than those they replace? Absolutely.

Software development is a two-way process. I can't possibly understand everything someone on the other end of the screen might be facing, so I keep the communication lines open. People are encouraged to get in touch to complain, to praise, and – most importantly – to recommend. When the people who are expected to use my software get in touch to say that something isn't very good and they have a suggestion on how to make it better, I pay very close attention. If what they have to say is well-reasoned, their suggestions will be part of the next update. If a person is just reaching out to complain because "reasons", then their concerns might result in updates that weren't specifically requested. Either way, it's this continual refinement of the solution that encourages further development.

My software is not great. There are scores of projects to prove this. However, with actionable feedback from the people who use the systems, they become less bad. This is what drives me: iterating to make something better.

1. I understand that Marketing and HR departments are important for many organisations, but the technical work required is rarely challenging. Instead, it's mostly "checkbox development", which is the worst sort of project a creative person can endure.

The Geminid meteor shower has been pretty nice to watch this year, thanks to the clear sky and reduced air traffic. For people living in Japan, this is one of the better regular meteor showers to observe during the year as there is very little haze in the sky from humidity or dust from distant deserts to obscure the heavens. I've looked forward to December for the past decade primarily because of this meteor shower.

Nozomi, however, prefers to keep her nose to the ground.

When Nozomi and I head out for our evening walks mid-December, she tends to get a little bit of extra time in the park. We survey the darkest corners of the public space, which often makes it possible for me to observe some of the smaller rocks passing through our atmosphere while the puppy does her thing and focuses on the scents of the less-travelled pathway. I'll sometimes let her know that a bright shooting star passed overhead, knowing full well that she's both deaf and completely uninterested in anything higher than two metres from the ground. It doesn't matter, though. I enjoy staring at the sky and sharing the excitement that comes from the various lights. She enjoys sniffing the ground and piecing together who or what might have travelled the same path recently. We both get our exercise and we both have an opportunity to enjoy some of the wonders that life offers … even if they're in completely opposite directions.

A few days ago I wrote about participating in some demo cryptocurrency challenges and, despite saying that they're not very fun anymore, I've kept joining the weekly challenges. The excuse I tell myself is that this is an opportune way to see how people will try to game a financial system in order to spot similar patterns in the "real" market. This has proven to be completely baseless, though, as the patterns that are seen in the demos are clearly vicious to the point of criminality. There's the saying "don't hate the player, hate the game" but, when I look at how easy it is to manipulate the selling price of a cryptocurrency that is operating in a sandbox environment with tens of thousands of participants, I can't help but wonder what sort of manipulation is happening with the markets where people actually invest a substantial amount of their savings.

Maybe the real reason I'm still doing these things is because I'm looking for the thrill that comes when a person gambles, even with funny money.

Shortly after the most recent blog post, the weekly challenge reset so that everyone had $5,000 USD to invest in various coins again. I had pulled in a respectable $2-million after seven days, which was quite a bit better than the $10,500 USD the week before. However, during the third challenge, I invested a bit more time into spotting the patterns that bots would use in order to artificially inflate the value of a currency, then sell out, only to repeat the cycle on a predictable basis. Doing this from my phone or tablet for 20-minutes here and there throughout the day proved to be rather profitable. By the end of the challenge, the total earnings for the third week turned out to be $216-million:

Being somewhat of a pessimist as of late, this seemed like the maximum one could expect. Taking five grand and turning it into $216-million over the span of seven days while also spending a great deal of time offline for work, family, sleep, and walking the dog seems almost absurd. The best investors on the planet would likely struggle to do something similar in the real market, and I could see over the course of the week that the bots that were manipulating the price of coins were starting to notice my interference. There were changes afoot.

Yesterday, being Monday, saw the start of the fourth week. Once again, everyone was reset to $5,000 USD to invest in various cryptocurrencies. I put half into Etherium and sat on the remaining $2,500 until lunchtime. However, when I checked my balance just before noon, I found that the price-tampering bots had all but destroyed the value of Etherium. The coin was worthless, leaving me with just $2,505 in total to try and invest with. Just as before, I started looking through the different coins for volume patterns. Which ones were up more than usual? Which ones were unnaturally down? Which ones were fluctuating at regular intervals?

It's this last pattern that is the most pernicious and profitable. If I were a Ferengi, my lobes would tingle every time I stumbled across one of these repeating patterns. They're easy to exploit, too. A currency goes up like a rocket to some ridiculous value, usually north of $300,000 USD, then gradually drops to its expected price. Then the pattern repeats. And repeats. And repeats again. If there are no people interfering with the bots, this cycle can repeat for hours, "earning" the machines trillions in profits. However, once people start buying and selling, the bots react very quickly in an effort to reduce their losses. So long as a person pays attention and reacts fast enough, though, they can earn several million in an hour. Or, in my case, over a billion:

Today is the second day of the week-long challenge.

As the screenshots will show, I'm doing all of this with an iOS application. I have no bots manipulating prices or watching the market on my behalf, nor do I have any intention to get into that game. However, I am curious to know who has made these bots and what their goal might be. Manipulating demo markets may be fine as an academic exercise in how financial markets can be gamed by people with excessive wealth, but this is hardly news. Wealthy people have been gaming systems for millennia. Could this be just something a few people are doing "for the lulz"? Maybe this is just a testing ground before the bot(s) are released on the real markets with a decent-sized initial investment? Maybe this is just all in my head and the cryptocurrency fluctuations are the result of a poorly-implemented faux supply-demand mechanism in the demo provider's service. I really do not know.

All this aside, given my propensity for numbers, I have a feeling the personal goal will be to consistently out-do my previous week's worth of pseudo-plundering.

The title is very misleading, as this $2-million isn't real money, but cryptocurrency. Heck, it's not even cryptocurrency, but "Demo Trading" of crypto currencies. Still, on Friday night I went to bed thinking I was doing pretty good having turned a $5,000 USD starting balance into $10,500 USD after five days. This morning I noticed a pattern and, because this is all a demo, decided to see how far I could take it. The result was unexpected:

My actual experience with cryptocurrencies have been far less interesting to write about. Like a lot of people, I have put real assets¹ into various coins only to have 35% of it lost due to market fluctuations, various fees, and bogus ROI claims². That said, it's something that I'm trying to learn more about as it seems to be a viable means of hedging against inflation for a lot of people around the world. I'm not too worried about inflation in Japan just yet, but I am looking for some decent alternatives to having money sitting in a bank account collecting 0.02% interest per year … which is considered "good" here.

Given my actual performance with these digital assets, I've been looking at ways to try and gain a bit more experience in order to lose less. The general rule of thumb is that anyone hoping to earn a quick buck should play at the casino rather than with cryptocurrencies, but I see the two as being about the same in terms of return until you start working with much larger numbers. A lot of the demo trading that I have done has generally started with a sum of about $5,000 USD. Some exchanges will offer this in various coins at the beginning while others will give you a local token that can be used to buy the currencies of your choice. For this most recent run, I used the OKEx exchange and was provided with $5,000 in the local USDT token to acquire things of my own choosing.

As one would expect, the first couple of days were slow. There were gains and losses as bids were won and lost. There was also a rather large dip in all the cryptocurrencies mid-week – in the real world – that also wiped out a good bit of capital for the people participating in the demo trading challenges. However, by paying attention and sticking mainly to the coins that I was most familiar with, it was possible to take the initial investment and double it over a period of five days. One of the most effective ways to do this, I found, was to buy and sell using either Bitcoin or Etherium, as this resulted in much lower fees, leaving more for profit. All in all, I needed just 35 trades to turn $5K into $10K.

It's easy to see why some people get sucked into this world and invest just about everything they have. Such short term gains are simply impossible in the real world.

On Saturday morning I noticed a pattern. Some less-common coins showed an interesting up and down pattern. Following the pattern with a 1-minute chart, it was easy to see that a coin would drop to a set value, then slowly rise to a set value before the process repeated again. If you timed it right, you could buy low and sell high numerous times. So I did. Twelve hours later the $10K had swelled to just under $90K. Not bad, considering it was the weekend and I couldn't be in front of a computer or phone all day. This morning the patterns were seen with different currencies, and with much larger margins. I could buy certain coins for $2 and sell them five minutes later for $35. Buy, sell. Buy, sell. Buy, sell. Stop. The patterns were usually good no more than five times before whatever mechanism was manipulating the prices would stop and move on to another currently. Still, by paying attention, the $90K had grown to half a million … then a million … then one-point-five … and it didn't stop. There were times when I would play the game for too long and end up losing – in one instance – $400K in worthless coins, but the gains far outweighed the losses.

Heck, in the time I've spent writing this blog post …

The real markets don't operate like this.

One does have to wonder how many people will do these demo market challenges – often with the promise of earning a small amount of Bitcoin or something else for successful outcomes – and over-estimate their abilities at the end. How many people finish these little games and think they can turn an investment of a few thousand dollars into double or triple the value within a week? Are there people that invest their entire line of credit into the real markets only to lose it all? Given the number of people participating in the cryptocurrency markets, there are bound to be a noticeable percentage of n00bs losing everything.

As with every type of investing, it's important to do the research, understand the risks, and never put in more than you are willing to lose.

1. Sometimes money, sometimes time, sometimes 32.68TB of network storage space.

2. The bogus ROI comes from Chia, which I won't get too much into at this point. As it stands, I've invested over 32TB of storage space for hash tables, not to mention all the electricity, computational power, and I/O capacity required to build those tables, only to get nothing in return. Not even a tease of a matched hash.

Four hundred and six days have passed since the previous "Five Things" post, though it certainly doesn't feel like so much time has elapsed. Between November 2018 and April 2020 there would often be at least one instalment per month, generally written from an iPad on a Sunday night while I waited for the boy to fall asleep¹². While there's no plan to return to a weekly "Five Things" schedule, it's not something I'll rule out at some point in the future. That said, the reason for this stream-of-consciousness post isn't really the meta subject of the post itself, but some of the recent ideas that have been echoing around my head for a little while now.

Old Notes. New Appreciations.

While cleaning the home office the other day, I happened to open one of my old development notebooks. These are B5-sized books containing 250 pages of graph paper, and they're the perfect size for me to carry around, jot notes, sketch interfaces, plan APIs or databases, and record just about anything else I might need in a free-flowing fashion that is unbound by the constraints that limit most note-taking applications on computers. Without the flexibility of pen and paper, I highly doubt I would be able to write as much software … as counter-intuitive as that may sound. Looking inside, I saw some of the concepts that were being sketched out and considered right before I started building out the fifth version of Midori Core, which would eventually go on to power about two dozen different projects. Many of the ideas were rather different from what I had been doing beforehand, and some of them seemed downright impossible to implement at the time due to the way the MySQL database engine works. However, reading through the list, I was surprised to see how many of the ideas were not only part of v5, but had actively contributed to the success of the new version and are now indispensable. Support for SQL Server and PostgreSQL have certainly been game-changers, and the consistent API template makes developing a new set of functionality from scratch an absolute breeze compared to everything that came before. Then there's all the internal consistency checks and system validation ….

However, it's not so much the notes themselves that struck me, but what the notes made clear.

A Lack of Imagination

Over the last six months or so, I seem to have lost the ability to think up new things. Solving problems at the day job with creative solutions is still something that's done quite regularly, but it seems an incredible amount of time has passed since I've had a really interesting idea to think through. A lot of what I am occupied with – based on the contents of the current development notebook – is solving problems that people create for themselves … either wilfully or by sheer misfortune. A decade ago I would have several ideas on the go with regards to new projects that could be considered, slapped together, and tested. Today I have zero … and this really disappoints me. Where has my imagination gone?

Chasing Other People's Goals

Perhaps I've just been engrossed in other people's goals and haven't really sat down to think about what I want to do? Freelance work has started to pick up since the summer and there are now five clients that I am regularly communicating with. For this I am quite grateful. The day job also consumes about 50 hours of my time per week, which isn't bad given that the problems are rarely "urgent" anymore. Being able to take the time to really think a problem through before working on a solution is a luxury that many of us rarely have. Hopefully this relaxed pace keeps up for another couple of months so that the New Year holidays can be enjoyed without stressing over "all the things that remain to be done", as I have done the last three winters.

Still, it would be nice to have a new personal project to work on.

A Collaboration with Dad?

This summer I learned that my father's health is deteriorating quite rapidly. For a brief time, I thought I might have to arrange a flight back to Southern Ontario between the many on-again-off-again lockdowns in Japan and Canada. Fortunately, this has not been necessary, but the fact of the matter is that I have been thinking about the many stories my father has told me from his life as well as the ones we shared together. As the family historian, I would really like to have these written down to be shared across time, just as I did with my grandfather's journals at the very beginning of the 10Centuries project. While my father does like to write, he has never kept a journal. This means that if something happens, his stories can only be recounted by the people who knew him. I've been thinking that maybe this would be an ideal time to sit down with him, either in person or online, and record a series of podcasts. These would be made available to the family at a time when my father is comfortable having them released, and they would remain part of the 10C repository forever. If nothing else, this would be a good way to reconnect and get to know the man a little better, as long-distance communication does have its challenges.

Yet, as I think through how I would like to compile and present my father's life in a digital manner, the challenges I faced with my grandfather's journals echo in my head.

Is 10C the Right Tool?

The first version of the platform was about blogging. The second version was about blogging with immutable records and clearly-defined hierarchies, which made it possible for me to adapt decades of my grandfather's journals to a digital format without the tacky "solution" of uploading a bunch of scanned images. The third version … we don't talk about. The fourth version of 10C focused on ephemeral posts, podcasting, public APIs, and document formatting. The fifth and current version focussed on improving social posts, using "IndieWeb" HTML formatting standards, improving document formatting, improving privacy options, and overall speed. However, is the current version of 10C suited for a biography? My grandfather's archives read like a blog, because that's a format that I understood and could easily adapt the paper documents to. For my father, I'd like to do something a little more elegant. Something that is better in line with a biography that consists of pictures, text, and audio.

As I've alluded to yesterday and today, I've been thinking a lot about document structures. Since working on a couple of textbook delivery systems and learning management systems, I've developed a better appreciation for how educational materials consisting of varying digital assets need to be presented. I dislike using the term "multi-media", but this is essentially what modern textbooks have become. They're interactive, respond to touch, and a little more approachable.

Blogs show information chronologically, which makes sense given that's how we live. However, when we think about our history, we don't think about things in terms of days, but in groups of time. A holiday in a foreign land. The first week of high school. That summer when everything went "just right". Grouping blog posts can certainly be done, but it's often a clunky ordeal consisting of tenuous meta connections at best. An idea I've been toying with is treating posts as sections of a chapter, which reside in a book consisting of sections or volumes or any other means to represent a collection. A collection could also consist of additional collections, which have their own chapters and sections. This would be an incredibly hierarchical creation that could be as flat as a blog, with one single collection, or structured like some of the most complete biographies and autobiographies we see in libraries.

This is something that could be wedged into 10Cv5, but I wonder if it's the right way to go. The goal of 10Centuries is to act as a permanent digital repository of who we are for at least a thousand years. What I see today, however, is a social network with features like blogging and sharing links to articles from around the web with our remarks attached. While I am very happy to see a community of great people using the social aspects of the site, I do wonder if it would make more sense to split the project so that Nice is its own entity, and 10Centuries becomes focussed on the long-term objective of being a person's historical record. There would be overlap, of course, as well as integrations to facilitate communication between the two tools, but this could allow the two projects to evolve independent of each other, while continuing to fill the needs of the people who use the tools.

All of this is still in the air – and in the current development notebook – as a project like this would take time away from the conversations I would like to have with my father, but it's something I'm thinking about.

1. Just like many younger children, he did not like going to sleep alone. Heck, he still doesn't. So I generally stay with him until he drifts off to sleep.

2. Writing several hundred words from an ageing tablet – an iPad Air 2 from 2014 – is no longer my idea of fun, particularly now that the keyboard on iOS has made typos so much easier to create. That said, it has managed to survive three years as a daily machine at a school before it was passed to me, and I've used it for several hours every day since. iOS 15 has made it slow down quite noticeably, though.

An article in a local NYTimes subsidiary recently published an article on Delphi, a system with the goal of determining the ethics of an action. This is an important area of research as we start to see more machines that operate semi-autonomously and need to determine whether a course of action is correct or not. Obvious examples of where ethical decision-making is necessary would be with self-driving vehicles. If a vehicle carrying two passengers is travelling down the road and an oncoming car swerves into the same lane, the vehicle will need to quickly determine the correct course of action. Does it slam into the oncoming vehicle, potentially killing the passengers in both cars? Does it slam on the brakes, offering the offending vehicle more time and space to correct its lane transgression? Does it rapidly move into another lane or onto a sidewalk? Each one of these decisions requires information that may not be available and each will also carry consequences. So, given the lack of time to collect additional data, the unforgiving laws of physics governing bodies in motion, and the immediate need to take action, which action would be the most correct?

Decisions are considered right or wrong based on the ethics of the observer, and ethics are incredibly subjective. While some may feel that severing the right hand off a man is justifiable because they stole a loaf of bread, others would find it reprehensible. Is a politician right to use their position of authority to the benefit of their friends or family? Is a doctor right to over-prescribe medications to help a terminally-ill patient not feel pain? Is a teacher right to teach their students a government-mandated concept or belief? Is a parent right to teach their children that people from a particular nation or genetic lineage are not worthy of respect? Is an individual right to kill a mosquito that has landed on their skin? Is an individual right to kill a mosquito that is on a table? Is an individual right to kill every mosquito on the planet with a genetically-engineered virus?

Most people will have an immediate answer to each of these questions and not one of them is a clear "Yes" or "No". Our sense of right and wrong has been built – and continues to be built – based on our lived experience, the circumstances we've faced, and the people who surround us.

There have been a lot of articles over this past decade on how various vision-based and decision-based systems are racist, sexist, or otherwise discriminatory. I generally disagree with the authors of these pieces because to be in ~ist, one has to possess prejudice. Software can certainly be written to enforce any biases the developer (or organisation) may have, but machines generally do not care where a person's ancestors may have lived, which sexual organs someone may have, or how they use them. Instead, I see these vision or decision-based systems as being ignorant; they lack a sufficient amount of information to reach a better solution. Generally when this sort of problem arises at the day job, I say "the system (or idea) is incomplete". My concern is that software that tries to determine the moral outcomes of a decision will be just as incomplete.

Who are the people that define right and wrong for Delphi and systems like it? Generally it will be academics, philosophers, and a handful of committee members with strongly-held beliefs. Essentially, very intelligent people with similar ideas who live in a very well-protected bubble outside of the wider society. While they will most certainly do their best to codify modern morality in a manner that a set of dynamic algorithms can understand, there will be a great deal that they cannot prepare simply because they suffer from the same problem that we all have; ignorance. We do not know what we do not know. And, to add a little more complexity into the problem, two people in the same society will have different ideas of right and wrong.

For the moment the researchers who are investing so much effort into Delphi and systems like it are keeping their goals simple, understanding the complexities of the project. However, as we see more of our actions monitored by corporations and governments around the world, I worry that "ethical systems" like this will be used to seek out those who deviate from a prescribed moral code. China already does something similar to this with their various surveillance programs. The speed at which dissenters can be found and silenced is nothing short of remarkable terrifying and given the recent societal directions of some Western nations, it would not be surprising at all to learn in the near future that some countries are preparing to do the same.

The final week of November is traditionally when the autumn temperatures give way to winter's chill, and this year is no different than any other. However, one happy difference is that this year is expected to be much colder than we've experienced since 2015, when overnight temperatures dropped to -10˚C and we experienced almost a full week of snow on the ground … albeit split between a handful of days in January and March. With any luck, this coming winter will be just as chilly with even more snowy days for people to enjoy.

As COVID-19 has mostly disappeared from the country, people all over the nation have been going out to catch up on two years of fun. Theme parks are packed. Malls are over-crowded. Highways are full of out-of-prefecture license plates. It's nice to see and, as it seems a lot of the more expensive places are still hotspots for people to congregate, I've been looking for some quieter, winter-friendly places where the family and I might go.

The obvious one for people in this part of the country is 白川郷¹, a remote village about three hours north of my home that showcases a traditional building style known as gasshō-zukuri.

This is a place that I've wanted to see in the wintertime for a while now, as the pictures have always looked so inviting. If I could give up some of the modern conveniences like fast Internet speeds and reasonably-priced specialty breads, then a move to Shirakawagō would be incredibly tempting. Perhaps this is something I can consider in another quarter century when I retire.

Another place that I'd like to bring the family – if they would be willing – is the Number 1 Tadami River Bridge in 三島². This is an incredibly scenic part of the country and I've only recently learned about it.

What might give people pause is the fact that it's located in Fukushima prefecture, though quite a distance from the infamous nuclear reactors that suffered a partial meltdown and spewed radioactive dust across much of the northern part of the country. Most mapping systems say the trip can be made in about six and a half hours, which means spending the night and perhaps enjoying a number of other scenic locations. If the family does not want to go, maybe Nozomi would be willing to enjoy a little road trip north, as she was born at a place just a few hours east of the scenic bridge.

Distant photo spots aside, with a colder-than-recent winter on the way and greater freedom to move between places in the country, I'm hoping that a lot of people can get outside and release some of the tension that has built up since the tail end of 2019. We say this every winter, but here's hoping that next year is better than this one.

1. Shirakawagō

2. Mishima

A little over two years ago, I wrote a blog post about Gary; a friend of mine. In it I lament the fact that he has coasted through 35 years of living and does not understand the responsibilities that come with being an adult in a committed relationship. The bulk of my concerns were laid out right in the middle of the piece:

While I like the man a lot, he's not exactly a go-getter. He likes to take things easy and allow events to unfold rather than forge his own future. He's worked part time at the day job since the very beginning, and would often rely on his parents to cover various bills. Despite being in a committed relationship with a woman who is very serious about having children, he's unwilling to take on the requisite responsibilities that come with "being a man". He could switch over to a full-time contract at the day job and have a steady income with a five minute conversation, but doesn't want to. He says that the workload would be too exhausting. He's not particularly keen on using any additional skills or knowledge to earn money, as it would take time away from whatever it is he does when not at work. He's not even particularly interested in seeking out a better-paying job — of which there are many — as that would eliminate any seniority he might currently enjoy … which means almost squat at the day job unless you're a full-time or unionised employee. How in the world is he going to afford to support a family? […]

The fact that Gary has never had a full-time job worries me. Can he keep up with a child? Will he hand the young person off to his wife and abstain from his responsibilities? How will he afford the never-ending list of things to buy? Diapers, wet sheets, clothing, toys, books, community activities, school …. The list is almost endless and there is no avoiding them. Heck, what kind of example will he set for his child if he doesn't accept a heavier burden of responsibility? […]

The man is 35 years old. He's not a child, yet continues to shirk responsibility as though he's still in high school. I care about his future just as much as I care about his present, but any conversation where it's suggested he work harder is shut down much faster than it starts. There's no denying that life is undoubtedly more enjoyable when moving at a relaxed pace and the most pressing responsibility is paying this month's cell phone bills, but this is no environment to start a family. I want him, his wife, and any future children to succeed in life. How can I communicate this to him in a way that he'll accept?

What I wrote two years ago about Gary's attitude towards life is just as true today as it was then. However, in the 26 months that have passed Gary has gotten married, seen his hours at the day job dwindle due to COVID, moved in with his wife's parents in an effort to save money and, as of yesterday, become a father to a child that was born far too early.

Looking at the short list of facts, my friend is a 37 year old man who works less than 10 hours a week, has a wife and child who both need a great deal of medical attention¹, two cell phones, a car, and a remarkable lack of ambition. I've tried to help him find more steady work, but everything is knocked away.

Gary has lived in Japan for 12 years and still cannot speak the local language worth a darn. Sure, he can do simple things, like buy groceries or get a cell phone, but he won't make the effort to learn enough of the language to work at a school, which is a job he once wanted more than anything else. Being an ALT – an Assistant Language Teacher – at a public school is no walk in the park. People expect you to be present and contribute to the classes. They expect you to lead all listening and speaking activities. They expect you to jump from class to class and never complain about not having time for lunch. However, it's a solid job with a solid contract that pays a guaranteed amount every month … and schools all across the prefecture are practically begging people to come work with them. The barrier to entry is very low.

A work-life balance of 60/40? Let's take the "generous" number there of 60% work and figure out how many hours that is per week:

• 168 hours in a week
• 56 hours sleeping, assuming it's 8 hours in a day

That leaves 112 hours each week that Gary is awake. 60% of that would be 67.2 hours of work and 44.8 hours of family time. In an average week I do about 60 hours for the day job and 20 hours of freelance work Monday to Friday. I don't work on weekends. Gary hasn't worked 67.2 hours in a month for as long as I've known him.

He wants to be "50% in for child raising", which is a laudable goal. However, he forgets that if he's 50% in for child raising, that means his wife will need to be 50% in for paying the bills. Given the current situation, that's not only unrealistic, it's cruel.

The man has nothing in the bank. The money he earns every month is less than half of what I pay in taxes. He lives with his in-laws because the cheap apartment he had in town previously is too expensive. He needs to buy clothing, diapers, food, a car seat, furniture, linens, and a thousand other things that people try to prepare before a child comes into the world. His older car is expensive to keep on the road. His cell phones, while older models, are not cheap.

While just the other week …

You know … I'll take that back. I can't call Gary a man. Because he isn't. He's still just a boy playing "house" and not understanding just what it is that he is responsible for. A husband and father are responsible for the financial well-being of the family unit in addition to being a proper spouse and role model for children. This doesn't mean that women cannot go out and earn, or that men should earn more but, when a woman gives birth, the last thing someone should ask them is when they'll get back into the workforce because diapers and formula are running out. It's great that Gary wants to be there to raise his child. It's great that he's thinking about a work-life balance to make sure he doesn't miss out on the opportunity to see the world through a new set of eyes as though it's the first time. However, a person who is damn near unemployed does not have the luxury to dictate terms like this. He should be out working multiple jobs to put food on the table and diapers on bums. Yeah, it'll be rough. Yeah, he's 37 and employers are going to have a hard time justifying hiring someone with a resume that has just two jobs in the work history section, both of which state "Part Time". And yeah, he's going to miss some of the moments as his son learns to control his body and interact with the world. Gary's wife will be angry that so much of the child-rearing is being left to her while he's at work, and that's a completely justifiable complaint given that her husband has been employed – part time – for 22 years and doesn't have anything to show for it.

Gary and I have known each other for over two decades. We've talked about a million different things, including the importance of taking on responsibility and putting the needs of family ahead of self. Yet it seems the weightier discussions have gone in one ear and out the other, just as it did when Gary's father tried to instil in him a proper work ethic.

Despite the disappointment and frustration I feel towards the guy, I want him to succeed. I want him to be able to provide his family with more than just the basics of survival. I'd like to see him life in an apartment or house without the need for generous support – and patience – from the in-laws. Heck, I'd like to see him stand up to the challenge of learning Japanese and actually using it.

My Japanese skills are not great, and every day I find a larger gap between what I think I know and what I actually know. My son will be five years old in a couple of months and he comes home with new vocabulary that I need to look up and learn every single day. If I don't do this, it'll be just a couple of years before we can't even communicate with each other. Will Gary do the same for his child? Or will he expect everyone to speak English at home?

So much of this has been laid out to Gary over the last couple of years and particularly now that he's responsible for a vulnerable little human. Unfortunately, any time our conversations turn serious, he gets quiet and puts the phone down. This is true whether we're using text or having a call.

I want him to succeed. I want his family to know they can rely on Gary to do the hard work when times are tough. I want him to learn that it's not okay to be a self-centred, lazy piece of crap when people need him to make sacrifices for the betterment of the family unit.

Unfortunately, I don't think this is something I can help him with anymore.

Good luck, Gary. You're going to need it.

1. Japan has an excellent medical system with a good portion of doctor visits and examination costs covered by the government, but it's most certainly not free. I generally pay about $25 every time I see a doctor at a clinic and it's about $20 for the kid. If we have to go to a big hospital, we're looking at spending at least $50. My most recent visit had me go through an MRI and do some blood tests, which came out to $175. Not at all a ridiculous amount, but it can add up quickly.

Almost six months have passed since I transferred divisions at the day job, leaving behind a better-paying position with loads of opportunity for one that is far less visible, far less challenging, and far less important. The change was made for a long list of reasons and I'm satisfied with the effects of the transfer. Not only do I feel a lot less anxiety every time I sit down in front of the keyboard, but I am able to approach problems without feeling unnecessary pressure as seconds turn to minutes and minutes turn to hours. So great was the burden half a year ago that I stopped wearing a watch and hid anything that reported the current time in my office.

This new role sees me doing similar things as before, as people report problems and I work with them to find solutions. However, these problems are often self-inflicted wounds created by people who wanted to find a simple solution to a complex issue, only to create a black hole of "legacy" that everybody is now afraid to touch or modify. Fortunately, it's these messy situations that often have the greatest rewards. A person can sit down to understand the original and current problems, talk to the various people that interact or are impacted by the problem, and find a solution that works for everybody. If a person does this correctly, they'll be rewarded with a daily stream of questions from colleagues who now trust them to offer answers that are realistic and actionable.

A few weeks ago a long-standing problem at the day job was eliminated with a few thousand lines of code that required two complete rewrites to get correct. Generally a rewrite points to a failure to understand requirements while two rewrites might point to incompetence or worse. For me, it was a number of factors that collided to create a situation where a solution needed to be built to replicate existing functionality of a system that had no documentation, no upgrade path to work on modern servers, and nobody at the company who new the ins and outs of the project. It was a pet endeavour for a manager who has left the company.

However, just because something is hard to understand does not mean it's impossible to overcome. Andy Weir said it quite well in his famous book, The Martian:

“At some point, everything’s gonna go south and you’re going to say, this is it. This is how I end. Now, you can either accept that, or you can get to work. That’s all it is. You just begin. You do the math. You solve one problem and you solve the next one and then the next. And if you solve enough problems, you get to come home.”

– Mark Watney

This is how I tend to approach every problem at the day job. Projects that are expected to be easy may turn out to be multi-year undertakings that can change the direction of entire organisations. Objectives that have a long list of expectations can sometimes be solved with a well-designed Excel sheet that sits on an executive's notebook computer. Or, as is the case with this most recent effort, it can be an opportunity to solve hundreds of little problems, one after the other, until a complete solution is in place and a team of people you'd never worked with before sees the value you bring to the table.

Working in the global division at the day job certainly offered a lot of opportunity. A person with the right ambitions and office-political skills could easily work themselves into a C-level role within a decade, setting goals while coordinating resources and ostentatiously proclaiming success. That's not what I look for from a career, though. I like solving problems. Real problems. And experience has shown that the better problems to solve are the ones that a handful of people care about.

On May 8th, 2019 the foundation of what became the fifth version of the Midori Core PHP framework that I've built so many projects on top of was committed to GitHub. Midori Core 5.0 — generally referred to as v5 for some of the public-facing sites — was designed around the idea that the database should not only contain the bulk of the data validation rules for a system, but the queries as well. Prior to this major release, every query that was sent to the database came from a flat file that was populated with properly-escaped values. With v5 I wanted to try doing something that I had only ever done with .Net-based enterprise projects that relied on the might and versatility of SQL Server: put the bulk of all read and write operations into stored procedures.

This worked rather well for a while … and then MySQL, the database engine used by Midori Core for its entire history, developed a critical bug in version 8.0.22¹ that has yet to be fixed. This bug is encountered when a stored procedure is writing data to a table that has triggers in place that validate or transform data before it's written to disk. What this means is that just posting something to a 10C-powered site could crash the database, forcing it to restart; a process that takes about 10 seconds. During the restart, every site and service on 10C that requires reading non-cached data is non-operational. As one would expect, this situation is completely unacceptable. Bugs were filed. Data was shared. Teeth were gnashed. Unfortunately, a fix has not yet been issued and it doesn't look like anyone is going to get to it anytime soon. As a result, every system that uses Midori Core v5.x is forced to use no version of MySQL newer than 8.0.21.

However, v5 does have done thing going for it that none of the other versions of Midori Core could do: it can work with SQL Server and PostgreSQL.

Soon after MySQL started crashing every time a butterfly flapped its wings, I started thinking about making the back-end of the system work with other database engines. Microsoft's SQL Server was the first option, as it's the database I've worked with the longest and it does have a Linux version that can run quite well on any modern version of Ubuntu. PostgreSQL is a system I've avoided for a long time but have recently come to understand better as a result of a large project at work. Both of these options work quite well with stored procedures and triggers, making them viable alternatives to MySQL, and both have a large community of developers and administrators who genuinely seem to enjoy helping people when there are problems to overcome. Which one would be better for something like 10Centuries, though?

As with anything involving technology, the answer is "it depends".

I do have one system built on Midori Core that operates with a SQL Server back end. It's been running for well over a year and hasn't encountered a single database-related problem. The site isn't heavily trafficked, receiving perhaps 50-thousand requests per week, and runs on a t3.small EC2 instance on Amazon's Web Services. Average response time for most operations is under 90ms, which is acceptable.

Then there's the new system that I've been working on for the last few weeks which is using the same Midori Core v5 foundation as the other system, but with PostgreSQL handling the data. Many of the core tables are designed the same between the two databases. The functions that request data from these core tables are essentially the same, aside from some syntactic differences. Average response time for most operations is under 70ms, which is also acceptable.

Of course, the dozen or so systems that I've built with MySQL ask the back end also have relatively quick response times, but none quite as consistent as the projects running SQL Server or PostgreSQL. The key difference between the systems running MySQL and those that are not is the type of queries that are sent to the database. Since the MySQL bug has caused so much trouble with one of the underlying goals of the v5 project — namely, keeping all of the validation and interaction rules in the database itself — I've had to drastically simplify how systems running that database engine work with the data. This means more rules and validation happening in the middle-layer — the API — rather than the database. While this is fine for smaller projects involving a single developer, it can create "gaps" when multiple people work on the same project. Gaps are not good, as they lead to inconsistent results.

So with v5 now being compatible with three different relational databases — without the use of an ORM² — why would I think about the next major release of the core framework?

For something completely different: modularity.

The majority of the projects that I've built on the Midori Core have the same foundational files containing the same foundational functions that I have used for years. The patterns are consistent and the methodology is reliable. That said, when a bug — or security problem — is found in the core files, it's a pain to go through a dozen or more projects, find the offending code, patch it, then deploy releases across a myriad of servers. What I would really like to do is structure the projects in such a way that Midori Core has its own source code repository — which it does — and other projects are forks of the Core — which is not how things are structured right now. Then, when the Core receives updates, that can be rolled out to the other projects via pull requests that kick off some unit testing to ensure that there won't be serious problems when deploying the updated code in a number of different projects. By having the Core able to work with multiple database engines, I'm one step closer to having a better update process in place. The projects become more modular.

There are other items that I would want to tackle with a fresh version number, such as better caching mechanisms, a more consistent use of "history tables" in the database, more consistent use of CSS3 across sites to take advantage of visitor colour theming preferences, and more. All of this could be built into the v5 platform, perhaps as part of the upcoming 5.4 release of Midori Core, but I am very tempted to use this recent functional improvement as the impetus for a better-structured system that I can continue to build on for several years before thinking about the next problem to solve.

Does this mean that 10C will be upgrading to v6 in the near future? Not likely. Heck, 10C was using the v4 platform for a full 8 months after v5 was in a stable-enough place to handle the work. That said, I would be very curious to know whether a PostgreSQL-based 10Centuries would be noticeably faster than the current MySQL-powered one.

1. MySQL 8.0.22 was released on October 19, 2020. More than a year later, the bug that prevents me from upgrading the database continues to exist.

2. An Object Relation Model … basically a piece of software that makes my job 50x harder when something doesn't work as expected. I do not use ORMs in my projects, preferring the "old school" approach of writing the queries myself. This allows for some better tweaking and more nuanced questions.