Notes are Unstructured Groups of Structured Data

One of the very first applications I wrote for the Palm platform was a note-taking application designed for my specific use case at work, a home appliance repair centre, where I would typically receive hundreds of phone calls a day and needed to keep track of hordes of information with near-instant access. This was back in 1999, and the paper notes that my colleagues and I would share often got lost or misplaced depending on who the last person to touch "The Book" was. Luckily, because most of these notes had a very specific pattern to them, creating an application that would allow these notes to be recorded and synchronized between devices was a simple weekend activity.

We needed a name, a phone number, make and model of the machine, and a few notes to know what that person needed and whether the record was "done". Even at the ridiculously youthful age of 20, this program did not take too long to make and it worked very well for its purposes. After a month or so, we started adding refinements, such as not storing records that have been complete for more than a week and a desktop application that could also read the synchronized database, allowing us to search every record regardless of its age. As a combination To Do/Note-taking tool, it excelled.

That was nearly 20 years ago and, while I have long since moved on from the stressful world of appliance repair¹, I have not yet found a note-taking application that I can use for anything else as effectively.

Notebooks and Electronics.jpg

I take a lot of notes. Every day I fill a dozen or more pages of A5-sized paper with bullet points, arrows, diagrams, half-formed sentences, questions, answers, and anything else my three pens² might allow. I've tried a lot of note-taking systems over the years and a bunch of applications that were designed around these various systems … but nothing sticks. The problem is that the vast majority of the notes I take are completely free-form and cannot be separated from the supporting scribbles.

The bullet-point list cannot be a separate entity from the checklist or the margin-sketch that explains a feature, structure, or idea. These three very different things must coexist in order to have any meaning. Some notes appear as little more than the editorial cartoons in the back of some local newspaper but, contained in that rough doodle, is a larger idea. How can these possibly be put into the same note-taking application when they have fundamentally different structures and intentions?

Systems like Evernote try to accommodate this by scanning images uploaded and extracting the text via OCR so that we can search the documents … but this doesn't work for an editorial-like drawing unless we do a bunch of post-upload work, adding tags and descriptions and other things that are wholly unnecessary if left as a drawn-upon piece of paper in a book. One of the more annoying problems comes when you try to add more to the note later. It must be done digitally and within the constraints of the application.

This just doesn't work for me.

In the past I've tried to use tablets with custom apps and expensive styluses in order to "solve" this problem, but these things never worked as advertised. The handwriting recognition was suboptimal and the tablets were never sensitive enough to capture my handwriting properly. Everything was just messy and out of alignment. While a lot of what I do on paper is free-form, there are groups of structure as well. Why is it that none of the millions of software developers on the planet have been able to solve this given the incredible advances in both hardware and software over the years?

Why haven't I been able to solve this problem?

Some people are able to write notes in a top-down fashion, where everything can fit nicely into a Word document. I envy them. That said, if we approach the problem of looking at notes as groups of structured and unstructured data with geographical positions on a page of infinite length and width, then a number of problems go away. The most difficult one being "How do I translate this object into something digital that can be indexed, edited, and added to?"

I've been playing with the idea of making a note-taking application that works in layers³; every layer being a different type of structured data with it's own set of tags, attributes, and limitations. I have no idea if this will work or not, because layers introduce their own complexity, but I would really like to have a single place where I can put all my notes and have them instantly indexed and searchable.

Time will tell …


  1. If the Maytag man really was as bored as the commercials made him out to be, it's because he worked in Pitcairn or some other remote location where Maytags were few and far between. They break down just as often as any other manufacturer.
  2. Blue, Black, and Red. All three being Mitsubishi UB-150 ballpoints with a really nice 0.5mm tip.
  3. This will use 10Centuries as its central repository, of course. If I can make the darn thing work the way I want, then it may make sense to build a stand-alone app.