If you want to get better at programming, unfortunately the best way is Spaced Repetition Systems.

I say "unfortunately" because man, can it be a slog.

However, if you want to become a better programmer, there’s a lot more at stake than just your ability to program.

There are coordination issues, your ability to make trade-offs, architectural issues, all that sort of thing.

This is squishy knowledge, and therefore is much better suited to a text medium: a prose-like medium.

This is where the code diary comes in.

At a certain stage in your programming career, perhaps after your first decade, the complexity of what you're asked to do has reached such a point that trying to capture it on the back of a double-sided flashcard is a bit like trying to contain the ocean within a pond.

So I'm in my terminal now, and then I'm going to use the z command to go to the diary folder, and then let's have a look at what's in there by listing the directories.

As you can see there are a bunch of sub-directories like algorithms, apps, architecture, audio, AWS.

Each of these is sort of like a tag or category, and within each of these folders are lots and lots of markdown files.

Let's have a look at some of those markdown files now. And what am I looking for, I'm looking for tree and bat.

And then you can see that within the algorithms folder I have, for example, always-check-upper-and-lower-bounds.md, avoid-dependency-on-starting-time.md etc etc.

Now you can see all my entries, just like that.

Let me demonstrate how I integrate the code diary into my normal coding workflow.

So I'm going to move to one of my projects that I'm working on, and then go to some random file, for example, the product.rb file.

Now, let's imagine that in the process of working on this file, I learn some sort of deep lesson about programming. What I might do then is type the :Diary command which is a custom Vim command that moves me to the TODO.md file within my code diary folder.

Then I go to the bottom of this TODO file and add a new entry. You can see here the most recent entry I added was "when processes suddenly fail with killed - i.e. that’s the output I see on screen- suspect RAM issues".

So that's going to be the name of the lesson, and then underneath, I give the story of what happened.

"I was installed -well, there's a typo there- pytorch on an EC2 instance with only 1GB of RAM. This package when downloaded with pip was over 750mb and the process kept getting killed. When I upgraded the instance to 4GB of ram, the problem went away".

So this here's the stub of a fuller code diary entry, and I will use it to create an overall lesson, perhaps merging it with other observations.

Eventually, I compile these little snippets of lessons into fuller markdown articles, and then I publish them on my website Semicolon & Sons

For example, we're looking at the Code Diary page here. And if I go into, for example, the Architecture category, and then check out Pub Sub System Pros And Cons, we can see an article.

So there's an "Abstract description of pub/sub event systems", then some thoughts about how it might affect coupling along with some code examples, and then some classic use-cases.

Essentially I've gathered all my research and thoughts about this style of coding into one place.

And then at the end, of course, there are some original resources, some prime resources for further reading.

I have about six different types of code diary entries.

The first is Pros and Cons of X.

You just saw an example of that with the pros and cons of pub/sub systems document I just showed you.

The next type is a Screw-up story.

Here's an example of an entry in the category screw-up stories.

The title is phrased in terms of the lesson I learned, or the prospective rule I want to have, and that is "always check the timestamp when reading logs".

Basically, the short version is that I got some error, and I looked at the logs and I saw there was an error there and I jumped into fixing it.

In fact, that error had already been solved and I would have known that if I'd paid attention to the timestamp, and therefore I wasted some time by not following this simple prospective rule.

The next category of entry, number three is, "Everything you need to know about X".

This is basically a brain dump of exploring the API of some sort of technology.

For example, here I have a file - "Docker Basics" And this is just all the tidbits I learned about how to use Docker.

I don't really want to put this into my regular Anki, but I'd still like to have a reference for it.

The next category of code diary entries is, number four, "Join me as I poke around at X and understand how it works".

This usually happens when I encounter something that makes me scratch my head, and then I ask, what's going on at one layer deeper in the stack?

For example, I have this document here, "Understand why colors disappear with pipes", and I created this document to understand the phenomena that I noticed, whereby if you grep for something with color and then you put it into a pipe, sometimes the color disappears.

You can see here that there are certain types of escape characters and so on and that is part of the explanation.

The next category of entry I have here, number five, is "Workflows and best practice checklists I prepare for myself".

At this point I think I have about 75 of these, I'll show you one of them now, unfortunately it's not pretty like the others because I haven't prepared it for public publishing yet, but this is basically a checklist of things to consider when translating web apps (I'm going to make this a bit larger) and for the things that I usually forget to translate from working on international software.

For example, 'attribute names for validations', 'validation errors built into the framework', 'emails sent by auth libraries', 'any custom SMS/emails', 'error messages from our APIs endpoints', 'hard-coded enum values', etc, etc.

By having this checklist, I'm able to reduce the amount of errors related to translation in the finished software.

The sixth category of Code Diary Entry that I mentioned here is 'Annotated code walkthroughs'.

This is essentially when I find some lump of code that I want to understand better, and then I go through it line by line and put comments above each of those lines explaining what's going on.

For example, I've done this here with a Makefile. I'm not very good at using Make but I think it's a tool worth learning, a little bit at least.

So I found an interesting Makefile-based installer online, and then I just figured out how it worked and put lots of notes inline.

So, what are the benefits of using a code diary?

Well, the first one is that not all information anki-fies well, not everything is suitable for SRS.

One example is subjective information.

So if you have a question like, what does Array.reduce do? That is super super suitable for ankification because it has a clear definitive concise answer.

However, the question, "should I design my architecture around those ideas?" is much more qualitative. It has a lot of nuance, it has pros and cons, and so on.

Therefore, it doesn't really make sense to put that in SRS. However, I still want to have a space for me to wrestle with the concepts and to digest the material, to form opinions. That space is the code diary.

Back in college, we had this idea of LAIRs, last-article-I-readers. These were people whose opinions always conformed exactly to whatever the most recent technical article they read espoused.

And I noticed that this tendency appeared in myself, including in my programming work.

And I asked myself why, and, ultimately, I diagnosed this as being due to not having a space where I was forced to reconcile new ideas with old ideas.

What a code diary does is give your opinions, both momentum and depth.

This means that when you get some new information in the form of a persuasively-argued article, then you can refer back to your previous ideas. Ideas you've taken from other articles or your own first-hand experience, and then reconcile what's in that article to what you already know.

Often you’ll end up finding out that you weren't that persuaded by the new article, and this is good because this means you're starting to have opinions of your own, a solid foundation.

Nearly every programming team contains at least one loony who takes as gospel, some best practices they've read on the internet... Best practices that were usually written in a completely different context -a context that has no bearing on the kind of work you're doing- and then enforces these with a battery of linters, and annoying code review comments.

All this in place of sitting down and actually thinking, thinking about the precise shape of this particular project, and its needs.

So, ask yourself and be honest, which camp do you fall into?

The second type of knowledge which doesn't anki-fy well, but it's still worth keeping around somewhere is, short-lived technical knowledge for example, about frameworks.

I feel like the equivalent to putting, for example, React into an anki deck, would be to get a visa for one year to a foreign country to work there, but take out a 40-year rental contract. In all likelihood, you will not be able to stay in that country for all that time or you won't want to, and therefore you'd have wasted a whole lot of money.

Of course, you still need this knowledge in your day-to-day work, and may well do for the next couple of years.

Therefore, a compromise is needed, and what I do is I just store all the things I learned about React or Rails or whatever, in my code diary. That way I can reference my code diary if I need to know how to do something, but at the same time, I'm not drilling it into my long term memory.

Another thing that doesn't anki-fy well is highly ordered information.

For example how an SSL handshake works.

This is because there's this random ordering to the cards you'll see in a Spaced Repetition System, and it becomes very difficult to see the entire big picture.

Therefore I found over time that it's better for me to just keep a document with this highly ordered information, and perhaps review that whole document on a regular basis.

The second reason to keep a Code Diary, is to give you a space to have project retrospectives.

So there's no better time than immediately after a project to have a retrspective for learnings and for your mistakes and to kind of shepherd your thoughts in general.

For example, I've just finished a consulting project that involved about two developer years worth of time building COVID-19 testing center software for airports.

Once the project was done, I spent about a half-day thinking about all the things I would have done differently, and the ideas I came up with here were as much technical as they were human or business-related.

Some quick examples: One, we released to the Apple Store originally.

This was a mistake because the software was so time-sensitive, releasing to the App Store added about six weeks of approval delay.

We ended up rewriting it for web just to be able to get it out sooner.

Two, we didn't anticipate that multiple doctors would log in with the same login details and that caused issues with our security system.

And lastly, we also didn't anticipate that the internet connectivity would be a major issue at some of these testing centers, and we didn't communicate that well in the app, leading the operators of the app to assume that there was a bug with the software rather than try to think about how to get a better connection.

Another big advantage of a code diary is that it's in a prose-like form that can be read, understood and digested by other programmers. This has a lot of value.

For example, if you have Junior programmers on your team. You can send them your particular code diary entries in a topic to get them up to speed more quickly or use them for internal training.

And also if you're trying to build some sort of reputational real estate for yourself, for your consultancy, then having a lot of these code diary entries made public can be useful to bring eyeballs to you and what you do.

I recognize that this is basically the programmer equivalent of posting breakfast on Instagram, but hey, we're human right.

The last reason to keep a code diary is for emotional closure. This is probably the reason closest to my heart, I'd say.

So, basically, whenever you're programming, sometimes you make a really really bad mistake, something that's just shockingly terrible and afterwards you’re probably going to beat yourself up a bit, at least I know I do.

But with a code diary, I can instead take that feeling and then sublimate it into a diary entry, and then I have this feeling that well, at least I've learned my lesson. At least, you know I've taken something from this experience, and then I can let go of the guilt or shame I feel for having screwed something up.

Every time a programmer decides to start taking their technical competence seriously, it makes a difference.

And it brings us that little bit closer to the future where technology gets us out of the mess that we've gotten ourselves into.

So, when you decide to really get good at coding, this is a decision that is bigger than you are, even if you don't see it yet.