One thing I particularly love about org-mode is org-indent-mode, which
visually indents content under each heading based on its level. It makes long
org files much easier to read and navigate.
Occasionally, I need to edit Markdown files — and every time I do, I miss that
clean visual hierarchy. So I vibe-coded a first version over a weekend.
Vibe-Coding
The idea translates naturally from org-indent.el, which ships with Emacs.
Headings marked with # instead of *, same concept.
The first version worked partially, but was full of edge
cases: code fences confusing the heading parser, list items indenting wrong,
wrapped lines losing alignment. I kept using it day-to-day, tweaking it when
something looked off, and simplifying whenever I found a cleaner approach.
Eventually it reached a state I was genuinely happy with.
At that point I thought it might be useful to others too, so I decided to share it.
First Attempt: Merging into markdown-mode
My first instinct was to contribute this directly to markdown-mode, the de-facto
Markdown package for Emacs, so all users could get it without installing
anything extra. It turns out this is a long-anticipated feature — there have
been open issues requesting it for years:
I opened a pull request to add the feature directly into markdown-mode.
Going Standalone Package
The markdown-mode maintainer reviewed the PR and suggested that this would be
better as a standalone package since it would be a burden for them to maintain
the new feature.
I took the suggestion, refactored the code into its own package:
markdown-indent-mode, and submitted it to MELPA.
The MELPA Journey
Submitting to MELPA was a learning experience in itself. I picked up a few tools
along the way for checking Elisp package quality:
checkdoc — checks that docstrings follow Emacs conventions
package-lint — catches common packaging issues
melpazoid — an automated checker specifically designed for MELPA submissions
The MELPA maintainers are volunteers who typically review PRs on weekends, so
the process took a few days.
What markdown-indent-mode Does
Here's what it does — nothing fancy, just the basics done cleanly:
Uses line-prefix and wrap-prefix text properties for visual indentation
Content under a heading is indented according to the heading level
Leading # symbols are hidden
List items are indented to align with their content
Everything is purely visual: no buffer content is ever modified
Installation
This package is available on MELPA. You can use use-package to install it and
add hook to enable it for markdown-mode.
In the Org agenda view, repeated and non-repeated tasks look identical. This similarity makes me hesitant to mark a task as DONE; if a task that should be recurring is missing its repeater (e.g., ++1w), it will disappear permanently from the agenda instead of being rescheduled.
It would be helpful if repeated tasks were visually distinct in the Org agenda, for example, by displaying the repeater interval (++1w) as part of the entry.
Solution: Setting the Prefix Format
Thanks to a tip from yantar92 (the current Org maintainer), I was able to add the repeater to the Org agenda using org-agenda-prefix-format. The process involves three steps:
Shorten the scheduled leaders to make space for the repeater.
Add a function to retrieve the formatted repeater.
Include the new function in org-agenda-prefix-format.
This requires only a few lines of Elisp:
;; Shorten leaders to reserve space for the repeater.
(setq org-agenda-scheduled-leaders '("Sched." "S.%2dx"))
(setq org-agenda-deadline-leaders '("Deadl." "In%2dd" "D.%2dx"))
(defun my/org-agenda-repeater ()
"Return the repeater string for the current agenda entry."
(if (org-before-first-heading-p)
"-------" ; Align with the time grid
(format "%5s: " (or (org-get-repeat) ""))))
;; Add `my/org-agenda-repeater' to the agenda prefix format.
(setcdr (assoc 'agenda org-agenda-prefix-format)
" %i %-12:c%?-12t%s%(my/org-agenda-repeater)")
Alternatives
Using the Agenda Finalize Hook (Hacky)
Before learning about yantar92's tip, I used a hackier solution (gist):
Fetch the repeater using text properties associated with the agenda entry.
Use org-agenda-finalize-hook, which runs after the agenda view is built but before it is displayed, to insert the repeater string.
Using Column View (Cluttered)
Prior to that, I tried another solution using column view (M-x org-agenda-columns). This method involves setting the :COLUMNS: property to display the SCHEDULED date as a separate column.
While this approach works and is built-in, the resulting display is verbose and difficult to parse visually. This led me to seek a better solution with custom Elisp.
Fun fact: a coworker thought I was using a GUI application when they saw my terminal Emacs.
Introduction
At my workplace, most coding tasks must be performed on a remote workstation (or a browser-based IDE) due to policies prohibiting local storage of source code and the limited development tools available on company laptops.
Coding with Emacs on a remote workstation requires access via SSH or a remote desktop connection. I have experimented with various solutions, including Tramp, Chrome Remote Desktop, GTK Broadway, xpra, and wprs. However, these proved buggy, resource-intensive, or slow. Consequently, I settled on the reliable terminal-based approach using SSH and Tmux.
Issues
Initially, I encountered several issues using Emacs in the terminal, as things were not working out-of-the-box:
Emacs was no longer colorful.
Some keybindings did not work.
I could not copy text to the native clipboard.
Version control diff highlighting was not visible.
Opening links in a local browser was not easy.
Over time, I discovered solutions and workarounds for these issues and began to enjoy some extra benefits I had not expected:
Using the same Emacs instance across multiple computers.
Eliminating Crostini on my Chromebook.
Reducing the need to synchronize files and resolve conflicts.
Remote file editing is faster than Tramp.
Providing a more native terminal experience via Tmux multiplexing (compared to vterm, shell, etc.).
No font size scaling issue.
Solutions
Here's how I configured Emacs for comfortable terminal use.
Pick a Good Terminal Emulator
First, pick a terminal emulator that supports at least:
true color, and
OSC 52 (for system clipboard integration).
Ideally, it should have fewer default keybindings or allow customization to avoid conflicts with Emacs.
The default Chrome OS terminal (hterm) is what I currently use, as it meets all three requirements. I do not use Linux, macOS, or Windows frequently enough nowadays to offer recommendations for those platforms.
I use Emacs inside Tmux because it provides a persistent session that can be reattached from any computer or after losing an SSH connection. I also automatically reattach to the Tmux session upon SSH connection with this SSH config:
Host myhost
RemoteCommand tmux -u new -A -D -s main
RequestTTY yes
Alternative ssh command:
ssh -t myhost tmux -u new -A -D -s main
Alternative shell script approach:
# .bashrc
# Attach to the main Tmux session if it is
# - in SSH session
# - not inside Tmux
# - not inside Emacs
if [[ -n "$SSH_CLIENT" && -z "$TMUX" && -z "$INSIDE_EMACS" ]]; then
# "-A" : reattach if session-name already exists
# "-D" : detach other clients (ensure $SSH_TTY is always correct)
tmux new -A -D -s main
fi
Tmux also allows me to have multiple workspaces using Tmux windows (or sessions).
If I need extra terminals, I split new panes or new windows. I find the Tmux terminal better than Emacs's term, shell, vterm, or eat, because it is more native, faster, and easily distinguishable from an Emacs buffer.
Also, I can add information like the system status (CPU, RAM, Disk, etc.) and the current time to the Tmux status bar.
Optionally, change the default Tmux leader to reserve C-b for moving the cursor.
set-option -g prefix M-\
Make Emacs Colorful
The screenshots below show the difference with and without true color (24-bit color) enabled.
Without true color, low contrast
With true color, high contrast
There are a few ways to get true color in a terminal Emacs session:
Use a TERM that supports true color, e.g., xterm-direct. (This won't work inside Tmux, which changes the term to tmux-256color.)
Set the COLORTERM environment variable to truecolor. (preferred)
# .bashrc
export COLORTERM=truecolor
There are a few ways to get true color support in Tmux:
Use a TERM that supports true color, e.g., xterm-direct, or
Start tmux with the RGB feature, e.g., tmux -T RGB, or
Override the features of your terminal in .tmux.conf, e.g.:
# .tmux.conf
# See https://github.com/tmux/tmux/wiki/FAQ#how-do-i-use-rgb-colour.
set -as terminal-overrides ",xterm-256color:RGB"
Tip: Use this script to test for true color support.
Copy Text to Native Clipboard (OSC 52)
OSC 52 works by printing an unreadable sequence, \033]52;c;[base64 data]\a, which instructs the terminal to alter the system clipboard with the base64-encoded data. To verify that OSC 52 is working for your terminal setup, run printf "\033]52;c;$(printf \"Hello, world\" | base64)\a" from the terminal. It should put "Hello, world" into the system clipboard.
In the Emacs config, install the Clipetty package, which sends text that you kill in Emacs to the native clipboard.
To enable clipboard support in Tmux, add these lines to .tmux.conf:
# .tmux.conf
set -g set-clipboard on
# Required to make Clipetty work better on re-attach by appending
# "SSH_TTY" to "update-environment". See
# https://github.com/spudlyo/clipetty?tab=readme-ov-file#dealing-with-a-stale-ssh_tty-environment-variable
set -ag update-environment "SSH_TTY"
If it still does not work, try running the printf verification command above and check the Tmux clipboard wiki.
Open Link in Local Browser
I have two workarounds for opening a web link from a remote Emacs session in a local browser:
Use a mouse click if the terminal emulator can recognize links and open them locally.
Copy the link to the local clipboard and paste it into the browser.
For (2), I added an advice to browse-url to place the URL into the native clipboard via OSC 52. I then paste it into a local browser. I also added key and mouse bindings for the function to make copying more seamless.
When pasting from the native clipboard, I would like to delete the region if one is active.
This is like (delete-selection-mode) for xterm-paste.
(define-advice xterm-paste
(:before (&args) delete-active-region)
"Delete the selected text first before pasting from xterm."
(when (use-region-p) (delete-active-region)))
Update Key Maps that Works in Terminal
The idea is to have a set of keybindings that the terminal can respond to.
This part varies from person to person. For example, I have remapped:
C-x C-; to C-x ; for comment-line (C-; is unsupported).
C-c C-, to C-c , for org-insert-structure-template (C-, is unsupported).
Alternatively, if your terminal supports the Kitty Keyboard Protocol (KKP), you can use kkp.el to make terminal Emacs handle those "impossible" keybindings (thanks to sned@ for the tip).
Show Diff Highlighting with Margin
diff-hl does not work in the terminal by default, and this issue had annoyed me for a while until I searched for a solution. It turns out that diff-hl already has a solution: using the "margin" to show the diff.
I added this Elisp config to turn on diff-hl-margin-mode whenever I am inside a terminal.
Note that there was a conflict where any mouse movement over Emacs would deactivate the Tmux prefix key (this is fixed in Tmux 3.5!). See https://github.com/tmux/tmux/issues/4111.
;; init.el
(xterm-mouse-mode +1)
# .tmux.conf
# Enables mouse support in Tmux (switching windows and panes,
# resizing panes, etc.). Setting mouse on or off does not disable
# Emacs's xterm-mouse-mode.
set -g mouse on
Conclusion
For most Emacsers, GUI Emacs is still a better choice because it:
Has no network latency.
Has better multimedia support (images, PDFs, etc.).
Can handle all keybindings.
Can interact with the clipboard natively.
Can open links in a local browser easily.
Is more customizable (font size per buffer, tooltips, etc.).
However, if you, like me, need to work remotely or want to use the same Emacs instance across multiple computers via SSH and Tmux, I hope the tricks above can improve your setup, even at the expense of the aforementioned GUI features.
10 years ago, when I became a software engineer, my mentor showed me how to replace all occurrences of a string (or a regex) under a large codebase effectively with a tool called codemod. It turned out to be one of the most useful text-editing techniques in my next 10 years, saving me many hours of "find and replace" manually.
Over time, I discovered additional options for different use cases. Here they are.
For Simple Replacement - sed
sed is my favorite for simple cases because it comes pre-installed in most operating systems.
# replace all instances of `foo` (regex) with `bar` (regex) in a file
sed -i.bak "s/foo/bar/g" path/to/file
# replace all instances of `foo` (regex) with `bar` (regex) found by `find`
find . -name '*.cpp' | xargs sed -i.bak "s/foo/bar/g"
# replace all instances of `foo` (regex) with `bar` (regex) in modified files under git
git diff --name-only | xargs sed -i.bak "s/foo/bar/g"
# replace all instances of `foo` (regex) with `bar` (regex) in modified files under hg
hg status --rev .^ -n | xargs sed -i.bak "s/foo/bar/g"
If I encounter a performance issue, I will switch to fastmod.
For Very Large Code Base - fastmod
fastmod is a fast partial replacement for codemod. It is written in Rust and is designed for large-scale codebase refactoring.
# replace a regex pattern in specific files or directories:
fastmod 'foo' 'bar' -- {{path/to/file path/to/directory ...}}
To install, run cargo install fastmod, assume you have Rust's Cargo installed.
As an Emacser, I must also provide a way to achieve this in Emacs.
M-x dired
Mark the required files and directories (m to select, t to toggle all)
Press Q (dired-do-find-regexp-and-replace) to replace string by regex
Press y to accept.
Press ! to accept all.
For Interactive Replacement in Emacs - wgrep
wgrep stands for "Writable Grep Buffer". It is a very flexible interactive replacement in Emacs. It gathers all occurrences in a buffer and then you can use whatever you want edit them.
For Interactive Replacement in Emacs Project - project-query-replace-regex
project.el provides a command called project-query-replace-regex (default keybinding: C-x p r) that allows you to interactively replace a regex pattern within a project.
Epilogue
In most cases, I prefer sed and fastmod since they allow me to do it in a single command, which makes it easier to iterate the desired command (execute and revert for the right "foo" and "bar").
I host some services at home by mapping public domain names to my home IP. It is a typical self-hosting setup that allows me to access those services anywhere, even outside of my home. For example, I use a WebDAV service to sychronize with mobile.
However, I noticed the network throughput is very low (10~20 Mbps) when I am at home, connecting to the same local network. I would expect at least a few hundred Mbps.
Cause: Requests Routed Outside
The requests to my home server are routed outside my home's network and come back because my service's domain is mapped to a public IP.
Public DNS Record
Type
Data
service.example.com
A
Home IP (e.g., 123.123.123.123)
While my ISP gives me 500M+ Mbits/sec of download, it is stint on the upload bandwidth! My local network throughout is throttled by the ISP upload speed.
I benchmark the throughput with iperf3. With a public IP, I get 21 Mbps. It matches the upload bandwidth from the ISP.
These results match my hypothesis! The question becomes – Can I resolve the domain to a private IP when I am home?
Solution: Local DNS Rewrite with AdGuard Home
Yes! One easy solution is to rewrite the DNS record of my domain to a private IP with a local DNS (AdGuard Home).
Local DNS Record
Type
Data
service.example.com
A
Server Private IP (e.g., 192.168.1.2)
It works! I can get a much higher local throughput at home using the public domain name! Also. it works seamlessly when leaving or arriving home because my phone flush its system's DNS cache when switching network.
Solution: Response Policy Zone (RPZ) with Bind9
Another way is to set up response policy zone (RPZ) with bind9/named.
// /etc/named.conf
options {
...
// Add the subnet for recursion
allow-recursion { 127.0.0.1; 192.168.1.0/24; };
// Add external DNS to forward requests
forwarders {
8.8.8.8;
8.8.4.4;
}
// Set up the response policy zone
response-policy {zone "rpz"; };
...
};
zone "rpz" {
type master;
file "rpz.zone";
};
;; /var/named/rpz.zone
$TTL 1h
;; (Serial, Refresh, Retry, Expire, Negative Cache TTL)
@ SOA localhost. root.localhost. (2024120462 8h 30m 1w 1h)
@ NS localhost.
;; List of overriding domains
service.example.com A 192.168.1.2
service2.example.com A 192.168.1.2
# Test if it works. It should return 192.168.1.2
nslookup service.example.com localhost
Alternative: Hairpin NAT
Thanks to this reply, it turns out this problem could be solved by routers with "Hairpin NAT" support (a.k.a. "NAT Loopback"). Unfortunately, mine (Google Wifi) doesn't seem to support Hairpin NAT and I rely on a local DNS rewrite.
2023-05-10: Thanks to @RogerBW for telling me this problem is known as "Split-horizon DNS". As a hindsight, I was solving a known problem from scratch. Here is how:/
Disclaimer: The opinions stated here are my own, not those of my company.
I thought I would always be a GNU Linux user but I switched to Chrome OS early this year.
Why Switch to Chrome OS?
There were multiple reasons why I switched.
My company builds Chrome OS and I was interested in trying our own dog food.
Chrome OS can run Linux (via Crostini) which allows me to use Emacs and keeps my workflow.
Every new year need some changes and I believe the variety is worthwhile.
As a result, I started to use my spare Chromebook more often and it has been 3 months already. I loved it. Light and thin. Fanless and silent. Long battery life. Bluetooth just works.
What Changed After the Switch?
Let's talk about what has changed after switching from Linux to Chrome OS.
Less Bias on Tiling Window Manager
I thought a tiling window manager was superior to a stacking window manager. After using Chrome OS, I think both are fine. I am not less productive with Chrome OS's stacking window manager than my beloved i3wm. It works well for most daily scenarios:
Alt+= to maximize the window.
Alt+[ and Alt+] to tile the window on the left and right respectively.
Alt+N to focus to a specific application from the taskbar, like the i3's scratchpad.
Alt+Tab to switch windows. It works most of the time, given the two main windows I have are Emacs and Chrome.
However, I still complain about the overuse of the Alt key in Chrome OS's system keybindings which can conflict with the underlying application. Hope that it will be improved soon.
Less Time Tweaking Dotfiles
I am spending much less time tweaking my dotfiles like i3wm, rofi, randr, wayland, ibus and so on. I might still configure the Crostini like the sommelier daemon, but on average, much less often. As a result, there is more time for other things, the family, the kids, real work, and even just taking a rest.
More Okay to the Google Eco-System
There are advocates to reduce the dependency on big tech for privacy and autonomy. I love this vision and also self-hosting some services to own my data. I tried to self-host Google Photo and/or Google Drive with an open-sourced alternative, I don't find a good solution as these two services require backing up a large amount of data, and need a special setup for transcoding video.
The idea of self-hosting Drive and Photo faded away after using Chrome OS more often. It makes sense to use Google Drive on Chrome OS because the integration is so good. The more I use Chrome OS, the more I am okay with using the Google ecosystem.
Conclusion
I am very happy with Chromebook and Chrome OS so far. I even switched my work laptop to Chromebook a few weeks ago so now I am using Chromebook for both personal and work. I don't know how long I will stay. Likely until there is a decent Linux laptop with great hardware and driver support.
This post first talks about the PARA method and then my Org-mode implementation for organizing tasks and notes.
The PARA Method
The PARA Method is a modern system for organizing digital information. It stands for Projects-Areas-Resources-Archives, the four categories of all digital information.
Project: a series of tasks linked to a goal, with a deadline.
Area: a sphere of activity with a standard to be maintained over time.
Resource: a topic or theme of ongoing interest.
Archive: inactive items from the other 3 categories.
Project vs Area vs Resource
"Projects" and "Areas" are inherited from GTD. "Distinguish Project and Area" is a common challenge for GTD and PARA adopters, including me. I once had many unimportant projects without deadlines that cluttered my agenda. There are many discussions and articles on it1 and the model answer is to check (1) the nature of the task and (2) if there is a deadline.
A project has (1) a specific outcome/goal and (2) a deadline.
An area has (1) a standard to be maintained that (2) is continuous over time.
In addition to the model answer, I have my interpretation,
Projects contain the tasks I should focus on (because of the importance, urgency and desirability). They have higher priority over the other categories. I work on them first, especially when there is a chunk of uninterrupted time or my energy level is high. If a "project" does not come with a deadline, I set one for it.
Areas contain the tasks I have to do. I reserve some time for these tasks every day to keep areas maintained. I can also skip them for a while when the current projects are too demanding.
Resources contain the tasks I am interested to do. When there is no pressure from projects or areas, I can choose to work on these tasks or simply take a rest.
Project
Area
Resource
Need my focus
Yes
No
No
Have to do
MaybeYes
No
Want to do
MaybeMaybeYes
Priority
High
Medium
Low
These three questions help decide which PARA categories a task should go to:
Does it require my focus within a timeframe? If yes, make it a project.
Is it something I have to do? If yes, move it to an area
Is it something I want to do? If yes, move it to a resource.
PARA is Pragmatic
On one hand, I organize my tasks. On the other hand, I know it is not necessary to get it perfect.
Firstly, it takes a lot of effort. I move forward and do the actual work when the tasks are by-and-large in the right places that help to prioritize. The end goal should be the actual outcomes instead of organizing.
Secondly, it is impossible. PARA is a system of single classification. Everything is either a project, an area, or a resource. In practice, a task or note can fall into multiple projects, areas, or interests. For example, "planning a trip for the 10-th anniversary" could arguably belong to either area "spouse" or area "travel".
Organizing should be forgiving. It is okay to organize wrongly as long as it works. PARA is not a system for perfectionism but for pragmatism.
Let's talk about my implementation!
Org-mode Implementation
I organize my tasks with PARA using a single todo.org file.
Tags and Categories
I use tags for the actual areas and resources. I assign each of them a specific tag to the corresponding subheadings. This gives me a set of controlled vocabularies for tagging projects and notes. For example, I know I should use writing instead of write or blogging for area "writing". I don't create tags for projects (which are short-term and grow over time) to keep my tag set small and useful2.
I use categories for the PARA top categories, Project-Area-Resource. When looking at my org agenda view, I immediately know which tasks are my top focuses (Project), which are things I need to do (Area) or interested to do (Resource). I also know which exact area or resource they belong to with the tags on the right.
Tip: I can filter tasks by category and tag by pressing / (M-x org-agenda-filter) and clear filters with | (M-x org-agenda-filter-remove-all).
Organizing my Notes
For project notes, I add or link them directly under the corresponding org subtree in todo.org as I avoid adding project tags. When a project is completed or canceled, I convert the reusable part into the area or resource notes, and archive the project subtree to todo.org_archive.
I place the areas and resources notes under ~/note. I tag them with the tags defined in todo.org and reserve the directory hierarchy3 for the note types. This gives me the flexibility to have multiple tags for a note.
note/20221125T211904--org-attach__emacs.org is a regular note.
note/meta/20200202T222222--using-org-mode__emacs.org is a meta note that connects other notes, like the org-attach__eamcs.org above.
note/reference/article/20221226T150449--creating-family-rules__parenting.org is a reference note for the article "Creating Family Rules", tagged with my resource "parenting".
note/reference/book/20220907T134123--building-a-second-brain-tiago-forte__pkm.org is a new book note for the book "Building a Second Brain" by Tiago Forte.
So far, I don't archive notes under ~/note. If I need to, I might use the tag ARCHIVED and write some elisp to filter them.
Conclusion
Defining how I am going to organize my digital information reduces the cognitive load when working with tasks and notes. I can add or retrieve the information without thinking too much.
I love to learn how other people works and I hope this post could be interesting and helpful to you, too. If you like it, you probably also check out these:
Tassin's implementation which combines Org-mode and Evernote for tasks and notes respectively.
My last post mentioned the out-of-the-box experience of Denote is not so great, especially when having notes across many subdirectories. By default, Denote searches and creates notes from the root denote-directory. I need to know the exact file location before searching for a note. I need to move note from the root directory to subdirectories after creating the note.
The good news is that, these issues are now fixed and this post talks about how.
#1 Search by Filename in Subdirectories
By default, Denote searches only in the denote-directory. To find a note hidden inside a subdirectory, the user needs to know the path to that file and navigate there. Needing to know the exact directory to be able to search for a file is counter-intuitive.
However, thanks to a recent (2022-11-18) refactoring by nobiot@, it is no longer the case! The commit makes notes inside subdirectories searchable with common Denote commands like denote-open-or-create and denote-link-*. This greatly improves the out-of-the-box experience of Denote. Update Denote to a newer version including this new change!
#2 Full-Text Search in Subdirectories
By default, Denote does not come with any command for full-text search.
However, it could be done easily with consult-ripgrep.
(defun consult-denote-ripgrep ()
"Search with ‘rg’ for files in denote-directory where the content matches a regexp."
(interactive)
(consult-ripgrep denote-directory ""))
#3 Create Notes in Subdirectories
By default, Denote creates notes in the root denote-directory. Every time after creating a note, I need to move them manually to the right subdirectory. This is cumbersome.
However, it turned out to be my fault of not reading the official documentation, which has already suggested two ways to select a specific directory when creating a new note:
includes subdirectory to denote-prompts, or
uses M-x denote-subdirectory.
I go with (1) because it works with all Denote commands that could create a note, like denote-open-or-create and denote-link-or-create. It also allows other customizations to denote-prompts. For example, I prefer setting "subdirectory" before "title" (reorder title and subdirectory) and updating keyword later with denote-keyword-* (remove keyword from denote-prompts).
(setq denote-prompts '(subdirectory title))
If you have too much subdirectories, use denote-excluded-directories-regexp to exclude some of them.
Comparing with Org-roam Again
With these tweaks, I can now use Denote as my primary note-taking package.
I mentioned a concern with the link type denote: in my last post. It turns out to be not concerning because it is easy to migrate to or away from this link type with some elisp. I don't use other features like org-roam-daily, org-roam-bibtex, or org-roam-ui at all. The last step before removing Org-roam is probably migrating the existing links.
Denote is getting popular in the Emacs community recently. I decided to give it a shot, learn how it works and see if I should migrate from Org-roam to it. I am writing this blog post to compares Denote with Org-roam.
In essence, Denote is a note-taking package for Emacs based on its file-naming scheme, which works for all kinds of files. Org-roam is a note-taking package for Org-mode, utilizing a database indexing org files.
Key Differences
Identifier and Linking
Denote uses the timestamp in its file-naming scheme as the identifier and uses its link type denote: for linking. This approach allows notes to be taken in other formats (.md and .txt).
Org-roam uses the ID properties as the identifier and the built-in link type id: for linking. It sticks with the Org-mode ecosystem. It utilizes an external database to indexes note metadata, like titles, tags, and locations and searching notes efficiently without parsing any org files. When working on multiple computers, the user might need to re-index the org files.
This is the biggest architectural difference between Denote and Org-roam. If all notes are written in Org-mode, I find Org-roam provides cleaner org files because it does not introduce a new link type.
Supported Formats
Denote works for all files. Not only can the user take notes in Org-mode, Markdown, or Txt, but also name a PDF file so that it could be identified, linked, searched and tagged.
Org-roam works for only Org-mode files. It leverages the mature Org-mode ecosystem. For example, the user can use org-attach to associate a PDF file with a note.
Frankly speaking, Denote is designed for Emacs users. Org-roam is designed for Org-mode users.
File-naming Scheme
In a file-based note system, it is inevitable to have coupling between the filename and the content (e.g., keeping #+TITLE: and file name in Org-mode).
Denote answers with its file-naming scheme and need to keep filename consistent and update-to-date so that search results remain valid after changing title or tags. Maintaining the naming scheme could become a burden without any automation. Denote provides denote-rename-* and denote-dired-rename-* for the job.
Org-roam does not enforce any file-naming scheme. The default org-roam-capture-templates uses the creation timestamp and title slug as the filename, and that is all.
Note that the Denote file-naming scheme has a limitation: cannot define a tag with multiple words. For example, the tag "note-taking" becomes two tags "note" and "taking" unless combining them to "notetaking".
Searching Mechanism
Utilizing its file-naming scheme, Denote can efficiently search for a note by filename without parsing the content of the file. Users can search notes with titles and tags by prefix - and _ respectively. The out-of-the-box experience does not fit my need. Users need to write some emacs lisp or learn Dired to search efficiently.
Org-roam provides better out-of-the-box searching experience. With a little customization, I can search notes by folder name, title, and tags, recursively across sub-folders, sorted by the last modification time. This suits most of my needs.
It is nice to eradicate the dependency on a database with Denote. However, I found it hard to replicate the Org-roam searching experience with Denote, especially sorting notes by last modification time within subdirectories. Note that full-text search could be achieved by counsel-rg for both.
Denote Comparison to Org-roam
Pros
No database is needed. Seamless across multiple machines.
A simple but powerful file-naming scheme that works for all files.
Allow notes in other formats (txt, md). (Not for me. All my notes are in org.)
Cons
Uses non-standard link type denote:.
Worse out-of-the-box searching experience. (I believe it is hackable)
Tag with multiple words like "note-taking" becomes "note" and "taking"
My Choice
It is tempting to eradicate the dependency on a database with a simple file-naming scheme. However, I am hesitant to introduce denote: to my org files and want to develop a searching experience that on is par with Org-roam first.
I decided to stick with Org-roam but keep exploring Denote in the next couple of weeks. Stay tune :-)
I have been using Syncthing with Caddy's WebDAV to synchronize my Org Mode files across computers and mobiles for few weeks. I am satisfied with it and writing this post about the setup.
Why not just Syncthing?
My previous setup use solely Syncthing to synchronize them across computers and mobiles. While Syncthing works great on computers, it is problematic on mobiles because mobile can switch networks, have a limited battery, and have limited background sync capability (especially on iOS). Out-dated files or conflicts happen more often to me when I use a mobile device.
I noticed some Org Mode mobile apps (like "beorg" and "Orgzly") support WebDAV sync. Hence, I tried to add a WebDAV entry point for my org files and see how it goes.
Overview
I run Syncthing and WebDAV both watching the same org directory on a tiny server, Raspberry Pi 3B. Computers and mobiles are connected with Syncthing and WebDAV respectively.
WebDAV Setup
I first tried Nextcloud but chose Caddy eventually to set up WebDAV.
Attempt #1: Nextcloud
My first attempt was using Nextcloud's WebDAV, after searching "Syncthing WebDAV" and found this story.
I added the Syncthing's local folder to Nextcloud via external storage so it could be accessed from WebDAV. It worked. It proved that Syncthing with WebDAV is a viable solution.
However, Nextcloud is too heavy for my Raspberry Pi 3. The latency is high enough to degrade the mobile experience. This could be a good solution for existing Nextcloud users with a faster hosting machine.
Attempt #2: Caddy WebDAV
Since my Raspberry Pi is already using Caddy as the web server, why not give Caddy's WebDAV a shot? Caddy is written in Go and should have better performance.
The WebDAV module was not built-in and I need to built a new Caddy binary with xcaddy.
To get the file permission right, I previously ran the Caddy web server as a non-caddy user (the same user as syncthing). My new approach is to use bindfs to bind the syncthing directory to a directory readable and writable by the caddy user. This way, I can securely run the Caddy process as the caddy user.
I use beorg on iOS and Orgzly on Android. Both of them support "WebDAV" with "Auto-sync". "Auto-sync" performs a sync when a note is changed or when the app is resumed. This greatly reduces the chance of conflict, when compared with Syncthing.
In addition, I found beorg also comes with a LISP runtime (BiwaScheme) and allows users to customize the app in a way similar to Emacs. The customization is limited at the moment but it has the potential!
Syncthing Setup
I use most of the default configuration with two tweaks.
Tweak #1: Reduce File Watch Delay
I reduce the fsWatcherDelayS from 10s to 1s in all instances. This makes the synchronization happens more instantly and seamlessly between devices. For example, when I make a change on the phone, it updates the Raspberry Pi and synchronizes across other computers in a second.
Tweak #2: Restart Syncthing on Wakeup
When waking up from sleep, Syncthing can take quite a while to recover the connection.
Syncthing used to provide an option called restartOnWakeup to restart Syncthing on wakeup and avoid this long delay. However, this option has been removed in 2022. I replicated this behavior myself, by writing a script to restart Syncthing, and running it on wake up.
A Script to Restart Syncthing
Creates a script to hit the REST endpoint (/rest/system/restart) on the local Syncthing instance.
LOCAL_ENDPOINT=http://localhost:8384
LOCAL_API_KEY=<API_KEY>
echo "Restarting Local Syncthing"
curl --silent -H "X-API-Key: ${LOCAL_API_KEY}" \
--connect-timeout 1 \
-X POST \
${LOCAL_ENDPOINT}/rest/system/restart
Run the Script on Wakeup (MacOS)
For MacOS, uses sleepwatcher to detect wakeup event and execute the above script.
First, install and start sleepwatcher service.
# To install sleepwatcher
brew install sleepwatcher
# To start sleepwatcher now and restart at login
brew services start sleepwatcher
Second, creates another script ~/.wakeup, which will be executed by sleepwatcher on system wakeup.
/path/to/syncthing-restart.sh
Run the Script on Wakeup (Linux)
For Linux, creates a script under /usr/lib/systemd/system-sleep/, which will be executed both "pre" and "post" system sleep.
#!/bin/sh
case $1 in
post)
/path/to/syncthing-restart.sh
;;
esac
Unfortunately, the Linux systemd method does not work for ChromeOS via Crostini. My solution is to use a script to check the system timestamp every second. If the current timestamp and the last timestamp differ more than 3 seconds, then I consider it as a system wakeup.
#!/bin/bash
last=$(date +%s)
while true; do
current=$(date +%s)
if [[ $(($current - $last)) -ge 3 ]]; then
echo "Detected wake up. Restarting Syncthing."
/path/to/syncthing-restart.sh
fi
last=${current}
sleep 1
done
Then, I create a systemd service (~/.config/systemd/user/sleep-watcher.service) to run this script in the background.
[Unit]
Description=Sleep Watcher Service
[Service]
Type=simple
StandardOutput=journal
ExecStart=/path/to/the/above/sleep-watcher.sh
[Install]
WantedBy=default.target
Once it is ready, reload systemd, start and enable the service.
Syncthing works brilliant on computers and laptops with the tweaks mentioned above. WebDAV is simple and cheap to host. It is secure to use over the Internet when pairing with BasicAuth and HTTPS (both supported by Caddy). It is not a perfect synchronization tool but good enough to fill the mobile gap.
Our team works on Android and uses adb logcat a lot. I have heard coworkers complaining about how spammy the logs are and why some logs are missing from the output of adb logcat.
Most of us, including me, start with piping the output to grep for filtering, (i.e., adb logcat | grep <pattern>). This is intuitive for command-line users. This works in practice but there are downsides if it is the only way to do Android logging (i.e., not using tag and priority.)
Trick 0: Set the Minimum Log Priority for Tag
The first downside of not using tags and priorities is that code writers tend to print all logs to at least the default priority, Debug (D). This is because they cannot see the lowest priority Verbose (V) logs from adb logcat. As a result, some logs that should be used in debugging a specific class are printed all the time, spamming everyone.
The second downside is not knowing how to see the Verbose (V) log. For a log to appear in adb logcat, its log priority needs to be at least the minimum log priority set for its tag. For example, the following code will make all logs with the tag tag1 appear in adb logcat:
# Set the minimum log priority to "VERBOSE" for <tag1>
adb shell setprop log.tag.<tag1> VERBOSE
# Set the minimum log priority to "VERBOSE" for <tag1> persistently
adb shell setprop persist.log.tag.<tag1> VERBOSE
Trick 1: Filter with Tags and Priority
If I am interested in only some tags and priorities, then filter logs <tag>[:priority]. In practice, I usually do this.
# Suppress all logs and print all logs for <tag1> and <tag2>.
adb logcat -s <tag1> <tag2>
# Print all fatal logs + error logs for <tag1> + all logs for <tag2>
adb logcat *:F <tag1>:E <tag2>
You can also use highlighted color to match priority by adding -v color.
adb logcat -v color <other arguments>
Trick 2: Filter with Process
If I am interested in the log from a specific process, then print only the logs from that process with --pid.
# Print logs from the <package>:<process>.
adb logcat --pid=$(adb shell pidof -s <package>:<process>)
# Combining trick 1 and trick 2.
# Print the logs for <tag> from the <package>:<process>.
adb logcat --pid=$(adb shell pidof -s <package>:<process>) -s <tag>
Note that if the process crashes and restarts, the pid will change and we need to return the command with the new pid.
Trick 3: Filter with Regex
adb logcat itself can filter logs with regular expression without relying on grep.
adb logcat -e <regex>
This is similar to adb logcat | grep -E <regex> but should be more efficient since the filtered logs do not even get outputted. This is usually not a big deal but can save some bandwidth if the log is sent through the network (working remotely).
Trick 4: Read the Documentation
All my tricks above are well documented in adb logcat -h or the official doc. You might find even that can fit your need by reading it.
I have a habit of hitting the save shortcut (C-x C-s) occasionally. However, if the buffer does not have a visited file (like treemacs, vterm, scratch, help, etc), Emacs asks me where to save the file and I need to cancel with triple ESC.
For example, hitting C-x C-s within the *scratch* buffer.
This is annoying. To fix it, I remap save-buffer to the function below1 which does nothing for non-file-visiting buffer.
(defun my/save-buffer (&optional arg)
"Like `save-buffer', but does nothing if buffer is not visiting a file."
(interactive "p")
(unless (or (buffer-file-name) ; regular buffer
(buffer-file-name (buffer-base-buffer))) ; indirect buffer
(user-error "Use 'M-x save-buffer' explicitly to save this buffer with no visited file"))
(save-buffer arg))
(global-set-key [remap save-buffer] #'my/save-buffer)
Now, I get the message below and do not need to cancel the save operation.
Update: My friend AhLeung has an alternative using save-all-buffers:
(defun my/save-all-buffers ()
"Save all motified file-visiting buffers without asking."
(interactive)
(save-some-buffers t))
(global-set-key [remap save-buffer] #'my/save-all-buffers)
Scratchpad is a useful i3 feature to always have your favorite apps at hand. A typical configuration is to define a key binding to show or hide the scratchpad window. For example,
# Show the Emacs scratchpad window, if any.
bindsym $mod+j [class="(i?)emacs"] scratchpad show
However, i3 does nothing if the targeted scratchpad window does not exist. When it happens, I need to set up the scratchpad window manually by
creating the Emacs window,
moving it to the scratchpad workspace, and
pressing the key binding again
I found it is annoying. This post describes my solution to automate these manual steps.
Solution
Step 1: Create a script called i3_scratchpad_show_or_create.sh.
#!/bin/sh
if [ $# -ne 2 ]; then
echo "Usage: "${0}" <i3_mark> <launch_cmd>"
echo "Example: ${0} 'scratch-emacs' 'emacsclient -c -a emacs'"
exit 1
fi
I3_MARK=${1}
LAUNCH_CMD=${2}
scratchpad_show() {
i3-msg "[con_mark=${I3_MARK}]" scratchpad show
}
# try showing the scratchpad window
if ! scratchpad_show; then
# if there is no such window...
# launch the application.
eval "${LAUNCH_CMD}" &
# Wait for the next window event.
i3-msg -t subscribe '[ "window" ]'
# Set a mark
i3-msg mark ${I3_MARK}
# Move it to the scratchpad workspace
i3-msg move scratchpad
# show the scratchpad window
scratchpad_show
fi
The key ideas are to
automates the manual scratchpad setup mentioned above, and
speeds up by querying with con_mark= instead of title= so that it does not need to wait for the application title, which might not be ready at launch.
Step 2: Update the i3 config
# Show or create the Emacs scratchpad window.
bindsym $mod+j exec /path/to/i3_scratchpad_show_or_create.sh \
'scratch-emacs' 'emacsclient -c -a emacs'
That is! When pressing $mod+j, I can always get the scratchpad window.
Update: After writing this post, I found https://gitlab.com/aquator/i3-scratchpad to be a more sophisticated solution. It brings its own features like setting position, dimension and display for the window but seems to be laggy when showing/hiding window.
For many years, I've used full-size keyboards, but they've presented a couple of recurring ergonomic problems: the mouse feels too far away from the typing area, and I often find mouse movement to the left restricted when it hits the keyboard. These issues have led me to explore smaller keyboard layouts. The legendary HHKB is one I've heard about, but I'm concerned about adapting to life without dedicated arrow keys or relying on navigation like hjkl.
The Research
After some research, I concluded my two requirements:
75% layout
silent switches.
75% is a fairly new keyboard layout. There are not many choices in the market (which might be a good thing).
Finally, I picked Niz Micro 82 (35g) because:
It comes with electrostatic capacitance (EC) switches. I would like to try them.
It provides the option to have no RGB lighting, which is a negative feature to me, even if I can turn it off.
It has good reviews on the Internet.
The 75% Layout
The 75% layout gave more room for the mouse and solved my original headache. I can reach the arrow keys at a shorter distance and get rid of the INS key that I don't use but mishit occasionally.
However, there were issues in the first week:
I could not locate the arrow keys accurately.
I could not hit the right shift key accurately.
I missed the Numpad.
I wondered if I should get a less aggressive keyboard like a TKL. Fortunately, issues 1 and 2 were gone after a month. My muscle memory has adjusted to the new keyboard.
The EC Switches
The 35g EC switches are incredibly silent and comfortable to type. The "silent red" is the closest Cherry MX switch to it, IMO.
I sometimes misfired a key while "placing" my finger on it in the first two weeks. The problem disappeared gradually with time. It has not happened in the last two weeks!
Conclusion
Overall, I much prefer this 75% EC keyboard to my old 100% Filco Ninja Majestouch-2 (Brown Switch).
The primary challenge is the loss of the Numpad's ability for simple one-handed number touch-typing. This isn't easily resolved. Potential workarounds include simply getting used to the new layout, leveraging Fn key combinations for numbers, custom reprogramming the keyboard, adding a standalone Numpad, or adopting a split keyboard setup with a Numpad in between.
The limits of language are the limits of one's world. ~Ludwig Wittgenstein
My limited vocabulary should be a limit of my writing.
Looking up Synonyms with Power Thesaurus
I find looking up synonyms is a way to extend one's active vocabulary. For instance, I can replace beautiful (active vocabulary) with something more specific and accurate, like attractive, lovely, gorgeous, charming, splendid, appealing (passive vocabulary).
I am bad at writing, especially in English, which is not my native language. I misspell words, have limited vocabulary, and misuse grammar without being aware of it.
On the other hand, I write and need to write a lot. Emacs is my main writing tool and I use it to take notes, compose commit messages and blog posts, etc. Over the years, I have learned some tricks to write better with this awesome tool. I am going to share them in the next couple of posts,
Let's start with the basic – spell-checking.
Check and Correct Spelling with Flyspell
I use Flyspell, an Emacs built-in minor mode, for performing on-the-fly spelling checking and correcting.
When enabled, misspelled words will be highlighted. I can use (flyspell-auto-correct-word) to correct the word at the point. Most of the time, the first candidate is the correct one. It is smart enough to me.
I also use the context menu a lot (by middle-clicking the misspelled word). I can choose the right word from all candidates or save the word, so that it does not complain anymore.
If you don't like touching the mouse, flyspell-correct is a package to correct misspelled words with other interfaces, like helm and ivy, all inside the keyboard.
Here is a demo for (flyspell-correct-wrapper).
Configuration
Flyspell provides two minor modes:
flyspell-mode for writing
flyspell-prog-mode for programming (only text inside comments and strings is checked)ema
Here is how I enable them respectively and set up the packages mentioned above.
(add-hook 'text-mode-hook 'flyspell-mode)
(add-hook 'prog-mode-hook 'flyspell-prog-mode)
(use-package flyspell-correct
:after flyspell
:bind (:map flyspell-mode-map ("C-;" . flyspell-correct-wrapper)))
;; Replace with flyspell-correct-helm if you are a helm person.
(use-package flyspell-correct-ivy
:after flyspell-correct)
Caveat
This trick does not work if the misspelled word is valid. For example, when misspelling "summit" as "submit". So, you should still check the spelling even with tool like this.
Pavel Korytov wrote an inspiring blog post to get a consistent set of keybindings between i3 and Emacs. It caught my attention because I use exactly these two tools heavily. Before the integration, I have to define a different keybinding for switching window inside Emacs and i3. After the integration, I can use the same keybinding everywhere. Check out the video in the original blog post to see how cool it is!
I followed the blog post and implemented it immediately. It worked and I loved that! However, the proposed (i3 -> Emacs -> i3) solution introduced a coupling between i3 and Emacs. i3 has to know Emacs, and Emacs has to know i3. It does not fit into the Law of Demeter. It makes the solution harder to scale to other tiling window managers or applications.
Hence, I implemented another (i3 -> Emacs) solution with less coupling (only the focus command for now).
my/wm-integration handles the command from the window manager.
Emacs does not need to know what the window manager is, and do not need to call i3-msg. It return a non-zero code if the command is not handled, or a zero code if handled.
nit. Emacs still use the protocol defined by i3, like the focus command. Well… we have to use a protocol anyway. Let's pick the i3 protocol in this post.
2: Create a Script i3-msg-proxy
#!/bin/sh
#
# Proxy the `i3-msg` command to the focused window.
# Proxy to Emacs if it is the active window
if [[ "$(xdotool getactivewindow getwindowclassname)" == "Emacs" ]]; then
command="(my/wm-integration \"$@\")"
if emacsclient -e "$command"; then
exit
fi
fi
# fallback to i3
i3-msg $@
This script proxies the i3-msg command to the focused window. If the focused window has handled it (return zero), then exit. Otherwise (return non-zero), fallback to i3.
This mechanism (using the return code) can work with other applications, like Tmux.
3: Update the i3 config
bindsym $mod+Left exec i3-msg-proxy focus left
bindsym $mod+Down exec i3-msg-proxy focus down
bindsym $mod+Up exec i3-msg-proxy focus up
bindsym $mod+Right exec i3-msg-proxy focus right
Finally, update the i3 config to proxy the command to the focused window. Now, I can move my focus between i3 and Emacs!
Conclusion
The key difference between this and the original solution is how Emacs calls i3 back when it failed to handle the command. The original solution calls i3-msg inside Emacs. This solution returns non-zero from Emacs.
pass is a simple command-line password manager that follows the Unix philosophy.
It encrypts passwords with gpg and stores them in the file system. It has git integration so one can track the password versioning and synchronize the password store with a git repository. It has clients across different platforms and applications, including Android, iOS, Chrome, Firefox, and also Emacs! I know this tool probably from its Emacs doc.
pass is nice and all, but I don't find a strong reason to use it or switch from BitWarden, which is also open while not requiring me to set up gpg on the phone and browser.
My mother-in-law has left three months before, leaving my wife and me to care for our son at home. I took a leave as a stay-at-home father, focusing on child care and housework like cooking.
With the end of July 4 (Independence Day holiday), I am ending this duty and go back to work. My wife will also be taking time off for the next two months before we can send our son to daycare.
Here are the things I have learned in the last three months.
1. Can Focus on One Thing
Compared to doing work and providing child-care at the same time, my
mental health has improved during the leave. I can cook when I cook,
feed when I feed. I don't think about work or rush to the next meeting
while parenting.
2. Stronger Bonding
Spent more time with my son, we have developed stronger bonding. He
calls me all the time now. We played and drew together. I have taken
lots of valuable photos and videos for the moments that never repeat.
3. Lacking Personal Time
My personal time dropped dramatically. I only have it when my son is
sleeping. I can do some simple stuff (surfing the internet, read a few
pages) but never deep and focused works (writing, coding). I also did
much less aerobic exercise.
4. Parenting is Wearying
The tiredness of parenting is different from the tiredness of working.
At work, I communicate with typical people and can space out if I am
tired. On the other hand, kids always request attention from adults. It
drains the mental power of adults. In addition to sleep deprivation and
other mundane chores, the adults could become extremely tired and
emotional.
I especially like how this mom describes her experience1.
I have heard of "saturated fat", "unsaturated fat", "high-density lipoprotein" (HDL), and "low-density lipoprotein" (LDL) many times, but have never known or cared which are good and which are bad. I thought my exercise habit could burn everything and I could eat anything without much concern.
In a recent annual physical exam, my LDL (bad cholesterol) was 107 mg/dL, higher than the suggested maximum of 100 mg/dL. The doctor said that high LDL is caused by diet.
Even though the level isn't much above the limit, it's still cause for concern. I googled those terms above to learn about them. In short, trans fat, saturated fat, and cholesterol increase LDL.
After that, I acquired the superpower to see through the amount of "cholesterol" and "saturated fat" in foods. Before, I spread a big block of butter on bread. Now, plain bread just works. Before, I drank whole milk like regular milk. Now, I reduce my whole milk intake.
Make 24 is a traditional math game with simple rule – given 4 numbers, you are asked to make 24 with addition +, subtraction -, multiplication * and division /. Each number should be used once and only once.
For example, (3, 7, 2, 1) --> 3 * (7 + 2 - 1) = 24.
Most Make 24 puzzles are easy but a few are extremely hard. This post collects some of the hardest puzzles and discuss them in 3 types. Before jumping into the solutions, let’s take a while to try these 6 puzzles.
These puzzles have only one solution that are not simply multiplication like (3 * 8), (4 * 6) or (2 * 12). The solutions look simple but they are hard to come up with for most people.
These puzzles use division but the dividend is not divisible by the divisor. Some people might think this is violating the rule but, of course, it is not.
These puzzles share a common pattern -– (x, y, z, z) where x * z + y = 24 or x * z - y = 24. The extra z is canceled by placing as the divisor.
Let z = 5 in (1, 5, 5, 5), then we have 5 * 5 - 1 = 24.
Let z = 7 in (3, 3, 7, 7), then we have 3 * 7 + 3 = 24.
That means, since 2 * 9 + 6 = 24, I can add a 9 to create a puzzle in this type.
These puzzles are the hardest of the hardest. Similar to type 2 puzzles, they use division while the dividend is not divisible by the divisor, but they don’t have a simple pattern to spot them out easily.
Fortunately, these puzzles are extremely rare. These two are the only two if all numbers are less than or equal to 10. Simply remember them.
By the way, there is one more solution for (1, 4, 5, 6). I will leave it for the clever readers.
Update on 2021-10-05: https://www.4nums.com/game/difficulties/ has ranked for all combinations by difficulties. You can find other hard puzzles there. For examples, (1, 3, 4, 6), (3, 3, 5, 7), (2, 7, 7, 10).
As mentioned in the last post, I switched to Org-mode. I kept adjusting my workflow with this new tool and it has been stabilized for a month. I think it is time to talk about the new workflow for task/time management with Org-mode. This blog post consists of four parts: the principles, the definitions, the workflows, and finally the implementations.
1 The Principles
Principles remain valid no matter what the tool is.
1.1 Do Not Add Tasks Indiscriminately
Not every task should go into the system. Avoid filling the system with bullshits and diluting the things that matter. I only add tasks that I really want or need to do.
To clarify1, the task management system described below is not the "inbox" in GTD. I still capture things into my inbox but not all of them will be converted to a task in the task management system (org agenda files) eventually.
1.2 Not All Tasks Have To Be Done
There are two reasons for this. First, tasks could be deprioritized or even unnecessary. Second, we have limited time and cannot do everything. We should have an opinion on the priority.
1.3 Reduce The Number Of Open Loops
Open loops are tasks that have been started but not finished. They stay in our minds and occupy some of our limited working memory so that we cannot focus on another task we are working on.
Also, open loops reduce agility, according to Little's Law. The more the open loops, the longer time to finish each of them on average.
1.4 Reduce Decision Making Of What To Do Next
The system should suggest to the user what to do next so that the user can reserve the will power to the real task. This also avoids skipping hard tasks with easy tasks unconsciously.
* PROG Write a blog post on task management with Org-mode
DEADLINE: <2020-11-07 Sat> SCHEDULED: <2020-10-31 Sat>
Each Org-mode user could define their own set of TODO keywords and use scheduled dates and deadlines differently. For example, some people use only two TODO keywords, "TODO" and "DONE", while some use more. Some people set "scheduled dates" to all the tasks while some people set it to some of the tasks. These nuances could result in a hugely different workflow, although they are using the same Org-mode. Let’s take a look at how I use them.
2.1 TODO Keywords
I use as few TODO keywords as possible but not too few. For example, it is common to use only two states ("TODO" and "DONE") but this does not align with the principles I mentioned above. I need a state for "open loops" so that I can keep the number of them small. I also need to distinguish a smaller set of "next actions" from all tasks.
So far, I defined these five keywords:
TODO Keyword
Definition
TODOTasks that are not started and not planned. They could be the backlogs or the GTD’s someday/maybe. These tasks could be converted to NEXT during a review.
NEXTTasks that are not started but planned to do as soon as I can. When there is no actionable PROG (e.g., blocked), I start one of those and convert it to PROG.
PROGTasks that are working in progress (open loops). I work on these tasks before starting another NEXT task to avoid too many open loops at any moment.
INTRTasks that are interruptions. They are urgent things that I should drop everything else and work on it. For example, production issues.
DONEThe tasks that are completed.
This diagram illustrates the transition of those states.
+------+
| INTR |
+------+
|
v
+------+ +------+ +------+ +------+
| TODO |-->| NEXT |-->| PROG |-->| DONE |
+------+ +------+ +------+ +------+
2.2 Scheduled and Deadline
In the past, I tended to set a date for all tasks. If I want to do A, B, and C on Monday, then I schedule them for Monday. This sounds intuitive but, in reality, I ended up rescheduling many incompleted tasks at the end of every day. It was not only wasting time but also depressing.
Later, I changed to rely more on the TODO keywords. For example, if a task is still in progress, I keep the state unchanged as PROG instead of rescheduling it every day until it is done. I am now using the "scheduled date" to hide a task until the date I should look at it again. Similar to the snooze feature in Gmail.
Date
Definition
SCHEDULED
Hide the task until the scheduled date.
DEADLINE
The deadline of the task.
For example, when a PROG task is being blocked, I set the SCHEDULED date to hide it until the date I want to revisit. On the scheduled date, if the task is unblocked, I will remove the SCHEDULED date. If the task is still blocked, I reschedule it again. It acts as the waiting for list in GTD.
3 The Workflow
I customize my org agenda view to drive my daily workflow. The customized agenda view has four sections. From the top to bottom, they are the tasks scheduled today, the INTR tasks, the PROG tasks, and finally the NEXT tasks.
My daily workflow goes from the top to the bottom.
3.1 Update Tasks Scheduled Today
At the beginning of the day, I review the tasks that are scheduled for today. The goal here is not to finish them, but to update or remove the scheduled date so that there is nothing left.
If the task is still blocked, reschedule it
If the task could be done in a few minutes, then do it and mark it as DONE.
Otherwise, remove the scheduled date and optionally update the TODO keywords.
Removing the scheduled date is the best outcome. It indicates the previous estimation was correct, at least not too early. Rescheduling indicates the previous estimation is inaccurate. I would avoid rescheduling the task to tomorrow indiscriminately and try to make a good estimation to reduce the number of rescheduling.
3.2 Find the Next Task to Work On
After reviewing all tasks scheduled for today, it is time to pick a task and do some real works. This step is straight-forward with the customized agenda view above.
Pick an INTR task if there is any.
If there is no INTR task, then pick a PROG task and work on it. If that task is blocked, set a SCHEDULED date to hide it.
If there is no INTR and PROG task, then start a NEXT task.
If there is no task in the agenda view, then review the TODO tasks and convert some to NEXT.
3.3 Review the System
The secret of having a system that works in the long-term is regular maintenance. I do it at least once a week. For examples,
Promote some tasks from TODO to NEXT. Demote or even delete deprioritized tasks.
Review the journal and add TODO if something needs follow-up.
Archive completed tasks and extract to permanent notes2.
4 The Configuration
Finally, here is the configuration for the above workflow.
;; TODO keywords.
(setq org-todo-keywords
'((sequence "TODO(t)" "NEXT(n)" "PROG(p)" "INTR(i)" "DONE(d)")))
;; Show the daily agenda by default.
(setq org-agenda-span 'day)
;; Hide tasks that are scheduled in the future.
(setq org-agenda-todo-ignore-scheduled 'future)
;; Use "second" instead of "day" for time comparison.
;; It hides tasks with a scheduled time like "<2020-11-15 Sun 11:30>"
(setq org-agenda-todo-ignore-time-comparison-use-seconds t)
;; Hide the deadline prewarning prior to scheduled date.
(setq org-agenda-skip-deadline-prewarning-if-scheduled 'pre-scheduled)
;; Customized view for the daily workflow. (Command: "C-c a n")
(setq org-agenda-custom-commands
'(("n" "Agenda / INTR / PROG / NEXT"
((agenda "" nil)
(todo "INTR" nil)
(todo "PROG" nil)
(todo "NEXT" nil))
nil)))
Since 2017, Evernote has been my productivity system for note-taking and task management. I thought it was my ultimate system to stick with. It has served me so well and I have even became a paid user. However, mentioned in the last post, I am pursuing more privacy in my digital life. I want to take my notes with a peace of mind, without worrying they are being watched or leaked.
This post talks about the privacy concern, my research on the alternatives, and the switch to Org-mode.
1. The Privacy Concern of Evernote
The user data in Evernote is encrypted "during transport" and "at-rest". This sounds secure to readers but actually not enough because Evernote can still see the content. Technically, it is still server-side encryption instead the client-side encryption (a.k.a. end-to-end encryption). The encryption key of the note content is held by the server instead of the client. If the server is hacked, or there is an evil employee, or Evernote needs to comply with legal obligations (i.e., the government asks for user data, and it does), those notes could be leaked.
2. Researching the Alternatives
I started researching the alternatives which do not compromise privacy. That means end-to-end encryption is required if the data is hosted on a third-party. This requirement actually simplifies my curation because many solutions are ruled out, including the popular Notion and Roam (o:
Evernote has a reminder feature for simple task management, which is already enough for me. Since it is a note-taking app, I can add rich context like images or even audio seamlessly. Tusk makes it usabe on Linux and the official Android app is simply awesome. Synchronization works out-of-the-box.
However, it fails on the privacy bit.
Review: Org-mode (26)
Org-mode is for keeping notes, maintaining TODO lists, planning projects, and authoring documents with a fast and effective plain-text system. It is absolutely capable.
However, the learning curve of Org-mode is steep and even steeper for non-Emacs users. Users are expected to read documentation and practice. Once familiarized with Emacs, its usability on Linux will jump from 0 to 5 (or even infinity). The Android client Orgzly is usable but could be better. Synchronization works with SyncThing.
Review: Joplin (22) and Standard Note (24)
Joplin and Standard Note are both strong note-taking alternatives to Evernote. However, they suck in task management because they simply cannot show all tasks scheduled for today. They both have a decent desktop app written in Electron. Synchronization works natively and users can turn on end-to-end encryption for privacy/security.
If task management is not a requirement, both Joplin and Standard Note are very good alternatives.
Review: Taskwarrior (21) and TODO.txt (20)
Taskwarrior and TODO.txt are both FOSS for task management. However, they both allow only one line per task, which makes adding extra context to the task very hard. Their clients are barely usable to me (or I have not learned enough). Secure synchronization works with a self-hosted server or with SyncThing for Taskwarrior and TODO.txt respectively.
Review: TiddlyWiki (?)
TiddlyWiki is a "non-linear personal web notebook". It looks fancy but I haven't tried it intensively because Org-mode seems working for me already at this point. I will leave it as a question mark.
3. Switching to Org-mode
After all these explorations, Org-mode looks promising to me and I started the Org-mode/Emacs 30-day challenge – keep using Org-mode/Emacs for a month. Note that I was still using Evernote during the transition but gradually reducing the dependency on it.
I have now finished the challenge and can claim myself an Org-mode user. I am happy with Org-mode and the new workflow. Since the org files are in plain text, I manage my notes with Git, review, and polish all the changes before committing it. It helps to improve the quality of my notes!
Beyond Org-mode, I also started seeing the beauty of Emacs and why people use it instead of Vim or other modern editors. Emacs is ostensibly an editor but actually a powerful Lisp platform/runtime, which makes it super extensible and capable. Users can customize Emacs and make it their own systems for their own purposes.
I am now using Emacs even more. For example, I replace Visual Studio Code, my previous editor/git porcelain, with Magit. This blog post is written in Emacs with ox-hugo. I probably will blog on these topics sometime in the future.
All in all, I am very happy with the privacy gain and the learning from this switch!
2022-12-17: The official archinstall script greatly simplify the Arch Linux installation, even with full disk encryption. Most steps in this guide (except 18 unlock automatically) are automated by that script. Check it out!
I recently re-installed my Arch Linux with full disk encryption (FDE),
as one of the first steps, to bring privacy into my life. This guide
documents the installation process in a step-by-step manner. I hope
this can help people who also want to practice privacy.
Each step in this guide are linked to the corresponding ArchWiki,
precised to section level. You are suggested to read these references
and, of course, the official installation guide because
it is better to understand the process instead of blindly
copy-and-pase, and
I had omitted some uncommon steps that you might need (e.g., non US
locale and keyboard layout)
This guide uses modern options like:
Options
This Setup
Disk Encryption
Yes, No
YesFirmware
BIOS, UEFI
UEFIDisk Partition
MBR, GPT
GPTBoot Loader
GRUB, Syslinux, systemd-boot, etc
systemd-boot
In addition, I have verified this guide twice by installing on both
hard disk and virtual machine respectively. I think it is
reproducible.
The final disk layout from this guide contains two partitions,
Partition
Size
Code
Name
Boot (/dev/sda1)
512.0 MiB
EF00
EFI system partition
Root (/dev/sda2)
Rest of the disk
8300 (default of gdisk)
Linux filesystem
Traditionally, it is suggested to create an extra swap partition. I
don't because there are more flexible alternatives over allocating a
fixed partition for swap. For example, uses swap file, or systemd-swap
to automate the swap file on demand.
When completed, gdisk -l /dev/sda should print the disk partitions
like these:
GPT fdisk (gdisk) version 1.0.5
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 41943040 sectors, 20.0 GiB
Model: VBOX HARDDISK
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 8EFD04A2-473C-4FCA-9C89-459EEB658DB0
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 41943006
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)
Number Start (sector) End (sector) Size Code Name
1 2048 1050623 512.0 MiB EF00 EFI system partition
2 1050624 41943006 19.5 GiB 8300 Linux filesystem
For AMD processors, install the amd-ucode package.
For Intel processors, install the intel-ucode package.
For exmaple, run
# pacman -S intel-ucode
16. Configure the Boot Loader with systemd-boot (ref, systemd-boot)
16.1. Install the EFI boot manager
# bootctl install
16.2. Create /boot/loader/entries/arch.conf
Replace intel-ucode.img with amd-ucode.img if you have an AMD CPU
Replace the UUID (not PARTUUID) to the one mapping to /dev/sda2
(Run blkid to find out)
title Arch Linux
linux /vmlinuz-linux
initrd /intel-ucode.img
initrd /initramfs-linux.img
options cryptdevice=UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX:cryptroot root=/dev/mapper/cryptroot rw
16.3. Replace /boot/loader/loader.conf to
default arch.conf
timeout 5
console-mode max
editor no
Exit the chroot environment by typing exit or pressing Ctrl+d. Then
run reboot.
If everything works, it should ask for a password to access the
cryptroot, like this screenshot below:
One last thing, if you computer, like mine, is sitting behind a DHCP
(e.g., a typical router), you will need to enable dhcpcd to access the
Internet. Run,
# systemctl enable dhcpcd
Congratulations! You have installed Arch Linux with Full Disk
Encryption!
18. (Optional) Unlocking the Root Partition at boot (ref)
The above setup requires manual interruption on each reboot by entering
the password to unlock the disk. There are ways to avoid at the expense
of reducing the security level because the decryption key is exposed.
The one I use is to include the decryption key to the _{initram-fs.img}
so that the systemd module encrypt can use it to decrypt the root
partition automatically.
Steps:
adds a key file to /crypto_keyfile.bin, the default location read
by the encrypt module used above.
adds the /crypto_keyfile.bin to FILES=() array in
/etc/mkinitcpio.conf, so that this file is copied to the image
(e.g., _{initram-fs.img}).
3 years ago, I switched from Mac OS back to Linux. The main reason was
i3wm — a tiling window manager for Linux — and its way to switch
between windows.
In this post, I am going to talk about how to switch windows on i3.
But before that, let's take a look at something we all know —
Alt+Tab.
The Ergonomics Problem with Alt+Tab
Alt+Tab (or Command+Tab on Mac) is the most typical way to switch
windows. It shows the list of opened windows, then the user presses
Tab repeatedly to go to the next window, and release all keys to
select.
These "find"s are distractions. Users need to switch context off from
the real task, find the other window, switch context back to the real
tasks. These "find"s could happen over a hundred times a day and they
become more complex when the number of opened windows goes up.
For example, when coding, I switch between 4 windows, the IDE, the
terminal, the browser, and the note app very often. Switching window
with Alt+Tab could drain my mental energy surreptitiously. These
four are just the minimum. In reality, the number could be double or
triple, as there are multiple browser windows (not to mention the tabs
inside…), and other random applications.
When I first met i3, it blew my mind with its alternatives to switch
windows. Here are three ways I like,
Scratchpad: Switch focus directly, without finding
Move: Move to an adjacent window
Match: Move to a window by string matching
1. Scratchpad – Switch to My Favorites
As the name indicates, this is useful for having a window with your
favorite editor always at hand.
i3 User's Guide
The first alternative is Scratchpad. It summons the targeted window
directly, without finding it from the window list.
In the above screencast, Super+k focuses to the editor. Super+Enter
focuses to the terminal. There is no finding in the whole flow, although
I have more than 10 opened windows.
It works by putting the application to the background and focus to it
when a user-defined hot-key is pressed. "Summoning" requires much less
mental energy than "finding" when the hot-key becomes muscle memory.
The configuration is well-documented in the user guide. Here is the
config for the above screencast:
# Make the currently focused window a scratchpad
bindsym $mod+Shift+minus move scratchpad
# Show the first scratchpad window
bindsym $mod+minus scratchpad show
# Show the terminal with Super+Enter
bindsym $mod+Return [class="Terminator"] scratchpad show
# Show the code editor with Super+k
bindsym $mod+k [class="code-oss"] scratchpad show
These window properties could find by xprop.
2. Move – Switch by What I See
The second alternative is moving to an adjacent window. In my opinion,
it is more visually-intuitive and predictable than the Alt+Tab.
Moving to an adjacent window is the default mechanism of window
switching, for most of the tiling window managers. I am not going to
belabor here.
3. Match – Switch by What I Know
The last one in this post is matching. It matches and narrows down the
list of windows.
In the above screencast, Super+g shows the list of all windows and I
search for gedit by typing gedit.
It is powered by rofi, a window switcher, and application launcher,
similar to Alfred for Mac OS. Here is the one-line configuration.
bindsym $mod+g exec "rofi -show window"
Conclusion
Scratchpad has constant complexity — O(1) — no matter how much
windows are there.
Move is intuitive and predictable way to switch to what I can see.
Match is especially useful to go to a particular window in another
workspace.
Among all three, Scratchpad is the one I love most.
Credit: The GIFs in this post are created with peek and screenkey.
Update 2020-10: I am going back to Hugo and writing blog post with Emacs+Org-Mode+ox-hugo.
I have been blogging with Hugo, a static site generator, for 5 months.
Recently, I decided to ditch it and migrate to WordPress.Org for a
more writer-friendly workflow. I want to spend more time and efforts
on writing instead of coding.
The Developer-Writer Dilemma
Hugo is developer-friendly. It allows me to customize my site, in
terms of the theme, layout, the content structure and also the Hugo
Shortcodes very easily. I learned a lot when building the static
site, the JAM stack concept, SCSS, HTML5 semantic tags, Opengraph,
Twitter Card, et cetera. I like how simple it is to bring up the site,
without setting up the LAMP or LEMP stack.
WordPress is writer-friendly. It allows me to write anytime, anywhere.
I can write posts from my computer or my phone. Its mobile app even
works offline, without an internet connection. This is a gamer changer
to me because I can spend my time writing while holding my little son
Casper to sleep. Editing with the new WordPress Block Editor
(Gutenberg), shipped with WordPress 5.0, is much efficient, when
compared to editing Markdown.
Developer and writer are both the identities I assigned to myself.
This creates a dilemma. When I think about the customization, being
able to build my site, Hugo gets me. When I think about the writing
experience, being able to write whenever and wherever I am, and focus
on the sentences instead of the Markdown syntax, WordPress gets me.
Why did I start this blog? It was actually not because of the
developer identity. I have already been a developer for years and this
identity is entrenched, without a shadow of a doubt. Instead, the
original intention was to become a writer, which is a new identity I
am pursuing.
When I was using Hugo, I spent more time on tweaking the codes instead
of writing things. I researched the best way of rendering an image in
Markdown with Hugo. I organized the contents and reorganized them
again to a "better" structure. I forked the theme I was using because
I don't like how it was implemented.
These are all fun things from a developer's perspective, but not for a
writer. Playing around the Markdown syntax is a distraction to the
writing of the underlying sentences and ideas. Also, managing the
posts in the WordPress admin console is actually more efficient,
compared to the content directory.
The Verdict – WordPress
Redesign your life so the actions that matter most are also the
actions that are easiest to do.
Atomic Habits, James Clear
"Be a writer" was what brought me to the blogging new year revolution.
WordPress allows me to spend more time and efforts on writing. Open
the browser or app, write and publish. Writing in rich text is just
much more intuitive than plain text markdown, even though I am very
proficient at Markdown.
After the migration, I found two things that stand out.
The writing experience with the WordPress Block Editor
(Gutenberg). It is definitely much more powerful and easy than
Markdown to build a web page with a rich layout.
Search without an external search engine. Get the latest search
result within the site. The expense is the ability to customize the
site easily and being able to manage all the configurations with
our beloved version control git.
While the current trend in the industry seems to be moving from the
LAMP-like stack or MEAN-like stack (Node.js) to the JAM-like stack
(static site generator). I am in the opposite direction with most
people. :p
My ideal workflow is to have WordPress-like (anytime, anywhere with a
decent editor) writer experience and Hugo-like developer experience.
Parting with Hugo
This is an amicable parting! I have to say that Hugo is an awesome
static site generator. I enjoyed the experiences with it when building
the first version of this blog and published the first few posts. Here
are the merits:
Hugo is easy to install. I just need to download a single binary.
While Jekyll (another static site generator need to pull a whole
bunch of Ruby dependency.)
Static site is very lightweight. I can deploy the site to the
free f1-micro instance with only 0.2 shared CPU and 0.6 GB memory.
Hugo is very developer-friendly. It has good documentation. The
Hugo Pipe makes it super easy to customize the site with SASS/SCSS.
However, Hugo is not perfect and I struggled with it. Here are some
complaints:
Markdown is limited. e.g., create a link that opens in a new tab
(<a> with target=“blank”) is not straight-forward and not
standardized by the Markdown spec.
Hugo does not have a good plugin system. All features / plugins
are coupled to the theme. If the theme does not provide the
functionality I need, I am going to suffer, write my own, or fork
the theme.
Cannot write anytime, anywhere. When I find a typo on my phone, I
need to correct it right away.
I have a feeling that I will meet Hugo again, in another scenario, in
the future.
The recent shelter-in-place / stay-at-home order has dramatically changed our routines. I don't need to commute to the office so I do not listen to podcasts. I don't have morning coffee since I have no access to the micro-kitchen in the office. I walk less as I stay at home all day. I started doing daily push ups because it is less awkward to do at home, compared to the office. I spend more time with Casper because he is just nearby. Even on my Strava feed, some friends seem to have disappeared while some friends start showing up more days.
Technically speaking, this changed tons of "habit cues", like the activity, place, people, and even our mood. Hence, it will definitely change our habits as well.
You never want a serious crisis to go to waste. And what I mean by that is an opportunity to do things that you think you could not do before.
– Rahm Emanuel
In terms of personal development, it is a waste if we don't deliberately build good habits from this big change. Don't expect good habits to come automatically. They usually don't! Instead, bad habits are more likely to form because their consequences are delayed while the rewards are immediate. Like eating junk food, surfing social media, just to name a few.
If good habits do not come automatically, how to build them effectively?
Keystone habits help other habits to flourish by creating new structures, and establish cultures where change becomes contagious.
In other words, keystone habits are habits that build other habits. Some common keystone habits are exercising, meditation, good sleep, journaling etc. (These examples might sound boring and not as cool as the "keystone habits" :-p )
Good keystone habits are worth investing in and their domino effects will make huge impacts in the long term. Time will magnify the results. Although we know keystone habits are great, the problem is, like all habits, they could break. According to US News, 80% of new year's resolutions fail by the second week of Feburary.
How to make good (keystone) habits stick? Well, let's build the keystone habit of keystone habits.
Habit Tracker (The Keystone Habit of Keystone Habits)
In my humble opinion, habit tracker is the keystone habit of keystone habits because it holds the explicit cues to other habits / keystone habits. It is beautiful because it allows me to keep only one, instead of all immature habits in mind. It makes habit building much easier!
However, TODOs and habits are fundamentally different. For example, habits repeat and repeat very often but TODOs are not. Habits are more okay to break if the reclaiming is fast. The value of habits are measured over time. That said, mixing habits and TODOs in the same list obscures the important TODOs.
Hence, I split them into a standalone habit tracker app. My second (current) habit tracker becomes a standalone app.
That is how I build habits now.
The Two Books
Habit is an interesting topic and it has gotten traction in recent years, thanks to two best seller books.
I learned the habit concepts from these two books. "The Power of Habit" tells more stories behind habits while "Atomic Habit" gives more practical advice. Both are good books but if you want to pick only one, then "Atomic Habit" is the way to go!
Finally, thanks for reading til now to this very last sentence :-)
I kept a daily journal. When people ask me why keeping a journal, my
2019 answer was to track TODO and activity log, then review at the end
of the day. My 2020 answer is to make every day meaningful and counted.
I will share the nuances in another post later.
In this post, I will share how I got started and the 3 components of my
daily journal.
Activity Log
Most Important TODOs
Daily Highlights
Activity Log
The activity log is what brought me to journaling after reading Cal
Newport's Deep Work:
For an individual focused on deep work, it's easy to identify the
relevant lead measure: time spent in a state of deep work dedicated
toward your wildly important goal. … When I shifted to tracking deep
work hours, suddenly these measures became relevant to my day-to-day:
Every hour extra of deep work was immediately reflected in my tally.
How much hours do I spend on deep and shallow work respectively? I don't
really know. So I started tracking my activities and annotating their
deepnesses.
This was my very first daily journal :-)
I found the activity log does not only track the deep and shallow
hours, but also proved the Planning Fallacy to me:
The planning fallacy, …, is a phenomenon in which predictions about
how much time will be needed to complete a future task display an
optimism bias and underestimate the time needed.
I have estimated a task that takes one day but it took me 3 days to
finish. Activity log gave me a sense of how my time is spent and how
much time do I need for a specific kind of task. Later at night, I can
also use this activity log to review what I have done in a day.
Most Important TODOs
I have two TODO lists for each day. The everything list and the most
important list. As the names imply, the everything list holds all TODOs
scheduled for the day and the most important list holds the most
important TODOs of the day.
So, why do I need two lists?
Without the important list, the everything list, which consists of many
important, unimportant, urgent and nonurgent TODOs, could be
overwhelming and confusing. The volume of TODOs usually exceed one's
capacity. As the
Completion
Bias states:
It feels good to chip away at the smaller tasks nibbling at your brain,
but the time you spend doing so is time you're not spending on bigger,
more important projects.
That means I tend to check off many small and easy TODOs but not
getting the important and hard ones done. To fix it, each morning, I
go through the everything list from my Productivity System and curate
TODOs to form the most important list. Then, focus on the most
important list before going back to the everything list.
That does not mean the everything list is useless. These two lists are
complementary. Without the everything list, brainstorming the most
important list out of nothing is an arduous task. Those important
TODOs could be too ad hoc and hence not really important.
The most important list also has a variation called the "1-3-5 Rule".
Daily Highlight
Daily highlight is my 2020 answer to daily journal.
The activity log and the most important TODOs list mentioned above had
worked very well for me, until I was reviewing the journal in the last
year end. I found my journal is boring and full of little things
that are not interesting to me anymore. Those "fix bug XXX", "weekly
YYY meeting" or "implement feature ZZZ" got the things done but I no
longer care about them.
I asked myself, how can I make every day I lived meaningful and
counted, so that some days in the future, I can reminisce?
Then, I added the daily highlight section to the daily journal. The
daily highlights could be anything, without any format but have to be
worth reading again. For examples, - an inspiring excerpt I read, - an
unforgettable mistake I made, - a bloody lesson I learnt, - a nice
chat with someone, - a work I am proud of.
Finally, thanks for reading til now to this very last sentence :-)
It is probably easier to explain what ineffective reading is first.
If you are "seeing" the words without processing, no matter how many
words you "see" per minute, it is ineffective. If you cannot recall or
refer to the key concepts of an inspiring book you have read in the
past, no matter how many books you have read, it is ineffective. If you
cannot implement what you read and make some differences in real life,
no matter how many books you are aiming to read in a year, it is
ineffective.
So, what is effective reading? "Effectiveness" here means maximizing
the understanding, the usage and the retention of knowledge for the
future per unit of time spent reading.
Ineffective reading is always a problem. So far, I have developed three
ways to make reading less ineffective.
Mindful Reading - for understanding
Summarization - for discoverability
Digital Flashcard - for retention
1. Mindful Reading
Mindful reading is the first key to read effectively. It ensures you are
really reading and understanding what the author is saying.
The converse of mindful reading is mindless reading. During mindless
reading, we are seeing the words instead of understanding them. We are
not able to recall what we read one minute ago. When I realize I am
reading mindlessly, I stop, do something else or take some rest.
Mindless reading is ineffective. It is a waste of time and energy.
2. Summarization
Understanding decays because we, humans, forget. It is okay to forget.
However, when we need to use a piece of forgotten knowledge, we want to
be able to reload the knowledge as fast as possible.
Summarization is like building a personal index. It allows you to
navigate through important concepts and to retrieve context and
knowledge swiftly. Without summarization, we will need to spend a huge
amount of time reading the material again to find what we need. This is
definitely ineffective.
The method I use is Tiago Forte's Progressive Summarization. It
highlights and formats the readings into layers. The higher layers are
compressed to make the notes discoverable and the lower layers provide
context.
Layer 0: the original, full-length source text
Layer 1: the content that initially bring into note-taking program.
Layer 2: the first round of summarization, the best parts
Layer 3: the second round of summarization, among all Layer 2, "the
best of the best"
Layer 4: the summary in my own words.
Layer 5: add my own personality and creativity and turn them into
something else.
A real example 1
This is my summary for "Clean Architecture Ch.22". When I read the
note, I first look at the top summary (layer 4), and then the yellow
highlight (layer 3). This should give me most idea of what this
chapter is about. If I need more, then I read the bold (layer 2)
highlights and then the plain text highlights to get more context. If
I still need more, then I could go to the original chapter.
3. Digital Flashcards
If you've got a leaky bucket, you're better off fixing the leak before
pouring water in the top.
I mentioned above that we, humans, forget and it is okay to forget. What
if we need to retain something in the brain? For instance, memorizing
vocabularies could reduce the effort of checking a dictionary (and thus
making reading more effective!). Internalizing key concepts could make
it easier to utilize them.
My method is to formulate the knowledge into digital flashcards and
review them with spaced repetition. (I review the cards every night when
I am holding my sleeping Casper.)
SuperMemo and Anki are among the popular software in this realm. No
matter which software to use, read these
twenty
rules published by SuperMemo about the best practices of formulating
knowledge. "Do not learn if you do not understand" is the wisdom.
A real example 2
After creating the summary above, I think "Dependency Rule" is a very
key concept that is worth memorizing, so I create a corresponding
flashcard. When reviewing the card, if this feels "easy" to me, I will
tap "16d Easy" to snooze the card 16 days later. If I cannot recall
the meaning in 10 seconds, then I tap "<10 min Again".
Conclusion
"The only way to go fast is to go well", Robert C. Martin
I think this applies to many things, including coding, driving and
reading. All of mindful reading, summarization, and making flashcards
slow us down in terms of the number of words and books you read in a
given time. However, they are ways to go fast in terms of
effectiveness.
I am still learning how to make the reading process more effective. If
you have any tips, please let me know. :)
In this era, there are "productive people" who use "productivity
systems" to manage day-to-day chores, TODOs, notes and so on. These
productivity systems are usually note taking apps (Apple Notes,
Evernote, Google Keep, OneNote), task manager apps (Todoist,
Wunderlist, TickTick), paper notebooks or the combination of them.
These "productive people" could be divided into two kinds.
The first kind works for the system. They fill the system with
unactionable tasks that will never get done and unreadable notes that
will never get used again. They do not maintain the system and let it
become a chaos. The value added to the system is more than the value
generated from it. At the end, they might stop using the system,
switch to a "better" app and hope the new one will work out.
The second kind has the system working for them. They might still add
unactionable tasks but will make them actionable. They might add
unreadable notes but will turn them into references that are
searchable and reusable. They maintain the system regularly and are
able to stick with it for a long time. The value provided by the
system is more than the value added to the system.
I was the first kind and now the second kind. So, what made the
difference? I would say it is the methodology. A grand piano helps but
one still need to know how to play it.
Don't get me wrong. I am not saying tools are not important. Both
tools and methodologies are important. It is just easier to acquire
tools than methodologies. One can buy a grand piano in one day but it
takes years of practice to be a great pianist.
For my own system, it is influenced by GTD but with some
modifications. I sticked with the 5 key steps but replaced the Tickler
system with Evernote Reminder.
Capture: Whenever I have something that might need to follow up, I
put them into my inbox. It could be a TODO, a random thought, a link,
a screenshot, a photo, a new vocabulary, a web clip, or, of course,
unactionable tasks.
Clarify: Later, I convert them to actionable tasks, usable
references, someday/maybe, or delete them.
Organize: I organize those items with
Tiago Forte's P.A.R.A. method,
which divides everything into 4 groups, Projects (short-term), Areas
of Responsibility (long-term), Resource (topics interested) and
Archived. This is the most practical organizational method I have
seen so far.
Review: Every Sunday, I review all notes created in the last week
and plan for the next week. I reschedule tasks so that I won't be too
free or too busy next week. Before a long weekend or holiday, I would
pull the someday/maybe items and actualize some.
Engage: When I have free time, I open my task list and do the most
important tasks. This have become the "keystone habit" to build other
habits, like memorizing vocuburaries and meditation. This also
embolden me that I can finish bigger projects, like blogging and
reading books, because the task list always reminds me.
Lastly, adding two screenshots which might tell more than the words.
2019 was a year of changes for me, marked by newly developed habits
and the new responsibility of becoming a father.
Here is what I have done.
I completed 9 English books. This number was a shameful ZERO for the
28 years before 2019. Even when I was a student and tasked to write
book reports, I just paraphrased the introduction and/or some random
pages of the books. Reading has instilled new ideas in me and also
ignited other habits.
I crammed 2000+ new vocabularies into my head. Last year, I learned
about spaced repetition from Learning How to Learn and started
memorizing vocabularies. Whenever I encounter new vocabularies (mostly
from reading), I add them to my Anki collection and review daily. I
use "crammed" instead of "learned" because I still cannot use most of
them. This is something I need to fix in 2020.
I meditated. After reading a few books that recommended meditation,
I started with Headspace. Being mindful, I must say, has the power to
positively impact other areas of life.
I kept a daily journal. When people ask me why I keep a journal, my
2019 answer was to track my to-dos and activity log, then review at
the end of the day. My 2020 answer is to make every day count. I will
share the nuances in another post later. (Update on 2020-03-15: See
Daily Journal)
I logged 1000 miles into Strava. This is actually far less than my
targeted 1500 miles. I have been running for 17 years and have no
intention of stopping, although with Casper, squeezing time for
exercising will be more challenging. On the bright side, this could be
an excuse to be less strict about pacing and mileage :p
We delivered Casper! This is marvelous even with the sleepless
nights and all the other compromises. Taking good care of the little
one and my wife is the overarching priority in 2020.
Life events are interconnected. If I hadn't taken "Learning How to
Learn," I wouldn't have started memorizing vocabularies. If I hadn't
memorized vocabularies, I wouldn't have started reading to collect
more English vocabularies. If I hadn't read books like Ray Dalio's
Principles, I wouldn't have started meditation.
As a new year resolution, I am going to start writing (yes, this
blog), about my thoughts, learning methods, and engineering. Let's see
what chain reactions will happen when I look back at the end of 2020.
This post explains why I was not using Kubernetes for personal
projects. If you are reading this post trying to set up Kubernetes, it
will disappoint you.
Yesterday, I saw a very cool self-hosted project (anki-sync-server)
and I would like to host it myself.
It had happened to me repeatedly that I hosted something (e.g.,
Wordpress) and ended up abandoning it. It was mostly because I forgot
the machine setup or how those apps were configured. They became
burdens, and I killed them to save hosting costs.
However, technology keeps evolving! We now have Docker and Kubernetes
for running portable containers. Configurations can be stored as
Dockerfiles in a git repository. The apps can be restored accurately.
How cool is it to own a personal cluster? How cool is it to be able to
launch random Docker containers in the cluster? It is just super cool,
at least from a software engineer's perspective.
Hence, I decided to set up a Kubernetes cluster on DigitalOcean this
afternoon. My goal was to set up a cluster that allows me to launch
whatever containers as easily as possible. My direction was to set up
Ingress and IngressController for routing traffic to the right pod. If
possible, I would try using NodePort instead of LoadBalancer to avoid
the extra $10 per month.
A few hours passed, I took a pause and went for a run. During the run,
I decided to give up. Here are the reasons.
First, the learning curve of Kubernetes is steep. That means the
re-learning curve is also steep. Imagine that even if the cluster is
set up today but I do not touch it for a few months (because I focus
on other things later or life just does not allow me to spend more
time on it), it is very likely that I will not be able to maintain it
without going through a steep re-learning curve. It will likely become
another burden that I choose to get rid of, again. The time I spent on
setting up the cluster is gone.
Second, it is expensive for small projects. Do you need a load
balancer for an app that has very little QPS? It is simply overkill.
You might argue that it is possible to avoid the load balancer.
However, these solutions (like setting up an extra Nginx with a public
IP as a proxy) are hacky to me. Hacky solutions increase complexity /
maintenance cost, and thus it is more likely to become a burden in the
future.
Everything comes with costs; the biggest is time, simply because it is
limited. When you choose to do something, you are giving up another at
the same time, inevitably. As of now, my priority is no longer playing
with those self-hosted services. Kubernetes is nice and all, but just
not for me at this moment.