▶ YouTube Summaries

← All summaries
protesilaos#▶ Watch

Developing denote-data - rename hook, stale entries and a lean cache

2026-10-13 Tue ⏱ 2 hr 11 min protesilaos

Live stream continuing work on denote-data, an opt-in, dependency-free, in-memory cache for Denote that speeds up lookups at large scale (link previews, future completion at point). Prot slims the cache, adds an internal abnormal hook so renames update the hash table without saving, and drops the idea of checking file existence on every cache read. Stale entries are instead removed lazily, when a command such as denote-link or denote-find-link tries to use a file that no longer exists. He ends doubting that after-save-hook is the right update trigger. Chat Q&A covers his keyboard layout, themes, keyboard macros, Magit and Jujutsu.

What denote-data is for

  • At huge scale Denote can be slow: e.g. previewing image links in an Org file means scanning all files per link. A cache does the work once and then only tracks deltas.
  • Opt-in; core Denote behaves as before. It will become its own package; it lives in denote.el for now for easier testing.
  • Not a database: pure Emacs Lisp, so no external dependency, but other programs can't read it. The abstractions are designed so someone could write a database-backed alternative.

Changes made

  • Use locate-library instead of locate-file (returns the .elc when present).
  • TODO: the async cache build loads default Denote settings, so it must pick up user options like denote-directory.
  • Removed the text slot from the entry struct. Storing every file's buffer string would put gigabytes in memory. Renamed forelinks to links (list of identifier strings). Struct changes need an Emacs restart to take effect.
  • New internal abnormal hook for denote-rename-file, called with old identifier, identifier, title, keywords, signature and new path. Useful beyond denote-data.
  • denote-data handles renames: remhash the old identifier key and write a fresh entry under the new one. Setting the struct's identifier in place was a bug, because the hash key stayed old. Verified that the cache updates while the buffer is still unsaved.
  • Removed the macro that ran file-exists-p on every maphash: checking the whole table on each read defeats the purpose of a cache. Replaced with plain maphash lambdas (written via a keyboard macro).
  • get-path-by-id no longer touches the hash table.
  • New abnormal hook called when denote-link targets a missing file. denote-data uses it to remove the entry (denote-data-remove-file) before the user error. denote-find-link and denote-find-backlink call it too. Tested: deleted files disappear from the completion candidates after one failed attempt.
  • denote-rewrite-front-matter now calls find-file-noselect once and reuses the buffer.
  • Ideas noted as TODOs: use file modification time to refresh only changed files; perhaps buffer-modified-p or diff-based checks (rejected for now to avoid depending on diff.el). No advice on delete-file / Dired deletion: removing entries lazily when used is cheaper and safer.
  • Final TODO: after-save-hook is probably the wrong trigger (e.g. save-some-buffers); use Denote's own command hooks instead and keep it self-contained.

Side notes from chat

  • Keyboard: HJKL layer for arrows, YUIO row for Home / PgDn / PgUp / End. Thumbs: Alt (stretch), Control (rest), Shift (curl); pinky Control-Meta and Super.
  • Themes: using modus-vivendi-deuteranopia. Tinted Modus variants are his pick for safe defaults. Open to a dark Ef Summer–like theme PR; has an unfinished Doric dark draft in that spirit.
  • fill-paragraph breaking long words annoys him. fill-region-as-paragraph-default is a 156-line function that should take a predicate option.
  • One setq per variable is easier to move, sort and undo, and costs the same as one setq with many pairs.
  • Keyboard macros beat templates for dynamic, one-off edits.
  • If only one third-party package: Magit, with Vertico and Consult close behind. Git history surgery is hard to teach without a mental model. Aware of Jujutsu but not an early adopter.
  • Display scaling on a 14" 1080p laptop: try a lower 16:9 resolution, since font size scaling isn't uniform across apps.