Public Digital Garden - Publishing Workflow

FYI: This is my custom workflow, that best fits my needs and use case. It is also WIP and constantly iterating. I don’t think it’s very optimised, but it works, and worth sharing with this caveat in mind.

Overview Steps

[[Public Digital Garden - Obsidian Authoring Workflow]] (broken link, note not found)
- Marking notes for public
- handling metadata in obsidian
- handing slugs

Process	Tool of choice
Preprocessing	Custom script
Cloud sync / storage	Backblaze B2 bucket
Website builder	Astro
Assets CDN	Cloudflare CDN

My Requirements

These are my specific requirements for a publishing workflow. You probably have different requirements, so I will explain why I chose the tools I did (and why you may want to choose differently).

I want to publish my Obsidian notes to my website. I want a unique page for each note.

Obsidian notes

My [[Public Digital Garden - Obsidian Authoring Workflow]] (broken link, note not found) uses Obsidian, so it needs to be compatible with Obsidian, as well as with the particular way I use Obsidian (e.g. filetree structure, workflows, backup, etc.).

Obsidian runs locally on my machine (versus a cloud CMS like Ghost or Wordpress which you need internet connection for). So I need a way to upload my new data to the internet.

In the long-term, I want to control as much as possible from Obsidian, without digging into the code. The workflow should expose configuration options naturally via the Obsidian frontmatter!

I want a way to trigger a “Publish” action on-demand. It could run daily on a cron, too.

Image assets

I want images in the website page for the note. This is tricky, because images are external files that need to be handled separately from the markdown notes.

This means that my images need to be hosted as a resource on an assets server, with a resource link and all. However, I still want my images on my local device in my Obsidian vault, especially for offline usage.

Syncing local files to website builder

I need my website builder to access the Obsidian notes. Both in my local development environment on my local device, and on the cloud production environment in some cloud server.

I need a sync and storage solution. I push up files, I pull down files.

High-velocity. The notes change very frequently, about once a day. I don’t need fine-grained version control.

Granular. Only 2-3 notes change a day, so I do prefer to only update the notes that are diffed, without affecting others. However, this may be negligible in terms of egress costs, since markdown files are so tiny—read more later.

Always-accessible. There are simpler solutions like SyncThing, which avoids cost-incurring cloud storage. But because it is peer-to-peer, it requires my laptop to be active when I want to rebuild the website. I want to trigger rebuilds when my laptop is inactive.

Executable. I need something can be triggered and completed by a script. Background sync services run in the background, and the website builder has no way of knowing if the sync down is completed yet or not.

What I don’t need. I don’t need a cloud CMS like Wordpress or Strapi.

One-way writing. I don’t need to edit my notes on the cloud, because my Obsidian notes are my source of truth. Edit/writes are one-way, and my storage only reads, never writes.

Unstructured. I can just use direct file storage. The notes do need to eventually have their metadata parsed for structure, for organisation. But for now, it’s simpler for Astro can handle it locally with its native Content Collections feature for type-safety.

So yeah this is basically S3.

Astro website builder

I am using Astro as my website builder. It’s lean, it’s fun. Latest app-router NextJS static rendering may be my next go-to.

I know frontend and want custom implementations. For the less technical, something like TiddlyWiki would suffice.

It has a native Contents Collections feature that I do like. At the time of implementation, it did not have remote content fetching, so it required content-files to be locally in the workspace. So that works its way in as a requirement. Not sure if this would change in the future.

The Astro workflow needs to be able to consume my data from the cloud sync service.

Triggering a “Publish” action

I use the Obsidian Shell Commands plugin to trigger a custom node script.

This is my “custom shell command” config. I don’t really know everything that’s going on in here. Notably, there’s:

Confirmation modal (confirm_execution)
Custom Obsidian command in the Command Palette (command_palette_availability)

  {
      "id": "p8rdg0g7yw",
      "platform_specific_commands": {
        "default": "cd prep-slugged && pnpm prep-sync && sleep 10 && echo \"Done!\"\n# && curl -X POST \"https://api.cloudflare.com/client/v4/pages/webhooks/deploy_hooks/[uuid]\""
      },
      "shells": {},
      "alias": "Process public garden notes",
      "icon": "lucide-upload-cloud",
      "confirm_execution": true,
      "ignore_error_codes": [],
      "input_contents": {
        "stdin": null
      },
      "output_handlers": {
        "stdout": {
          "handler": "notification",
          "convert_ansi_code": true
        },
        "stderr": {
          "handler": "modal",
          "convert_ansi_code": true
        }
      },
      "output_wrappers": {
        "stdout": null,
        "stderr": null
      },
      "output_channel_order": "stdout-first",
      "output_handling_mode": "buffered",
      "execution_notification_mode": "quick",
      "events": {},
      "debounce": {
        "executeEarly": true,
        "executeLate": false,
        "cooldownDuration": 60,
        "prolongCooldown": false
      },
      "command_palette_availability": "enabled",
      "preactions": [
        {
          "type": "prompt",
          "enabled": false,
          "prompt_id": ""
        }
      ],
      "variable_default_values": {}
    }

This is the script that triggers:

cd prep-slugged && 
\ pnpm prep-sync && 
\ sleep 10 &&
\ curl -X POST "https://api.cloudflare.com/client/v4/pages/webhooks/deploy_hooks/[uuid]"

Make sure to add relevant executables to your PATH in Shell Commands plugin, under Tab: Environments → Add directories to the PATH environment variable. I’m using b2-tools from homebrew, so I have to add /opt/homebrew/bin to PATH. Obviously, tailor to your OS.

/Users/chuangcaleb/Library/pnpm:
/opt/homebrew/bin

Preprocessing

Steps:

Select only notes marked for public, ignore private notes!
Process slugs
Create an output dist folder of processed notes. Don’t touch the original!

Selecting directories

I don’t want to have a dedicated /public notes folder, since my public notes should exist anywhere they fit best. Note the folder structure, explained in greater detail in [[Public Digital Garden - Obsidian Authoring Workflow]] (broken link, note not found).

I want to be a bit efficient and not have to process obviously notes that should not be public. I use soft folder structure, which helps to quickly ignore notes from folders that are used for Templater scripts, Bible references, and my journal entries.

/**
 * Recursively get all Markdown files in a directory, excluding some folders.
 */
function getMarkdownFiles(dir) {
  let results = [];
  const list = fs.readdirSync(dir);

  for (const file of list) {
    const filePath = path.join(dir, file);

    const stat = fs.statSync(filePath);

    if (stat && stat.isDirectory()) {
      if (!EXCLUDED_DIRS.includes(file)) {
        results = results.concat(getMarkdownFiles(filePath));
      }
    } else if (file.endsWith(".md")) {
      results.push(filePath);
    }
  }

  return results;
}

Resolving paths and custom slugs

Note: this section is a lot of unstructured rambling. WIP.

I already have the slug as a user-configured frontmatter metadata in the Obsidian note.

The presence of this slug property also determines if a note should be published or private. I do think this is the best way to do this, so that your slugs will always be consistent.

---
slug: my-custom-note
---
# My custom note!

I have routes like these that I want to resolve. My implementation is different from most I’ve seen online. Most implementations just use a shallow one-level route. I want to sometimes have strict “folders” of notes.

// usually I've seen this
/notes/my-custom-collection
/notes/my-custom-collection-episode-1
/notes/my-custom-collection-episode-2

// I like this
/notes/my-custom-story
/notes/my-custom-story/episode-1
/notes/my-custom-story/episode-2

These are how I want file paths to transform into page routes:

/main/Aang.md
aang

/public/post/Making My Website Guestbook.md
making-my-website-guestbook

/public/post/The Greatest Showman Series/The Greatest Showman Series.md
/the-greatest-showman-series

/public/post/The Greatest Showman Series/Tightrope - Analysis.md
/the-greatest-showman-series/tightrope-analysis

Thankfully, Astro content collections don’t require matching folder structure. You can pass a slug with path segments, and Astro builds segmented routes accordingly.

That simplifies our output. We can just spit out all the notes, into a top-level directory. We just need to modify the slug frontmatter value of each note.

We just need to strip base paths like main or public/post or whatever, and then just copy-paste the note in the output directory.

Resolve frontmatter values - slug paths

But we also need to resolve the links in frontmatter too.

---
up:
  - "[[Story Database]]"
  - "[[Film Reviews]]"
series:
  - "[[The Greatest Show (Prologue) – Analysis]]"
  - "[[A Million Dreams – Analysis]]"
  - ...
---
# The Greatest Showman Series

We need to collect a Map of “filenames” to “processed slugs”.

But we can’t do it in one pass. We need to do two passes. First pass to only collect all the slugs, and then a second pass to transform these to a slug that Astro recognises.

---
up:
  - story-database
  - film-reviews
series:
  - the-greatest-showman-series/the-greatest-show-prologue-analysis
  - the-greatest-showman-series/a-million-dreams-analysis
  - ...
---

Resolve frontmatter values - misc

I need to also resolve some other frontmatter properties too. I use Obsidian Linter plugin to add a created/modified timestamp to all my notes. I also like to use a wikilink, so that the notes appear as backlinks on my journal notes.

created: "[[2024-03-23|23 Mar 2024 (Sat), 21:06]]"
modified: "[[2025-07-12|12 Jul 2025 (Sat), 13:35]]"

I have a script to convert these wikilinks to ISO datetime format, for ingestion by Astro. Note that I ignore the timestamp, for simplicity, that’s too much granularity exposed in my public notes.

created: 2024-03-23T00:00:00.000Z
modified: 2025-07-12T00:00:00.000Z

I also use emojis in my frontmatter. Make sure your parser handles non-standard unicode.

Resolve redirects

I also store redirects configurations in the Obsidian note frontmatter directly. I’ve found it’s great!

redirects:
  - guilt-is-agency-(the-lion-king)-one-small-change

I updated my slugger to remove brackets from the slug. In the future, I may also want to update my slugs, without breaking already-published slugs.

During the first pass, every notes’s redirects field is collected, and then is dumped into a redirects.json file in the output directory.

[!resources]

more:: Creating A Redirect in Astro | RIMdev Blog ↗

more:: Backlinks in Astro / Code Macabre ↗

more:: Add Views Counter to your Astro Blog Posts ↗

more:: Synchronizing and backing up data with Syncthing and rclone ↗

Sync

Why Backblaze?

My v1 implementation uses a git submodule. I was already version controlling my entire vault with the Obsidian Git plugin. Then my v2 used Google Drive Desktop. See my source code’s README ↗ for more detailed breakdowns.

I ended up just using s3. Fits all the criteria.
Costs are actually negligible, for like 3MB storage lol. But it’s very scalable
Since it is S3, we can use it to expose image assets to a CDN!

Backblaze Sync

Syncing up uses the b2 CLI, downloaded through brew. it was baked into shell and I don’t have to manage node dependencies.

brew install b2-tools
b2 authorize-account

This is the shell command used.

b2 sync 
\ --delete --compare-versions size --replace-newer 
\ --exclude-regex '.*\\.DS_Store$'
\ './system/cloud-sync' b2://obsidian-caleb

Learnings:

we are using the flags --delete --compare-versions size --replace-newer
- b2 currently does not provide diffing by hash checksum
- these flags check by file size, this practically does it!
similarly, since redirects.json should barely change, this file should almost never have to incur a sync transaction cost!

Then on the sync down, I tried implementing the same, then realised you can’t call shell commands in the cloud runner. Haih.

import B2 from 'backblaze-b2';

// utility helper fn for downloading a file
const downloadFile = (file, b2) => {
	const response = await b2.downloadFileById({
		fileId: file.fileId,
		responseType: 'text',
	});
	// handle download...
}

// construct b2 client
const b2 = new B2({
	applicationKeyId: B2_KEY_ID,
	applicationKey: B2_APP_KEY,
});
const {data: authData} = await b2.authorize();


// list all files in bucket, under a subdirectory
const fileList = await b2.listFileNames({
	bucketId,
	prefix: B2_BUCKET_PREFIX,
});
const {files} = fileList.data;

// download each file
await Promise.all(
	files.map((file) => downloadFile(file.fileName, authData)),
);

Zipping

I got an email that I had almost maxed out my free tier.

Turns out that an upload/download transaction for each file is considered as “incurring 1 Class B Token”. You only get 2500 Class B tokens a day. I currently have ~60 notes, but this doesn’t scale.

So, I’ve added an extra step to my pre-process, to zip my notes directory.

const archiver = require("archiver");

const NOTES_ZIP_PATH = path.join(DIST_PATH, "notes.zip");
const output = fs.createWriteStream(NOTES_ZIP_PATH);

const archive = archiver("zip", { zlib: { level: 9 } });
archive.pipe(output);

const writeFile = (fileContent, destPath) => {
	archive.append(fileContent, { name: destPath });
}

archive.finalize();

Then, I unzip on download. But actually, this is also technically more network and storage-efficient!

if (path === '/notes.zip') {
	await fs.mkdir(path.dirname(B2_DEST_DIR), {recursive: true});
	// utilty function `streamToBuffer`
	const buffer = await streamToBuffer(response.body);
	await decompress(buffer, B2_DEST_DIR);
	return;
}

Assets CDN Worker

For a long while, I did not handle images. [[Web Image Hosting]] (broken link, note not found) is messy.

Syncing down the zip file of markdown notes is about 250KB. A single image can be up to ten times of that. I can have hundreds of images.

Syncing down markdown notes is great since they’re tiny and gives Astro type-safety. But hmm new remote collections just dropped.

I don’t want to sync down images locally every time the site is rebuilt… that’s gonna explode build times. So image assets should be kept in the cloud bucket.

I’ve used a Cloudflare worker for CDN. Cloudflare my beloved.

A rough and dirty Cloudflare worker script:

forwards resource requests to the /public directory of Backblaze B2 bucket
generates a fresh auth token
- doesn’t hardcode an auth token
- cached for a day (23h) for reuse
handles custom headers
- long caching on the CDN, since we implemented cache busting with hash suffixes
- removes Backblaze headers

export default {
  async fetch(request, env) {
    const url = new URL(request.url)

    const B2_BUCKET_NAME = env.B2_BUCKET_NAME     // e.g. my-static-bucket
    const B2_KEY_ID = env.B2_KEY_ID               // App Key ID
    const B2_APP_KEY = env.B2_APP_KEY             // App Key

    // ✅ Allow only /public/* paths
    if (!url.pathname.startsWith("/public/")) {

      // Gracefully handle /favicon.ico
      if (url.pathname === "/favicon.ico") {
        return new Response(null, { status: 204 });
      }

      // Return 403 to block access to all non-public files
      return new Response("Access denied", { status: 403 })
    }

    // 🛡 Authenticate with B2 (cacheable token)
    const token = await getB2AuthToken(B2_KEY_ID, B2_APP_KEY)
    if (!token) {
      return new Response("Unable to authorize", { status: 503 })
    }

    // 🌐 Build authorized file URL
    const filePath = url.pathname
    const b2DownloadUrl = `${token.downloadUrl}/file/${B2_BUCKET_NAME}${filePath}`

    const b2Response = await fetch(b2DownloadUrl, {
      headers: {
        Authorization: token.authToken
      }
    })

    // 🎯 Don't reveal if file is missing or not
    if (!b2Response.ok) {
      return new Response("Access denied", { status: 403 })
    }

    // 🧊 Fully buffer body to enable edge caching
    const headers = new Headers(b2Response.headers)

    // ✅ Explicitly set long-term cache policy
    headers.set("Cache-Control", "public, max-age=31536000, immutable")
    headers.delete("Set-Cookie")

    // 🔐 Optional: remove sensitive or irrelevant headers
    headers.delete("x-bz-file-id")
    headers.delete("x-bz-upload-timestamp")
    headers.delete("x-bz-content-sha1")
    headers.delete("x-bz-info-src_last_modified_millis")

    return new Response(b2Response.body, {
      status: b2Response.status,
      statusText: b2Response.statusText,
      headers
    })
  }
}

// 🔄 Utility: Auth with B2 and reuse token for 24 hours
let cachedToken = null
let tokenExpires = 0

async function getB2AuthToken(keyId, appKey) {
  const now = Date.now()

  if (cachedToken && now < tokenExpires) {
    return cachedToken;
  }

  const authHeader = "Basic " + btoa(`${keyId}:${appKey}`)

  // Construct full auth URL using env variable
  const authUrl = `https://api.backblazeb2.com/b2api/v2/b2_authorize_account`

  const authRes = await fetch(authUrl, {
    headers: { Authorization: authHeader }
  })

  if (!authRes.ok) {
    return null
  }

  const data = await authRes.json()
  cachedToken = {
    authToken: data.authorizationToken,
    downloadUrl: data.downloadUrl
  }
  tokenExpires = now + 1000 * 60 * 60 * 23 // 23 hours

  return cachedToken
}

Since I also manage my domain configuration with Cloudflare, I’ve configured a subdomain as the route for the worker.

cdn, images, and asset are good contenders for subdomain names, but assets is the most used and appropriate if I decide to host non-image assets.

I also configure Cache Rules.

This is my configuration:

Custom filter expression
- (http.host eq "assets.chuangcaleb.com")
Cache eligibility: Eligible for cache
Edge TTL: Use cache-control header if present, bypass cache if not
Browser TTL: Respect origin TTL

Oh, this also messed me up for like days worth of debugging. Ugh. Enable caching on the Backblaze bucket side too. This is because Cloudflare respects origin cache headers, and will not set infinite cache even if you force it in the Cloudflare worker.

{"Cache-Control": "public, max-age=86400"}

Future Enhancements

Rebuild website 2 days after an auto or manual deployment
excalidraw flowchart lol