The Problem

I don’t know if many people face the same issues as me, but I am easily annoyed with doing computer work manually, especially when it comes to managing and drag-and-dropping files between folders. A personal endeavor that I always wanted to achieve and do for a long time is to write technical or life-related blog posts, journaling and documenting my personal learnings in various topics, and having them immortalized on the internet through my personal website. My main issue was that I liked to make things seamless for me first, then come actually doing it for real, which is both a blessing and a curse. I’d say my thinking really fits the $\ln(x)$ graph where It’s difficult to start (building the flow first) then it’s much simpler when it reaches a point in x-axis (workflow is done, now I just freeform write everything I want and automate whatever I may put into the system). Figure: My personal $ln(x)$ anecdote

One of the the major constraints that I wanted to overcome was that the Obsidian vault is private and has four branches I switch between for different context (journaling and research, project brainstorming, technical skill enrichment, and job searching). The portfolio site is a public Astro repo so these two projects can’t share a directory because it would be too much of pain to include and exclude items in .gitignore.

The Architecture

The structure I decided to build on consisted of triaging/separating (don’t know if this is the correct word to use) the work between three separate folders, a little bit of management here and there, and using sync scripts (push to local) and git’s submodule feature (clones the private PortfolioVault private remote repo into the public-facing Portfolio website as a library/dependency with strict access to specific local folders/files I chose to include and expose). I must say that it works far better than I expected, and I am happy with what I currently have and deployed. Here are the main three layers:

Obsidian vault (private, local)
    ↓  sync-full.sh
PortfolioVault (private GitHub repo)
    ↓  git submodule
Portfolio site (public, Astro + Github Pages w/ CI/CD)

Layer 1 is my local vault at ~/repos/Obsidian. Posts + Gallery items live in blog/published/ and gallery. Moving a file from blog/drafts/ to blog/published/ is the publish action. This ensures that I don’t accidentally publish unfinished or unedited blog posts to my public facing website. It’s a manual process from my end, but I don’t mind because it’s just a drag and drop action on Obsidian and a quick script call.

Layer 2 is a private GitHub repo (vietbui1999ru/PortfolioVault) that only holds content safe to commit: published posts, gallery items, referenced attachments, and web clippings. It’s the staging layer between private notes and public site. I can possibly catch any mistakes I made if I committed files in a hurry from the local vault before they go live.

Layer 3 is the public portfolio. It mounts PortfolioVault as a git submodule at vendor/vault/. Astro reads from vendor/vault/Blogs/ via a glob content loader. No content lives in the source repo. To make this work, my public portfolio repo and my private Vault repo must communicate via a secret SSH key, which is generated one time to establish a secure connection and agreement between the two sources. This allows the portfolio to access and only read (no modification) the private vault’s content, while also not exposing the source and structure of the secret vault.

The Sync Scripts I made:

There are three scripts in ~/repos/Obsidian/scripts/:

sync-to-portfolio.sh : rsyncs blog/published/ to PortfolioVault (PV), rsyncs References/Clippings/ to PV, then calls the attachment script. Has a branch guard: aborts if not on main.

sync-attachments.mjs — scans all markdown in PV for ![[filename]] Obsidian embeds, copies only referenced files from ~/repos/Obsidian/Attachments/ to PV/Attachments/, and prunes any unreferenced files. Size-based idempotency (skip copy if destination has same byte count).

sync-full.sh — orchestrates the full pipeline:

1. Run sync-to-portfolio.sh
2. git add -A && git commit && git pull --rebase && git push in PortfolioVault
3. git submodule update --remote vendor/vault in the portfolio repo, then commit and push the pointer bump

One command to publish: bash ~/repos/Obsidian/scripts/sync-full.sh.

The Remark Pipeline

Astro processes every markdown file through three plugins, in order:

1. remarkPreview — extracts the first paragraph as a preview field for use in post listings.
2. remarkEmbeds — resolves ![[filename]] to <img> tags, copying the asset to public/blog-assets/{slug}/ with sha256-based idempotency. Excalidraw files get rendered to SVG (with script/event-handler stripping).
3. remarkWikilinks — converts [[Post Title]] to anchor tags using a pre-built index. Dead links log a warning but don’t break the build.

The wikilink index is built synchronously at astro.config.ts load time — a one-pass fs.readdirSync over vendor/vault/Blogs/ that extracts frontmatter with a regex (no gray-matter dependency).

What I might do differently in the future when I have more time reviewing

The attachment sync uses file size for idempotency. Two files of identical size but different content would be skipped. Switching to sha256 would be a one-line change. The hash function is already in adapters.ts. I didn’t bother doing this because attachments are images and the collision probability is negligible in practice, but I might have to do that in the near future once I introduce more filetypes into the vault.

The wikilink index (rebuild at every config reload) is synchronous and blocks Astro startup. I don’t have many posts yet (I think the issue would come up when I reach ~ 500 posts) so it’s fine for now. If it grows to that extent (which I highly doubt it would), the index could be written to .cache/wikilinks.json and only rebuilt when vendor/vault/Blogs/ has a newer mtime. I think it’s a nice little optimization and cache solution that can help with keeping the startup time fast and subtle.

Current Thoughts and Learnings

Overall, I’m fairly happy with the publishing and local->private->public blog pipeline, and I don’t intend to change it too much in the near future. This was my second time using git submodules (my first experience with git submodules is documented here), and I got into a deeper workflow in deploying my own custom scripts to securely call my personal private remote repo as a dependency for my portfolio website, which was really cool and showed me the depth of what Github Actions can do. With this automation out of the way, I am free of the chains of managing my folders and files before I even get to the point of writing and publishing stuff, which was a big hurdle for me to cross prior to this optimization.

A few other possible routes I explored and researched before ending up with this solution were:

1. symlinking blog folder from local repo and attaching directly to the public portfolio repo.
   • Pros: simple because I am very familiar with working with symlinks from my experience building my personal dotfiles setup, and it’s just some one liners and a few gitignore additions.
   • Cons: simple at first, gets tedious and convoluted in the future if I add more folders, filetypes. Easy to expose unwanted private documents if I’m not constantly managing and upating my gitignores and symlinks.
2. Using Syncthing as a fast file synchronization solution between my local repo and the portfolio. I personally use it for syncing written notes, tablet notes, and books and pdfs betweenmy computers and tablets around the house.
   • Pros: already have a deep system working from my personal server. Intuitive UI and features that abstracts thinking and just lets you sync what you need to sync without having to have a complex pipeline or mental model.
   • Cons: may introduce (surely will introduce) syncing issues like sync conflicts (same files different versions, have to manually access the devices to track which version is the latest and the one that I want to keep). I’ve experienced the struggle first hand when I wanted to sync and track my personal journaling notes on my macbook and my tablet. For example, when I access the journaling note on my tablet while also editing the same journaling note on my macbook, if I accidentally press something on the tablet and not pay attention, after a delay, both versions will be sent to my server (the main communicator/delegator Syncthing node) and would compete and conflict on who wants to be the newest versions. On my end, I would just see two conflict-generated files of the same note, and I would have to manually delete them. It’s a simple example, but I’ve experienced a sync issue where it duplicated my whole Vault as a sync backup, which was a pain to resolve.

The Problem

I don’t know if many people face the same frustrations I do, but I am easily annoyed by manual computer work — especially when it comes to managing and drag-and-dropping files between folders. A personal goal I’ve wanted to achieve for a long time is to write technical and life-related blog posts: journaling and documenting my personal learnings across various topics, then having them immortalized on the internet through my personal website. My main issue was that I liked to build a seamless workflow before actually doing the work, which is both a blessing and a curse. My thinking fits the $\ln(x)$ graph well: it’s difficult to start (building the flow first), but much simpler once it reaches a point on the x-axis — workflow done, now I just freeform write and let the automation handle the rest. Figure: My personal $ln(x)$ anecdote

One of the major constraints I wanted to overcome was that the Obsidian vault is private and has four branches I switch between for different contexts (journaling and research, project brainstorming, technical skill enrichment, and job searching). The portfolio site is a public Astro repo, so these two projects can’t share a directory — managing .gitignore exclusions across private and public content would become a maintenance nightmare.

The Architecture

The structure I decided on separates concerns across three layers, connected by sync scripts and Git’s submodule feature (which clones the private PortfolioVault repo into the public-facing portfolio as a library/dependency, with strict, controlled access to only the files I choose to expose). It works far better than I expected, and I’m happy with what I’ve deployed.

Here are the three main layers:

Obsidian vault (private, local)
    ↓  sync-full.sh
PortfolioVault (private GitHub repo)
    ↓  git submodule
Portfolio site (public, Astro + Github Pages w/ CI/CD)

Layer 1 is my local vault at ~/repos/Obsidian. Posts and Gallery items live in blog/published/ and gallery. Moving a file from blog/drafts/ to blog/published/ is the publish action. This ensures I don’t accidentally publish unfinished or unedited posts to my public-facing site. It’s a manual step, but I don’t mind — it’s just a drag-and-drop in Obsidian followed by a quick script call.

Layer 2 is a private GitHub repo (vietbui1999ru/PortfolioVault) that holds only content safe to commit: published posts, gallery items, referenced attachments, and web clippings. It serves as a staging layer between private notes and the public site, giving me a chance to catch mistakes before anything goes live.

Layer 3 is the public portfolio. It mounts PortfolioVault as a git submodule at vendor/vault/. Astro reads from vendor/vault/Blogs/ via a glob content loader. No content lives in the source repo itself. To make this work, the public portfolio repo and the private vault repo communicate via a one-time secret SSH key, establishing a secure, read-only connection between the two — the portfolio can read from the vault without modifying it, and without exposing the vault’s internal structure.

The Sync Scripts I made:

There are three scripts in ~/repos/Obsidian/scripts/:

sync-to-portfolio.sh: rsyncs blog/published/ to PortfolioVault (PV), rsyncs References/Clippings/ to PV, then calls the attachment script. Has a branch guard: aborts if not on main.

sync-attachments.mjs — scans all markdown in PV for ![[filename]] Obsidian embeds, copies only referenced files from ~/repos/Obsidian/Attachments/ to PV/Attachments/, and prunes any unreferenced files. Size-based idempotency (skip copy if destination has same byte count).

sync-full.sh — orchestrates the full pipeline:

1. Run sync-to-portfolio.sh
2. git add -A && git commit && git pull --rebase && git push in PortfolioVault
3. git submodule update --remote vendor/vault in the portfolio repo, then commit and push the pointer bump

One command to publish: bash ~/repos/Obsidian/scripts/sync-full.sh.

The Remark Pipeline

Astro processes every markdown file through three plugins, in order:

1. remarkPreview — extracts the first paragraph as a preview field for use in post listings.
2. remarkEmbeds — resolves ![[filename]] to <img> tags, copying the asset to public/blog-assets/{slug}/ with sha256-based idempotency. Excalidraw files get rendered to SVG (with script/event-handler stripping).
3. remarkWikilinks — converts [[Post Title]] to anchor tags using a pre-built index. Dead links log a warning but don’t break the build.

The wikilink index is built synchronously at astro.config.ts load time — a one-pass fs.readdirSync over vendor/vault/Blogs/ that extracts frontmatter with a regex (no gray-matter dependency).

What I might do differently in the future when I have more time reviewing

The attachment sync uses file size for idempotency. Two files of identical size but different content would be silently skipped — switching to sha256 would be a one-line change, and the hash function already exists in adapters.ts. I haven’t bothered because attachments are images and size collisions are negligible in practice, but I’ll likely revisit this as the vault grows.

The wikilink index (rebuilt at every config reload) runs synchronously and blocks Astro startup. With my current post count it’s imperceptible, but around 500 posts it could become noticeable. The natural fix would be to write the index to .cache/wikilinks.json and only rebuild when vendor/vault/Blogs/ has a newer mtime — a clean optimization that keeps startup fast without adding complexity.

Current Thoughts and Learnings

Overall, I’m fairly happy with the local → private → public publishing pipeline, and I don’t intend to change it significantly in the near future. This was my second time using git submodules (my first experience with git submodules is documented here), and going deeper — deploying custom scripts to securely pull from a private remote repo as a portfolio dependency — showed me just how much power Github Actions provides. With this automation in place, I’m free from the overhead of folder and file management before I even start writing, which was a significant hurdle.

A few other routes I explored before landing on this solution:

1. Symlinking the blog folder from the local repo directly into the public portfolio.
   • Pros: simple, and I’m comfortable with symlinks from my personal dotfiles setup.
   • Cons: gets unwieldy as more folders and filetypes are added. Easy to accidentally expose private documents if gitignores and symlinks aren’t kept in sync.
2. Using Syncthing as a file synchronization layer between my local repo and the portfolio. I already use it extensively for syncing notes, tablet annotations, and PDFs between my computers and tablets around the house.
   • Pros: I already have a working Syncthing setup on my personal server. The UI is intuitive and abstracts most of the complexity away.
   • Cons: introduces sync conflicts. I’ve experienced this first-hand when editing the same journaling note on my MacBook and tablet simultaneously — both versions get sent to the server node, which can’t resolve the conflict automatically. The result is two duplicate files requiring manual cleanup. I’ve even had it duplicate my entire vault once, which was a significant pain to sort out.

The Problem, There Is

Face the same frustrations, many people perhaps do not — but annoyed by manual computer work, I easily am. Drag-and-drop file management between folders, especially tedious it is. A personal goal, long I have wanted to achieve: write technical and life-related blog posts, journal my learnings across many topics, and on the internet through my personal website, have them immortalized. Build the workflow first before doing the real work, my tendency is — a blessing and a curse, this is. The $\ln(x)$ graph, my thinking fits: difficult at the start it is (build the flow first, you must), but simpler it becomes once the x-axis point is reached — workflow complete, now freely write and let the automation do its work, you can. Figure: My personal $ln(x)$ anecdote

One of the major constraints to overcome, I had: private, the Obsidian vault is, with four branches I switch between for different contexts (journaling and research, project brainstorming, technical skill enrichment, and job searching). A public Astro repo, the portfolio site is — share a directory, these two projects cannot. A maintenance nightmare, managing .gitignore exclusions across private and public content would become.

The Architecture, Revealed

Across three layers, the structure separates concerns — connected by sync scripts and Git’s submodule feature (which clones the private PortfolioVault repo into the public-facing portfolio as a library/dependency, with strict, controlled access to only the files chosen to expose). Better than expected, it works — happy with what I have deployed, I am. The three main layers, here they are:

Obsidian vault (private, local)
    ↓  sync-full.sh
PortfolioVault (private GitHub repo)
    ↓  git submodule
Portfolio site (public, Astro + Github Pages w/ CI/CD)

Layer 1, my local vault at ~/repos/Obsidian it is. In blog/published/ and gallery, posts and Gallery items live. The publish action, moving a file from blog/drafts/ to blog/published/ is. Accidentally publish unfinished posts to my public-facing site, this ensures I do not. A manual step, yes — but mind it, I do not. A drag-and-drop in Obsidian followed by a quick script call, it is.

Layer 2 is a private GitHub repo (vietbui1999ru/PortfolioVault). Only content safe to commit it holds: published posts, gallery items, referenced attachments, and web clippings. A staging layer between private notes and the public site, it serves as — a chance to catch mistakes before anything goes live, it gives me.

Layer 3, the public portfolio it is. At vendor/vault/, it mounts PortfolioVault as a git submodule. From vendor/vault/Blogs/ via a glob content loader, Astro reads. In the source repo itself, no content lives. To make this work, via a one-time secret SSH key, the public portfolio repo and the private vault repo communicate — a secure, read-only connection between the two, this establishes. Read from the vault without modifying it, the portfolio can, and without exposing the vault’s internal structure.

The Sync Scripts, Built I Have:

Three scripts in ~/repos/Obsidian/scripts/, there are:

sync-to-portfolio.sh: rsyncs blog/published/ to PortfolioVault (PV), rsyncs References/Clippings/ to PV, then calls the attachment script. A branch guard it has: aborts if not on main.

sync-attachments.mjs — scans all markdown in PV for ![[filename]] Obsidian embeds, copies only referenced files from ~/repos/Obsidian/Attachments/ to PV/Attachments/, and prunes any unreferenced files. Size-based idempotency (skip copy if destination has same byte count).

sync-full.sh — the full pipeline, it orchestrates:

1. Run sync-to-portfolio.sh
2. git add -A && git commit && git pull --rebase && git push in PortfolioVault
3. git submodule update --remote vendor/vault in the portfolio repo, then commit and push the pointer bump

One command to publish: bash ~/repos/Obsidian/scripts/sync-full.sh.

The Remark Pipeline

Through three plugins in order, Astro processes every markdown file:

1. remarkPreview — extracts the first paragraph as a preview field for use in post listings.
2. remarkEmbeds — resolves ![[filename]] to <img> tags, copying the asset to public/blog-assets/{slug}/ with sha256-based idempotency. Excalidraw files get rendered to SVG (with script/event-handler stripping).
3. remarkWikilinks — converts [[Post Title]] to anchor tags using a pre-built index. Dead links log a warning but don’t break the build.

At astro.config.ts load time, synchronously, the wikilink index is built — a one-pass fs.readdirSync over vendor/vault/Blogs/ that extracts frontmatter with a regex (no gray-matter dependency).

What Differently I Might Do, When More Time I Have

File size for idempotency, the attachment sync uses. Two files of identical size but different content, silently skipped they would be — switching to sha256, a one-line change it would be, and in adapters.ts, the hash function already exists. Bothered I have not, because attachments are images and size collisions negligible in practice are — but as the vault grows, revisit this I likely will.

Synchronously and blocking Astro startup, the wikilink index runs (rebuilt at every config reload). With my current post count, imperceptible it is — but around 500 posts, noticeable it could become. To .cache/wikilinks.json, write the index one could, and only when vendor/vault/Blogs/ has a newer mtime, rebuild — a clean optimization that keeps startup fast without adding complexity, yes.

Current Thoughts and Learnings

Happy with the local → private → public publishing pipeline, fairly I am — change it significantly in the near future, I do not intend. My second time using git submodules, this was (my first experience with git submodules is documented here), and going deeper — deploying custom scripts to securely pull from a private remote repo as a portfolio dependency — how much power Github Actions provides, this showed me. With this automation in place, free from the overhead of folder and file management before even starting to write, I am — a significant hurdle, that was.

A few other routes explored before landing on this solution, there were:

1. Symlinking the blog folder from the local repo directly into the public portfolio.
   • Pros: simple it is, and comfortable with symlinks I am from my personal dotfiles setup.
   • Cons: as more folders and filetypes are added, unwieldy it gets. Easy to accidentally expose private documents, if gitignores and symlinks not kept in sync are.
2. Using Syncthing as a file synchronization layer between my local repo and the portfolio. Already extensively I use it for syncing notes, tablet annotations, and PDFs between my computers and tablets around the house.
   • Pros: a working Syncthing setup on my personal server, already I have. Intuitive the UI is, and most of the complexity away it abstracts.
   • Cons: sync conflicts, it introduces. First-hand I have experienced this — editing the same journaling note on my MacBook and tablet simultaneously, when I did, both versions to the server node were sent, and resolve the conflict automatically, it could not. Two duplicate files requiring manual cleanup, the result was. My entire vault it once even duplicated — a significant pain to sort out, that was, hmm.