commit 8ea58519766d91da48052999e92db28b2b4e2947 (patch)
parent 3b7728a0ab1ec0ed188cedaf1a4b99b0b64fe0f2
Author: alex <alex@garbash.com>
Date: Wed, 13 Oct 2021 23:24:18 -0400
notes: Add notes on backups with dumpster(8)
This was a ton of fun to write!
Diffstat:
2 files changed, 66 insertions(+), 1 deletion(-)
diff --git a/index.html b/index.html
@@ -22,7 +22,6 @@ this site along with my field notes on how I set each thing up!
<h3>TODO:</h3>
<ul>
<li>Wildcard cert for internal services</li>
-<li>Basic backup solution</li>
<li>Start inviting people</li>
<li>Automate account creation</li>
<li>Homepage</li>
@@ -44,6 +43,7 @@ this site along with my field notes on how I set each thing up!
<li><a href="notes/008-local-irc.txt">Set up IRC for tilde members</a></li>
<li><a href="notes/009-wireguard.txt">Set up wireguard</a></li>
<li><a href="notes/010-irc-bouncer.txt">Set up IRC bouncer</a></li>
+<li><a href="notes/011-backups.txt">Basic backup solution</a></li>
</ul>
</body>
</html>
diff --git a/notes/011-backups.txt b/notes/011-backups.txt
@@ -0,0 +1,65 @@
+011-backups -- October 12, 2021
+
+My usual take on server backups is "don't put anything worth
+backing up on the server that's not stored in git elsewhere".
+
+This has treated me pretty well in the past. Source code,
+configuration files, and even documentation on setup are all
+stored in git both on the server and on my laptop, and so
+I can sleep at night knowing a catastrophic disk failure wouldn't
+mean I lost any serious work.
+
+This strategy breaks down when thinking about a tilde. First,
+the array of services we're providing is _much_ more complex
+than my normal blog server. Second, there are more people
+involved!
+
+I want to guarantee any tilde members that I will at least try
+my best to keep backups of their data in case of failure or
+accidental deletion.
+
+There are tons of backup tools, but a lot of them are fairly
+complex (with good reason I suppose.. compression, deduplication,
+etc). Since this tilde is about exploring OpenBSD, I took the
+opportunity to home-roll a simple backup solution with dump(8)
+and restore(8).
+
+The meat of it is in a script I'm calling "dumpster" that runs
+via cron with the day of the week (1-7) as the dump level
+and a weekly job dumping the whole system fresh:
+
+ #!/bin/sh
+ # dumpster -- taking out the garbash with dump(8)
+
+ # %u is 1=mon 7=sun (unless given in ARGV)
+ LVL=${1:-"$(date +%u)"}
+ BAKDIR="/bak/$(date +%F)_$LVL"
+
+ mkdir -p "$BAKDIR"
+ dump -$LVL -auf "$BAKDIR/root.dump.$LVL" /
+ dump -$LVL -auf "$BAKDIR/home.dump.$LVL" /home
+ dump -$LVL -auf "$BAKDIR/var.dump.$LVL" /var
+
+This dumps to /bak, which is a separate Linode Volume, which
+has better data redundancy guarantees than the VPS volume and
+can be detached/attached to other hosts in the event of VPS
+failure.
+
+As you can see, I'm only really dumping areas that have user
+data (/var for git, /home, and / for configs). /usr/* can be
+rebuilt from /var/backups/pkglist for the most part!
+
+A note to anyone trying this: the Linode Volume was a bit hacky
+to get set up, since it expects to be mounting against a Linux
+machine. Linode's console will error on attaching, but I found
+that rebooting the host made the drive appear as wdN and from
+there I was able to format it, etc.
+
+As a bonus, I took the opportunity to set up /altroot backups,
+which is a brilliantly simple way to ensure you can boot into
+a known-good state of your host even if something goes very
+wrong with the main drive!
+
+Overall, I went from a backup-avoider to a backup-fan in the
+process :) it's so cool to watch the daily script create dump
+files of things that changed!
