garbash-www

archive of ~alex garbash.com page
git clone git://git.alexkarle.com.com/garbash-www
Log | Files | Refs | README | LICENSE

011-backups.md (2620B) [raw]


      1 ---
      2 title: 011-backups
      3 ---
      4 
      5 # 011-backups
      6 
      7 October 12, 2021
      8 
      9 My usual take on server backups is "don't put anything worth
     10 backing up on the server that's not stored in git elsewhere".
     11 
     12 This has treated me pretty well in the past. Source code,
     13 configuration files, and even documentation on setup are all
     14 stored in git both on the server and on my laptop, and so
     15 I can sleep at night knowing a catastrophic disk failure wouldn't
     16 mean I lost any serious work.
     17 
     18 This strategy breaks down when thinking about a tilde. First,
     19 the array of services we're providing is _much_ more complex
     20 than my normal blog server. Second, there are more people
     21 involved!
     22 
     23 I want to guarantee any tilde members that I will at least try
     24 my best to keep backups of their data in case of failure or
     25 accidental deletion.
     26 
     27 There are tons of backup tools, but a lot of them are fairly
     28 complex (with good reason I suppose.. compression, deduplication,
     29 etc). Since this tilde is about exploring OpenBSD, I took the
     30 opportunity to home-roll a simple backup solution with dump(8)
     31 and restore(8).
     32 
     33 The meat of it is in a script I'm calling "dumpster" that runs
     34 via cron with the day of the week (1-7) as the dump level
     35 and a weekly job dumping the whole system fresh:
     36 
     37     #!/bin/sh
     38     # dumpster -- taking out the garbash with dump(8)
     39     
     40     # %u is 1=mon 7=sun (unless given in ARGV)
     41     LVL=${1:-"$(date +%u)"}
     42     BAKDIR="/bak/$(date +%F)_$LVL"
     43     
     44     mkdir -p "$BAKDIR"
     45     dump -$LVL -auf "$BAKDIR/root.dump.$LVL" /
     46     dump -$LVL -auf "$BAKDIR/home.dump.$LVL" /home
     47     dump -$LVL -auf "$BAKDIR/var.dump.$LVL" /var
     48 
     49 This dumps to /bak, which is a separate Linode Volume, which
     50 has better data redundancy guarantees than the VPS volume and
     51 can be detached/attached to other hosts in the event of VPS
     52 failure.
     53 
     54 As you can see, I'm only really dumping areas that have user
     55 data (/var for git, /home, and / for configs). /usr/\* can be
     56 rebuilt from /var/backups/pkglist for the most part!
     57 
     58 A note to anyone trying this: the Linode Volume was a bit hacky
     59 to get set up, since it expects to be mounting against a Linux
     60 machine. Linode's console will error on attaching, but I found
     61 that rebooting the host made the drive appear as wdN and from
     62 there I was able to format it, etc.
     63 
     64 As a bonus, I took the opportunity to set up /altroot backups,
     65 which is a brilliantly simple way to ensure you can boot into
     66 a known-good state of your host even if something goes very
     67 wrong with the main drive!
     68 
     69 Overall, I went from a backup-avoider to a backup-fan in the
     70 process :) it's so cool to watch the daily script create dump
     71 files of things that changed!