From b7371061b91452435327096f32764ae71921fde3 Mon Sep 17 00:00:00 2001 From: alex Date: Mon, 25 Oct 2021 00:16:56 -0400 Subject: [PATCH] notes: Port all notes to markdown Well, the retro vibes of a good ole text file were fun, but this is a website gosh dangit, and html just looks better on mobile! --- .gitignore | 1 + Makefile | 35 +++++++++++++++++++++++++++++++---- index.md | 22 +++++++++++----------- notes/001-domain-name.md | 23 +++++++++++++++++++++++ notes/001-domain-name.txt | 17 ----------------- notes/002-install.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ notes/002-install.txt | 43 ------------------------------------------- notes/003-httpd.md | 34 ++++++++++++++++++++++++++++++++++ notes/003-httpd.txt | 28 ---------------------------- notes/004-mail-server.md | 26 ++++++++++++++++++++++++++ notes/004-mail-server.txt | 20 -------------------- notes/005-ssh-hardening.md | 25 +++++++++++++++++++++++++ notes/005-ssh-hardening.txt | 19 ------------------- notes/006-use-the-src.md | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ notes/006-use-the-src.txt | 45 --------------------------------------------- notes/007-git-coding.md | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ notes/007-git-coding.txt | 69 --------------------------------------------------------------------- notes/008-local-irc.md | 37 +++++++++++++++++++++++++++++++++++++ notes/008-local-irc.txt | 31 ------------------------------- notes/009-wireguard.md | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ notes/009-wireguard.txt | 64 ---------------------------------------------------------------- notes/010-irc-bouncer.md | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ notes/010-irc-bouncer.txt | 86 -------------------------------------------------------------------------------------- notes/011-backups.md | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ notes/011-backups.txt | 65 ----------------------------------------------------------------- 25 files changed, 596 insertions(+), 502 deletions(-) create mode 100644 notes/001-domain-name.md delete mode 100644 notes/001-domain-name.txt create mode 100644 notes/002-install.md delete mode 100644 notes/002-install.txt create mode 100644 notes/003-httpd.md delete mode 100644 notes/003-httpd.txt create mode 100644 notes/004-mail-server.md delete mode 100644 notes/004-mail-server.txt create mode 100644 notes/005-ssh-hardening.md delete mode 100644 notes/005-ssh-hardening.txt create mode 100644 notes/006-use-the-src.md delete mode 100644 notes/006-use-the-src.txt create mode 100644 notes/007-git-coding.md delete mode 100644 notes/007-git-coding.txt create mode 100644 notes/008-local-irc.md delete mode 100644 notes/008-local-irc.txt create mode 100644 notes/009-wireguard.md delete mode 100644 notes/009-wireguard.txt create mode 100644 notes/010-irc-bouncer.md delete mode 100644 notes/010-irc-bouncer.txt create mode 100644 notes/011-backups.md delete mode 100644 notes/011-backups.txt diff --git a/.gitignore b/.gitignore index 2d19fc7..43b328c 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ *.html +notes/*.txt diff --git a/Makefile b/Makefile index dd52d26..ad517f1 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,27 @@ -BUILT = index.html root-index.html +BUILT = index.html \ + root-index.html \ + notes/001-domain-name.html \ + notes/001-domain-name.txt \ + notes/002-install.html \ + notes/002-install.txt \ + notes/003-httpd.html \ + notes/003-httpd.txt \ + notes/004-mail-server.html \ + notes/004-mail-server.txt \ + notes/005-ssh-hardening.html \ + notes/005-ssh-hardening.txt \ + notes/006-use-the-src.html \ + notes/006-use-the-src.txt \ + notes/007-git-coding.html \ + notes/007-git-coding.txt \ + notes/008-local-irc.html \ + notes/008-local-irc.txt \ + notes/009-wireguard.html \ + notes/009-wireguard.txt \ + notes/010-irc-bouncer.html \ + notes/010-irc-bouncer.txt \ + notes/011-backups.html \ + notes/011-backups.txt build: $(BUILT) @@ -7,12 +30,16 @@ clean: install: build mkdir -p /var/www/htdocs/~alex/notes - install -m 444 $(BUILT) /var/www/htdocs/~alex - install -m 444 notes/*.txt /var/www/htdocs/~alex/notes + install -m 444 *.html /var/www/htdocs/~alex + install -m 444 notes/* /var/www/htdocs/~alex/notes -.SUFFIXES: .md .html +.SUFFIXES: .md .html .txt .md.html: ./bin/buildpage $< > $@ +# Legacy txt symlinks since it's linked externally! +.md.txt: + cd `dirname $<` && ln -s `basename $<` `basename $@` + .PHONY: all clean install diff --git a/index.md b/index.md index 47e0b80..4673507 100644 --- a/index.md +++ b/index.md @@ -28,14 +28,14 @@ Game Plan ### Done: -* [Awesome domain name :)](notes/001-domain-name.txt) -* [OpenBSD install on Linode](notes/002-install.txt) -* [HTTP(S) server](notes/003-httpd.txt) -* [Email (SPF, DKIM, etc)](notes/004-mail-server.txt) -* [SSH hardening](notes/005-ssh-hardening.txt) -* [Obtained the source code for the system](notes/006-use-the-src.txt) -* [Set up git hosting via stagit(1)](notes/007-git-coding.txt) -* [Set up IRC for tilde members](notes/008-local-irc.txt) -* [Set up wireguard](notes/009-wireguard.txt) -* [Set up IRC bouncer](notes/010-irc-bouncer.txt) -* [Basic backup solution](notes/011-backups.txt) +* [Awesome domain name :)](notes/001-domain-name.html) +* [OpenBSD install on Linode](notes/002-install.html) +* [HTTP(S) server](notes/003-httpd.html) +* [Email (SPF, DKIM, etc)](notes/004-mail-server.html) +* [SSH hardening](notes/005-ssh-hardening.html) +* [Obtained the source code for the system](notes/006-use-the-src.html) +* [Set up git hosting via stagit(1)](notes/007-git-coding.html) +* [Set up IRC for tilde members](notes/008-local-irc.html) +* [Set up wireguard](notes/009-wireguard.html) +* [Set up IRC bouncer](notes/010-irc-bouncer.html) +* [Basic backup solution](notes/011-backups.html) diff --git a/notes/001-domain-name.md b/notes/001-domain-name.md new file mode 100644 index 0000000..7ed6236 --- /dev/null +++ b/notes/001-domain-name.md @@ -0,0 +1,23 @@ +--- +title: 001-domain-name +--- + +# 001-domain-name + +Fri Sep 17, 2021 + +garbash the word was all ~anthony's idea. + +It came out of a PR review as a self-deprecating take on his bash +(which actually turned out to be mine). + +We laughed and I realized the domain was for sale, so I snagged it +and resolved to find its purpose later. + +I've always wanted to be part of a tilde community, but struggle +opening up to strangers on the internet. Starting a tilde with a +friend, however, felt much more promising. All the ascii and none +of the awkwardness! + +So here we are a week later, standing up this site. Hope you enjoy +your stay! diff --git a/notes/001-domain-name.txt b/notes/001-domain-name.txt deleted file mode 100644 index 0437a61..0000000 --- a/notes/001-domain-name.txt +++ /dev/null @@ -1,17 +0,0 @@ -001-domain-name -- Fri Sep 17, 2021 - -garbash the word was all ~anthony's idea. - -It came out of a PR review as a self-deprecating take on his bash -(which actually turned out to be mine). - -We laughed and I realized the domain was for sale, so I snagged it -and resolved to find its purpose later. - -I've always wanted to be part of a tilde community, but struggle -opening up to strangers on the internet. Starting a tilde with a -friend, however, felt much more promising. All the ascii and none -of the awkwardness! - -So here we are a week later, standing up this site. Hope you enjoy -your stay! diff --git a/notes/002-install.md b/notes/002-install.md new file mode 100644 index 0000000..eed0a07 --- /dev/null +++ b/notes/002-install.md @@ -0,0 +1,49 @@ +--- +title: 002-install +--- + +# 002-install + +Tues Sept 21, 2021 + +I'm a huge fan of OpenBSD. The simplicity of the system, the cohesive +feel it has, the proactive stance on security... when we decided we'd +set up a tilde, I knew I wanted it to be on OpenBSD. + +The only problem? My preferred registrar (Linode) doesn't support it! + +Fortunately there's a comprehensive post on the Linode forum of how to +do it [1]. + +It took us ~45m, the longest OpenBSD install I've had since I first +flashed it on an old thinkpad. 40m of that was waiting for the node to +boot and reboot, etc (we kept messing up the configuration). + +1. Create a New Linode (any OS will do) +2. Once booted, shut it down +3. Under the "Storage" tab delete the ext4 partitions +4. Create two new disks, both "Raw" format: + - One labeled "install", 1GB (could do less) + - One labeled "os", the rest of the space +5. Boot in "Rescue" mode +6. In the serial console, wget the minirootXX.img + - Check the sha256 against the SHA256 file + - Check the signature using signify (on a different machine that + has signify) +7. Find the install disk with `lsblk` +8. Flash the img using: `dd if=minirootXX.img of=/dev/sdX bs=1M` +9. In the Configurations tab, create a new one: + - Full Virtualization + - Select a Kernel > Direct Disk + - /dev/sda - os + - /dev/sdb - install + - boot from sdb +10. Reboot into configuration, install OpenBSD from serial console +11. Halt/shutdown, and change configuration to boot from sda +12. Rejoice! + +In our case, our main problem was that we skipped the "Direct Disk" +kernel step so we were booting a Linux kernel and trying to load the +img... it panic'd every time! Took us a few boots to figure that out :) + +[1]: https://www.linode.com/community/questions/10329/openbsd-on-linode diff --git a/notes/002-install.txt b/notes/002-install.txt deleted file mode 100644 index 835e8c7..0000000 --- a/notes/002-install.txt +++ /dev/null @@ -1,43 +0,0 @@ -002-install -- Tues Sept 21, 2021 - -I'm a huge fan of OpenBSD. The simplicity of the system, the cohesive -feel it has, the proactive stance on security... when we decided we'd -set up a tilde, I knew I wanted it to be on OpenBSD. - -The only problem? My preferred registrar (Linode) doesn't support it! - -Fortunately there's a comprehensive post on the Linode forum of how to -do it [1]. - -It took us ~45m, the longest OpenBSD install I've had since I first -flashed it on an old thinkpad. 40m of that was waiting for the node to -boot and reboot, etc (we kept messing up the configuration). - -1. Create a New Linode (any OS will do) -2. Once booted, shut it down -3. Under the "Storage" tab delete the ext4 partitions -4. Create two new disks, both "Raw" format: - - One labeled "install", 1GB (could do less) - - One labeled "os", the rest of the space -5. Boot in "Rescue" mode -6. In the serial console, wget the minirootXX.img - - Check the sha256 against the SHA256 file - - Check the signature using signify (on a different machine that - has signify) -7. Find the install disk with `lsblk` -8. Flash the img using: `dd if=minirootXX.img of=/dev/sdX bs=1M` -9. In the Configurations tab, create a new one: - - Full Virtualization - - Select a Kernel > Direct Disk - - /dev/sda - os - - /dev/sdb - install - - boot from sdb -10. Reboot into configuration, install OpenBSD from serial console -11. Halt/shutdown, and change configuration to boot from sda -12. Rejoice! - -In our case, our main problem was that we skipped the "Direct Disk" -kernel step so we were booting a Linux kernel and trying to load the -img... it panic'd every time! Took us a few boots to figure that out :) - -[1]: https://www.linode.com/community/questions/10329/openbsd-on-linode diff --git a/notes/003-httpd.md b/notes/003-httpd.md new file mode 100644 index 0000000..a328fd8 --- /dev/null +++ b/notes/003-httpd.md @@ -0,0 +1,34 @@ +--- +title: 003-httpd +--- + +# 003-httpd + +Tues Sept 21, 2021 + +One of the first things I do when I set up a machine is set up +httpd(8) and grab a HTTPS cert via acme-client(8). + +Here's a quick rundown (though reading the man pages is worth +the time!). + + # sed 's/example.com/garbash.com/g' \ + /etc/examples/httpd.conf > /etc/httpd.conf + # sed 's/example.com/garbash.com/g' \ + /etc/examples/acme-client.conf > /etc/acme-client.conf + +Then go in and edit the files to add aliases if needed! + +To get the certs for the first time: + + # rcctl enable httpd + # rcctl start httpd + # acme-client -v garbash.com # get certs + # rcctl reload httpd # load certs + +Finally, to keep the certs up to date, add the following to the +crontab: + + # crontab -e + ... + ~ * * * * acme-client garbash.com && rcctl reload httpd diff --git a/notes/003-httpd.txt b/notes/003-httpd.txt deleted file mode 100644 index 88e8543..0000000 --- a/notes/003-httpd.txt +++ /dev/null @@ -1,28 +0,0 @@ -003-httpd -- Tues Sept 21, 2021 - -One of the first things I do when I set up a machine is set up -httpd(8) and grab a HTTPS cert via acme-client(8). - -Here's a quick rundown (though reading the man pages is worth -the time!). - - # sed 's/example.com/garbash.com/g' \ - /etc/examples/httpd.conf > /etc/httpd.conf - # sed 's/example.com/garbash.com/g' \ - /etc/examples/acme-client.conf > /etc/acme-client.conf - -Then go in and edit the files to add aliases if needed! - -To get the certs for the first time: - - # rcctl enable httpd - # rcctl start httpd - # acme-client -v garbash.com # get certs - # rcctl reload httpd # load certs - -Finally, to keep the certs up to date, add the following to the -crontab: - - # crontab -e - ... - ~ * * * * acme-client garbash.com && rcctl reload httpd diff --git a/notes/004-mail-server.md b/notes/004-mail-server.md new file mode 100644 index 0000000..cf3d0d8 --- /dev/null +++ b/notes/004-mail-server.md @@ -0,0 +1,26 @@ +--- +title: 004-mail-server +--- + +# 004-mail-server + +Tues Sept 21, 2021 + +We threw this together late in our first pairing session to set +up the site. I think having a solid email server is an important +part of standing up a site (allows forwarding cron email to an +inbox that's read, etc). And of course giving out email accounts +is crucial to attract people to a tilde ;) + +The setup mostly followed Gilles' excellent post [1], but I replaced +the rspamd bits with opensmtpd-filter-dkimsign, which is super +simple to set up (see the README that comes with the installed pkg). + +I'll post the whole config soon once we get git hosting set up! + +EDIT: Sat Sep 25 00:20:01 EDT 2021 + +Git hosting is up and here's the config [2] + +[1]: https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/ +[2]: https://git.garbash.com/alex/config/file/etc/mail/smtpd.conf.html diff --git a/notes/004-mail-server.txt b/notes/004-mail-server.txt deleted file mode 100644 index 0197a03..0000000 --- a/notes/004-mail-server.txt +++ /dev/null @@ -1,20 +0,0 @@ -004-mail-server -- Tues Sept 21, 2021 - -We threw this together late in our first pairing session to set -up the site. I think having a solid email server is an important -part of standing up a site (allows forwarding cron email to an -inbox that's read, etc). And of course giving out email accounts -is crucial to attract people to a tilde ;) - -The setup mostly followed Gilles' excellent post [1], but I replaced -the rspamd bits with opensmtpd-filter-dkimsign, which is super -simple to set up (see the README that comes with the installed pkg). - -I'll post the whole config soon once we get git hosting set up! - -EDIT: Sat Sep 25 00:20:01 EDT 2021 - -Git hosting is up and here's the config [2] - -[1]: https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/ -[2]: https://git.garbash.com/alex/config/file/etc/mail/smtpd.conf.html diff --git a/notes/005-ssh-hardening.md b/notes/005-ssh-hardening.md new file mode 100644 index 0000000..523b848 --- /dev/null +++ b/notes/005-ssh-hardening.md @@ -0,0 +1,25 @@ +--- +title: 005-ssh-hardening +--- + +# 005-ssh-hardening + +Tues Sept 21, 2021 + +Just a quick note/reminder that one of the FIRST things you should +ALWAYS do on a new machine is make sure: + +1. root cannot log in (PermitRootLogin no) +2. passwords are not accepted (ssh-key only -- PasswordAuthentication no) + +Both these are set under /etc/ssh/sshd_config. Make sure to upload +your ~/.ssh/id_rsa.pub first to ~/.ssh/authorized_keys (so as not +to lock yourself out!) and then make the edits and reload the daemon: + + # rcctl reload sshd + +We waited literally only 4 hrs to do this and we already had script +kiddies knocking down our /var/log/authlog :( + +If my old coworker Joe was right about one thing, it's that the +internet these days is a cesspool. diff --git a/notes/005-ssh-hardening.txt b/notes/005-ssh-hardening.txt deleted file mode 100644 index d8b2a0e..0000000 --- a/notes/005-ssh-hardening.txt +++ /dev/null @@ -1,19 +0,0 @@ -005-ssh-hardening -- Tues Sept 21, 2021 - -Just a quick note/reminder that one of the FIRST things you should -ALWAYS do on a new machine is make sure: - -1. root cannot log in (PermitRootLogin no) -2. passwords are not accepted (ssh-key only -- PasswordAuthentication no) - -Both these are set under /etc/ssh/sshd_config. Make sure to upload -your ~/.ssh/id_rsa.pub first to ~/.ssh/authorized_keys (so as not -to lock yourself out!) and then make the edits and reload the daemon: - -# rcctl reload sshd - -We waited literally only 4 hrs to do this and we already had script -kiddies knocking down our /var/log/authlog :( - -If my old coworker Joe was right about one thing, it's that the -internet these days is a cesspool. diff --git a/notes/006-use-the-src.md b/notes/006-use-the-src.md new file mode 100644 index 0000000..61734c9 --- /dev/null +++ b/notes/006-use-the-src.md @@ -0,0 +1,51 @@ +--- +title: 006-use-the-src +--- + +# 006-us-the-src + +Tues Sept 21, 2021 + +Use the source, Luke! + +One of the main reasons to use a FOSS OS is that you can see the code! +For me as a dev, it's been a lifechanging experience. Often it's faster +to just look at the code than try to decipher Stack Overflow answers, +and I always learn more that way! + +Another perk of the \*BSD's is that all of their source is in one repo. +This can of course make SCM slow, but from a curious-developer perspective +it's a dream come true. + +OpenBSD uses cvs(1) to manage their source, but they publish a read-only +git(1) mirror to GitHub, which I like to use for familiarity sake. + +Traditionally, all the source lives in /usr/src , and OpenBSD expects +you to put it there (for build purposes). + +To get it: + +1. Add yourself to the wsrc and wobj groups so you can build without doas + + # usermod -G wsrc,wobj + +2. Clone a bare repo to /var/git (default /usr/src not big enough for .git) + + # mkdir /var/git + # chmod 775 /var/git + # chown root:wsrc /var/git + $ cd /var/git + $ git clone --bare https://github.com/openbsd/src + +3. Check out a new worktree at /usr/src + + $ git -C /var/git/src.git worktree add /usr/src + +4. Find your favorite tool and build it + + $ cd /usr/src/bin/ed + $ make obj # for out of tree build, see make(1) OBJDIR + $ make + $ ./obj/ed + +How cool is that? diff --git a/notes/006-use-the-src.txt b/notes/006-use-the-src.txt deleted file mode 100644 index 7dc7e1d..0000000 --- a/notes/006-use-the-src.txt +++ /dev/null @@ -1,45 +0,0 @@ -006-use-the-src -- Tues Sept 21, 2021 - -Use the source, Luke! - -One of the main reasons to use a FOSS OS is that you can see the code! -For me as a dev, it's been a lifechanging experience. Often it's faster -to just look at the code than try to decipher Stack Overflow answers, -and I always learn more that way! - -Another perk of the *BSD's is that all of their source is in one repo. -This can of course make SCM slow, but from a curious-developer perspective -it's a dream come true. - -OpenBSD uses cvs(1) to manage their source, but they publish a read-only -git(1) mirror to GitHub, which I like to use for familiarity sake. - -Traditionally, all the source lives in /usr/src , and OpenBSD expects -you to put it there (for build purposes). - -To get it: - -1. Add yourself to the wsrc and wobj groups so you can build without doas - - # usermod -G wsrc,wobj - -2. Clone a bare repo to /var/git (default /usr/src not big enough for .git) - - # mkdir /var/git - # chmod 775 /var/git - # chown root:wsrc /var/git - $ cd /var/git - $ git clone --bare https://github.com/openbsd/src - -3. Check out a new worktree at /usr/src - - $ git -C /var/git/src.git worktree add /usr/src - -4. Find your favorite tool and build it - - $ cd /usr/src/bin/ed - $ make obj # for out of tree build, see make(1) OBJDIR - $ make - $ ./obj/ed - -How cool is that? diff --git a/notes/007-git-coding.md b/notes/007-git-coding.md new file mode 100644 index 0000000..522bea8 --- /dev/null +++ b/notes/007-git-coding.md @@ -0,0 +1,75 @@ +--- +title: 007-git-coding +--- + +# 007-gi + +ding -- Fri Sept 24, 2021 + +git(1) is one of my favorite tools. All good tilde's should host it! +After all, tilde's are for sharing and what better way to share than +publishing your code! + +Git Hosting +----------- +Out of the box, git supports hosting for users with accounts via ssh. +You can clone like so: + + user@host:path/relative/to/home +or: + + user@host:/abs/path/on/host + +For anonymous access, git-daemon(1) can be configured to serve over +the git:// protocol. On OpenBSD, enable and start it with the path +to the directories to serve: + + $ rcctl enable gitdaemon + $ rcctl set gitdaemon flags "--base-path=/var/git" + $ rcctl start gitdaemon + +The last bit of the puzzle is of course the shared git layout! For +git-daemon to work, we need all users to put their files under the +same dir (/var/git). But, we want to prevent accidental clobbering +via stray rm -rf, so we make a directory for each user and chown +it to their account so soley they can access it: + + /var/git/alex + .../www + .../config + /var/git/anthony + ... + +Then, for easy clone URLs, we ln(1) the dir into the home directory: + + ln -s /var/git/$USER /home/$USER/git + +Now they can clone via $USER@garbash.com:git/REPO + +Web Hosting +----------- +git hosting is one thing, but these days everyone likes to show off +their code in the browser for onlookers. Enter stagit(1). + +I tried cgit(1), one of the more popular git-frontends, but with httpd(8)'s +chroot(8)-ing, it was kind of a pain to get the more advanced features. + +stagit(1) generates static HTML for individual repos, which is a nice +balance of flexible and lightweight. + +The hardest part here was that I had to hack stagit(1) and stagit-index(1) +to support our two-tiered directory layout (by default it only supports +single directory layouts). This turned out to be not _that_ hard. See +my fork [1] for the specifics. + +These HTML files are then generated on-the-fly at push time via git-hooks, +specifically a post-receive hook. + +The whole process requires quite a bit of setup at repo-creation time +(assigning ownership, description, clone-url, and the post-receive hook), +so I rolled it all into a script globally available to our users: `newrepo`. +That too is available via the system config files [2]. Give it a look! + + +[1]: https://git.garbash.com/alex/stagit/ +[2]: https://git.garbash.com/alex/config/ diff --git a/notes/007-git-coding.txt b/notes/007-git-coding.txt deleted file mode 100644 index 4fd06b0..0000000 --- a/notes/007-git-coding.txt +++ /dev/null @@ -1,69 +0,0 @@ -007-git-coding -- Fri Sept 24, 2021 - -git(1) is one of my favorite tools. All good tilde's should host it! -After all, tilde's are for sharing and what better way to share than -publishing your code! - -Git Hosting ------------ -Out of the box, git supports hosting for users with accounts via ssh. -You can clone like so: - - user@host:path/relative/to/home -or: - - user@host:/abs/path/on/host - -For anonymous access, git-daemon(1) can be configured to serve over -the git:// protocol. On OpenBSD, enable and start it with the path -to the directories to serve: - - $ rcctl enable gitdaemon - $ rcctl set gitdaemon flags "--base-path=/var/git" - $ rcctl start gitdaemon - -The last bit of the puzzle is of course the shared git layout! For -git-daemon to work, we need all users to put their files under the -same dir (/var/git). But, we want to prevent accidental clobbering -via stray rm -rf, so we make a directory for each user and chown -it to their account so soley they can access it: - - /var/git/alex - .../www - .../config - /var/git/anthony - ... - -Then, for easy clone URLs, we ln(1) the dir into the home directory: - - ln -s /var/git/$USER /home/$USER/git - -Now they can clone via $USER@garbash.com:git/REPO - -Web Hosting ------------ -git hosting is one thing, but these days everyone likes to show off -their code in the browser for onlookers. Enter stagit(1). - -I tried cgit(1), one of the more popular git-frontends, but with httpd(8)'s -chroot(8)-ing, it was kind of a pain to get the more advanced features. - -stagit(1) generates static HTML for individual repos, which is a nice -balance of flexible and lightweight. - -The hardest part here was that I had to hack stagit(1) and stagit-index(1) -to support our two-tiered directory layout (by default it only supports -single directory layouts). This turned out to be not _that_ hard. See -my fork [1] for the specifics. - -These HTML files are then generated on-the-fly at push time via git-hooks, -specifically a post-receive hook. - -The whole process requires quite a bit of setup at repo-creation time -(assigning ownership, description, clone-url, and the post-receive hook), -so I rolled it all into a script globally available to our users: `newrepo`. -That too is available via the system config files [2]. Give it a look! - - -[1]: https://git.garbash.com/alex/stagit/ -[2]: https://git.garbash.com/alex/config/ diff --git a/notes/008-local-irc.md b/notes/008-local-irc.md new file mode 100644 index 0000000..20c180a --- /dev/null +++ b/notes/008-local-irc.md @@ -0,0 +1,37 @@ +--- +title: 008-local-irc +--- + +# 008-local-irc + +Fri Sep 24 23:56:43 EDT 2021 + +Tonight I took the first steps towards on-tilde communication. +It's far from done, but it'll give ~anthony and I something to +chat on while we set up the other services! + +The current plan is to have (for security reasons) a IRC server +ONLY listening on localhost. Then, we'll spin up a bouncer for +users to connect to so they can get chat history while offline. +That bouncer will be exposed externally (either over TLS or over +wireguard). + +The first step was to install ngircd. To be honest, I didn't +survey the scene toooo much. I did a search: + + pkg_info -Q irc + +And just picked the ircd that seemed most promising. + +Set up was a simple service start: + + rcctl enable ngircd + rcctl start ngircd + +And the config file was super well documented so even with my +very beginner knowledge of server admin-ship, I was able to get +it up in no time! + +The config, of course, is public [1] + +[1]: https://git.garbash.com/alex/config/file/etc/ngircd/ngircd.conf.html diff --git a/notes/008-local-irc.txt b/notes/008-local-irc.txt deleted file mode 100644 index eb512ef..0000000 --- a/notes/008-local-irc.txt +++ /dev/null @@ -1,31 +0,0 @@ -008-local-irc -- Fri Sep 24 23:56:43 EDT 2021 - -Tonight I took the first steps towards on-tilde communication. -It's far from done, but it'll give ~anthony and I something to -chat on while we set up the other services! - -The current plan is to have (for security reasons) a IRC server -ONLY listening on localhost. Then, we'll spin up a bouncer for -users to connect to so they can get chat history while offline. -That bouncer will be exposed externally (either over TLS or over -wireguard). - -The first step was to install ngircd. To be honest, I didn't -survey the scene toooo much. I did a search: - - pkg_info -Q irc - -And just picked the ircd that seemed most promising. - -Set up was a simple service start: - - rcctl enable ngircd - rcctl start ngircd - -And the config file was super well documented so even with my -very beginner knowledge of server admin-ship, I was able to get -it up in no time! - -The config, of course, is public [1] - -[1]: https://git.garbash.com/alex/config/file/etc/ngircd/ngircd.conf.html diff --git a/notes/009-wireguard.md b/notes/009-wireguard.md new file mode 100644 index 0000000..500a108 --- /dev/null +++ b/notes/009-wireguard.md @@ -0,0 +1,70 @@ +--- +title: 009-wireguard +--- + +# 009-wireguard + +Tues Sep 28, 2021 + +Wireguard is probably one of the coolest technologies I've encountered +in a long time. The simplicity of public key auth (ssh-style where the +protocol doesn't care how you get the public key on the server) all +in the kernel? Sign me up! + +On our tilde, we want to set up wireguard so that we can provide vpn-only +services (for security reasons such as not allowing brute-force password +attempts). + +The very first of these services is IRC--we want people to be able to +connect from mobile devices and personal computers, but our network is +currently not password protected and has no services like NickServ, etc. + +The solution? Have it listen on a wireguard IP and distribute wg keys +to trusted tilde members :) + +I'll start with the obligatory RTFM -- wg(8) and ifconfig(8) are both +really well documented. However, there was a bit of fun hackery that went +down on our tuesday pair-admining call that's worth documenting! + +~anthony and I needed a simple tool to manage wireguard keys and IPs. +When a new device is to be given access we want to: + + 1) Generate a private key, public key, and wg-quick(1) config file + to distribute to the user + 2) Obtain the next numerical hostname + 3) Add the peer to our wg endpoint on the server + +To do this, we used a small sh(1) script that has a catalog of names in +a flat file like so: + + host1 10.6.6.1 + host2 10.6.6.2 + ... + +And then each host has a directory: + + host1/ + private.key + public.key + client.conf + +The tool is called wggen(1) [1], and it ends up effectively: + + 1) Creating a directory for NAME + 2) Generating a wg(8) key using openssl(1): + + openssl rand -base64 32 > private.key + + 3) Creating a temporary wg endpoint to get the public key using + the grep/cut hack in wg(8)'s EXAMPLES + 4) tail(1)-ing the host file to get the next available IP + 5) Using all the above to generate the client.conf + 6) Adding the wgpeer line to /etc/hostname.wg0 and restarting the + prod endpoint with sh /etc/netstart + +I'll leave the exact details as an exercise for the reader to go look +at the git repo :) + +Needless to say, this was a lot of fun to write! + +[1]: https://git.garbash.com/alex/config/file/usr/local/bin/wggen.html diff --git a/notes/009-wireguard.txt b/notes/009-wireguard.txt deleted file mode 100644 index 3fb4756..0000000 --- a/notes/009-wireguard.txt +++ /dev/null @@ -1,64 +0,0 @@ -009-wireguard -- Tues Sep 28, 2021 - -Wireguard is probably one of the coolest technologies I've encountered -in a long time. The simplicity of public key auth (ssh-style where the -protocol doesn't care how you get the public key on the server) all -in the kernel? Sign me up! - -On our tilde, we want to set up wireguard so that we can provide vpn-only -services (for security reasons such as not allowing brute-force password -attempts). - -The very first of these services is IRC--we want people to be able to -connect from mobile devices and personal computers, but our network is -currently not password protected and has no services like NickServ, etc. - -The solution? Have it listen on a wireguard IP and distribute wg keys -to trusted tilde members :) - -I'll start with the obligatory RTFM -- wg(8) and ifconfig(8) are both -really well documented. However, there was a bit of fun hackery that went -down on our tuesday pair-admining call that's worth documenting! - -~anthony and I needed a simple tool to manage wireguard keys and IPs. -When a new device is to be given access we want to: - - 1) Generate a private key, public key, and wg-quick(1) config file - to distribute to the user - 2) Obtain the next numerical hostname - 3) Add the peer to our wg endpoint on the server - -To do this, we used a small sh(1) script that has a catalog of names in -a flat file like so: - - host1 10.6.6.1 - host2 10.6.6.2 - ... - -And then each host has a directory: - - host1/ - private.key - public.key - client.conf - -The tool is called wggen(1) [1], and it ends up effectively: - - 1) Creating a directory for NAME - 2) Generating a wg(8) key using openssl(1): - - openssl rand -base64 32 > private.key - - 3) Creating a temporary wg endpoint to get the public key using - the grep/cut hack in wg(8)'s EXAMPLES - 4) tail(1)-ing the host file to get the next available IP - 5) Using all the above to generate the client.conf - 6) Adding the wgpeer line to /etc/hostname.wg0 and restarting the - prod endpoint with sh /etc/netstart - -I'll leave the exact details as an exercise for the reader to go look -at the git repo :) - -Needless to say, this was a lot of fun to write! - -[1]: https://git.garbash.com/alex/config/file/usr/local/bin/wggen.html diff --git a/notes/010-irc-bouncer.md b/notes/010-irc-bouncer.md new file mode 100644 index 0000000..cd06c3b --- /dev/null +++ b/notes/010-irc-bouncer.md @@ -0,0 +1,92 @@ +--- +title: 010-irc-bouncer +--- + +# 010-ir + +uncer -- Tues Sept 28, 2021 + +After ~anthony and I set up wggen(1), we could properly access IRC +outside of ssh(1) (on our laptops, phones, etc). + +The next missing piece of the IRC puzzle was setting up a bouncer. +For those less familiar with IRC (read: me 6 months ago), a bouncer +is simply a special IRC client that is always on, staying in the +channels for you, listening. When you connect, you then connect to +the bouncer, which feeds you missed messages. + +This is necessary because IRC has no concept of history or buffered +messages built in. So if you're not active on the network, there's +no way to get missed messages. + +Of course bouncers provide all sorts of other nice features--a single +login point for multiple networks (garbash, libera.chat, etc), +auto-away, logging support, etc. + +For our users on this tilde, we wanted to make sure they could have +chat history without having to set up their own bouncer. + +We picked soju(1) [1], since I've set it up before and I'm a general +fan of the software coming from the sourcehut team. It was relatively +painless to set up on OpenBSD: + + $ pkg_add go sqlite3 scdoc # dependencies + $ git clone https://git.sr.ht/~emersion/soju/ + $ cd soju + $ make + # make install + +Then, I added a new \_soju user using adduser(8) and created the cfg +to listen on our wireguard port in /home/\_soju/soju.cfg: + + listen irc+insecure://10.6.6.1:6677 + db sqlite3 /home/_soju/soju.db + +Finally, I used sojuctl(1) to add myself as a user: + + $ sojuctl -config /home/_soju/soju.cfg create-user alex -admin + +Add made a small /etc/rc.d script: + + #!/bin/ksh + daemon="/usr/local/bin/soju -config /home/_soju/soju.cfg" + daemon_user="_soju" + daemon_logger="daemon.info" + + . /etc/rc.d/rc.subr + + rc_bg=YES + + rc_cmd "$1" + +And enabled and started soju: + + # rcctl enable soju + # rcctl start soju + +We're still ironing out the kinks in the user registration process, but +the current process is to connect to the soju instance first and add +the local network like so: + +In irssi: + + /network add -sasl_username -sasl_password -sasl_mechanism PLAIN garbash + /server add -auto -net garbash irc.garbash.com 6677 + /connect garbash + +Once connected, start a DM with the BouncerServ (provided by soju) + + /msg BouncerServ help + network create -name garbash -addr irc+insecure://localhost:6667 + +Finally, modify our garbash network username to run soju in "single +upstream mode" (aka it should only connect to this one network) by +changing our username to be /garbash (the network we just created): + + /network modify -sasl_username /garbash garbash + /connect garbash + /save + +And 10 commands and 2 connections later, we have a bouncer! + +[1]: https://soju.im diff --git a/notes/010-irc-bouncer.txt b/notes/010-irc-bouncer.txt deleted file mode 100644 index 0d7c398..0000000 --- a/notes/010-irc-bouncer.txt +++ /dev/null @@ -1,86 +0,0 @@ -010-irc-bouncer -- Tues Sept 28, 2021 - -After ~anthony and I set up wggen(1), we could properly access IRC -outside of ssh(1) (on our laptops, phones, etc). - -The next missing piece of the IRC puzzle was setting up a bouncer. -For those less familiar with IRC (read: me 6 months ago), a bouncer -is simply a special IRC client that is always on, staying in the -channels for you, listening. When you connect, you then connect to -the bouncer, which feeds you missed messages. - -This is necessary because IRC has no concept of history or buffered -messages built in. So if you're not active on the network, there's -no way to get missed messages. - -Of course bouncers provide all sorts of other nice features--a single -login point for multiple networks (garbash, libera.chat, etc), -auto-away, logging support, etc. - -For our users on this tilde, we wanted to make sure they could have -chat history without having to set up their own bouncer. - -We picked soju(1) [1], since I've set it up before and I'm a general -fan of the software coming from the sourcehut team. It was relatively -painless to set up on OpenBSD: - - $ pkg_add go sqlite3 scdoc # dependencies - $ git clone https://git.sr.ht/~emersion/soju/ - $ cd soju - $ make - # make install - -Then, I added a new _soju user using adduser(8) and created the cfg -to listen on our wireguard port in /home/_soju/soju.cfg: - - listen irc+insecure://10.6.6.1:6677 - db sqlite3 /home/_soju/soju.db - -Finally, I used sojuctl(1) to add myself as a user: - - $ sojuctl -config /home/_soju/soju.cfg create-user alex -admin - -Add made a small /etc/rc.d script: - - #!/bin/ksh - daemon="/usr/local/bin/soju -config /home/_soju/soju.cfg" - daemon_user="_soju" - daemon_logger="daemon.info" - - . /etc/rc.d/rc.subr - - rc_bg=YES - - rc_cmd "$1" - -And enabled and started soju: - - # rcctl enable soju - # rcctl start soju - -We're still ironing out the kinks in the user registration process, but -the current process is to connect to the soju instance first and add -the local network like so: - -In irssi: - - /network add -sasl_username -sasl_password -sasl_mechanism PLAIN garbash - /server add -auto -net garbash irc.garbash.com 6677 - /connect garbash - -Once connected, start a DM with the BouncerServ (provided by soju) - - /msg BouncerServ help - network create -name garbash -addr irc+insecure://localhost:6667 - -Finally, modify our garbash network username to run soju in "single -upstream mode" (aka it should only connect to this one network) by -changing our username to be /garbash (the network we just created): - - /network modify -sasl_username /garbash garbash - /connect garbash - /save - -And 10 commands and 2 connections later, we have a bouncer! - -[1]: https://soju.im diff --git a/notes/011-backups.md b/notes/011-backups.md new file mode 100644 index 0000000..08e52ce --- /dev/null +++ b/notes/011-backups.md @@ -0,0 +1,71 @@ +--- +title: 011-backups +--- + +# 011-backups + +October 12, 2021 + +My usual take on server backups is "don't put anything worth +backing up on the server that's not stored in git elsewhere". + +This has treated me pretty well in the past. Source code, +configuration files, and even documentation on setup are all +stored in git both on the server and on my laptop, and so +I can sleep at night knowing a catastrophic disk failure wouldn't +mean I lost any serious work. + +This strategy breaks down when thinking about a tilde. First, +the array of services we're providing is _much_ more complex +than my normal blog server. Second, there are more people +involved! + +I want to guarantee any tilde members that I will at least try +my best to keep backups of their data in case of failure or +accidental deletion. + +There are tons of backup tools, but a lot of them are fairly +complex (with good reason I suppose.. compression, deduplication, +etc). Since this tilde is about exploring OpenBSD, I took the +opportunity to home-roll a simple backup solution with dump(8) +and restore(8). + +The meat of it is in a script I'm calling "dumpster" that runs +via cron with the day of the week (1-7) as the dump level +and a weekly job dumping the whole system fresh: + + #!/bin/sh + # dumpster -- taking out the garbash with dump(8) + + # %u is 1=mon 7=sun (unless given in ARGV) + LVL=${1:-"$(date +%u)"} + BAKDIR="/bak/$(date +%F)_$LVL" + + mkdir -p "$BAKDIR" + dump -$LVL -auf "$BAKDIR/root.dump.$LVL" / + dump -$LVL -auf "$BAKDIR/home.dump.$LVL" /home + dump -$LVL -auf "$BAKDIR/var.dump.$LVL" /var + +This dumps to /bak, which is a separate Linode Volume, which +has better data redundancy guarantees than the VPS volume and +can be detached/attached to other hosts in the event of VPS +failure. + +As you can see, I'm only really dumping areas that have user +data (/var for git, /home, and / for configs). /usr/\* can be +rebuilt from /var/backups/pkglist for the most part! + +A note to anyone trying this: the Linode Volume was a bit hacky +to get set up, since it expects to be mounting against a Linux +machine. Linode's console will error on attaching, but I found +that rebooting the host made the drive appear as wdN and from +there I was able to format it, etc. + +As a bonus, I took the opportunity to set up /altroot backups, +which is a brilliantly simple way to ensure you can boot into +a known-good state of your host even if something goes very +wrong with the main drive! + +Overall, I went from a backup-avoider to a backup-fan in the +process :) it's so cool to watch the daily script create dump +files of things that changed! diff --git a/notes/011-backups.txt b/notes/011-backups.txt deleted file mode 100644 index 25269cd..0000000 --- a/notes/011-backups.txt +++ /dev/null @@ -1,65 +0,0 @@ -011-backups -- October 12, 2021 - -My usual take on server backups is "don't put anything worth -backing up on the server that's not stored in git elsewhere". - -This has treated me pretty well in the past. Source code, -configuration files, and even documentation on setup are all -stored in git both on the server and on my laptop, and so -I can sleep at night knowing a catastrophic disk failure wouldn't -mean I lost any serious work. - -This strategy breaks down when thinking about a tilde. First, -the array of services we're providing is _much_ more complex -than my normal blog server. Second, there are more people -involved! - -I want to guarantee any tilde members that I will at least try -my best to keep backups of their data in case of failure or -accidental deletion. - -There are tons of backup tools, but a lot of them are fairly -complex (with good reason I suppose.. compression, deduplication, -etc). Since this tilde is about exploring OpenBSD, I took the -opportunity to home-roll a simple backup solution with dump(8) -and restore(8). - -The meat of it is in a script I'm calling "dumpster" that runs -via cron with the day of the week (1-7) as the dump level -and a weekly job dumping the whole system fresh: - - #!/bin/sh - # dumpster -- taking out the garbash with dump(8) - - # %u is 1=mon 7=sun (unless given in ARGV) - LVL=${1:-"$(date +%u)"} - BAKDIR="/bak/$(date +%F)_$LVL" - - mkdir -p "$BAKDIR" - dump -$LVL -auf "$BAKDIR/root.dump.$LVL" / - dump -$LVL -auf "$BAKDIR/home.dump.$LVL" /home - dump -$LVL -auf "$BAKDIR/var.dump.$LVL" /var - -This dumps to /bak, which is a separate Linode Volume, which -has better data redundancy guarantees than the VPS volume and -can be detached/attached to other hosts in the event of VPS -failure. - -As you can see, I'm only really dumping areas that have user -data (/var for git, /home, and / for configs). /usr/* can be -rebuilt from /var/backups/pkglist for the most part! - -A note to anyone trying this: the Linode Volume was a bit hacky -to get set up, since it expects to be mounting against a Linux -machine. Linode's console will error on attaching, but I found -that rebooting the host made the drive appear as wdN and from -there I was able to format it, etc. - -As a bonus, I took the opportunity to set up /altroot backups, -which is a brilliantly simple way to ensure you can boot into -a known-good state of your host even if something goes very -wrong with the main drive! - -Overall, I went from a backup-avoider to a backup-fan in the -process :) it's so cool to watch the daily script create dump -files of things that changed! -- libgit2 1.1.1