diff --git a/%5BAdmin%5D-Course-container-setup-%5BWIP%5D.md b/%5BAdmin%5D-Course-container-setup-%5BWIP%5D.md deleted file mode 100644 index bbc285b..0000000 --- a/%5BAdmin%5D-Course-container-setup-%5BWIP%5D.md +++ /dev/null @@ -1,15 +0,0 @@ -# Course container setup - -## Container deployment - -1. Create a new `bot-${REPO_OWNER_LOWER}` user on Gitea -2. Generate a new token for `bot-${REPO_OWNER_LOWER}` with permissions: - - issue `rw` - - repository `rw` - - organizaion `r` -3. On `${REPO_OWNER_LOWER}-joj` repo settings, add the gitea token of `bot-${REPO_OWNER_LOWER}` as value to `Settings->Actions->Secrets->Add Secret`, with name `TEAPOT_GITEA_TOKEN` -4. Edit `root/deploy.conf` based on needed software/tools -5. Deploy a new container for the course - - create a zfs volume - - setup permissions - - deploy diff --git a/%5BAdmin%5D-JOJ-server-setup.md b/%5BAdmin%5D-JOJ-server-setup.md new file mode 100644 index 0000000..a2f5e56 --- /dev/null +++ b/%5BAdmin%5D-JOJ-server-setup.md @@ -0,0 +1,87 @@ +# JOJ server setup guide + +## Host setup + +Prepare unprivileged lxc containers + +- `apt install sudo zfs-dkms zfsutils-linux jq lxc uidmap` +- ssh setup: + - generate a new ssh key `ssh-keygen -t ed25519` + - add to bot-joj +- clone joj3-hs `git clone ssh://git@focs.ji.sjtu.edu.cn:2222/JOJ/JOJ3-hs.git` +- copy joj3-hs/admin/config + - `/etc/sudoers.d/joj-deploy` + - `/etc/subuid` + - `/etc/subgid` + - `/etc/lxc/lxc-usernet` + - `/home/ja/.config/lxc/default.conf` +- as `ja` user + - `mkdir -p ~/.local/share/lxc` + - `chmod o+x ~/.local/share/lxc` + + +## Guest setup + +Before setting up a JOJ container for COURSE, ensure the following exist and are properly setup: + +- bot-COURSE user +- COURSE organisation +- COURSE-joj repo + +All actions are performed as `ja`. + +- Clone the course-joj repo +- Edit etc/joj-container-config.conf based on needed software/tools +- Run `joj-container-deploy COURSE` + +*Note.* the ssh key, act_runner token, and teapot token are backed up in `~/courses/COURSE` + +### Advanced go-judge setup + +Network access: +- add `-net-share` to service file +- add `/etc/ssl` to `/etc/go-judge/mount.yaml` + +*Note.* used in sfocs-joj + +Mount directories: +- edit `/etc/go-judge/mount.yaml` +- permissions might need to adjusted, eg. group `nogroup` with write access if mounting a directory from `$HOME` + +*Note.* used in `sfocs-joj` (elm-packages) and `ece477-joj` (latex font setup) + + +## Trouble shooting + +### Containers stop automatically + +Fixed using `loginctl enable-linger frown` + +### Mounting `sysfs` fails + +- check umask from `/proc/xxx/status` where `xxx` is the `pid` of systemd --user +- adjust `/etc/pam.d/common-session-noninteractive` with content +``` +session optional pam_umask.so umask=0002 +``` + +### lxc-copy fails + +Reason: `apparmor` bug in some debian versions + +Edit `/etc/apparmor.d/usr.bin.lxc-copy` + +``` + mount options=(rw,move) -> /home/ja/.local/share/lxc/{,**}, +``` + +### lxc fails to assign CPUs + +On the host run: + +``` +echo "+cpuset" > /sys/fs/cgroup/user.slice/cgroup.subtree_control +echo "+cpuset" > /sys/fs/cgroup/user.slice/user-1000.slice/cgroup.subtree_control +echo "+cpuset" > /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/cgroup.subtree_control` +echo "+cpuset" > /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/cgroup.subtree_control +``` \ No newline at end of file