JOJ server setup guide

Host setup

Prepare unprivileged lxc containers

apt install sudo zfs-dkms zfsutils-linux jq lxc uidmap
ssh setup:
- generate a new ssh key ssh-keygen -t ed25519
- add to bot-joj
clone joj3-hs git clone ssh://git@focs.ji.sjtu.edu.cn:2222/JOJ/JOJ3-hs.git
copy joj3-hs/admin/config
- /etc/sudoers.d/joj-deploy
- /etc/subuid
- /etc/subgid
- /etc/lxc/lxc-usernet
- /home/ja/.config/lxc/default.conf
as ja user
- mkdir -p ~/.local/share/lxc
- chmod o+x ~/.local/share/lxc

Guest setup

Before setting up a JOJ container for COURSE, ensure the following exist and are properly setup:

bot-COURSE user
COURSE organisation
COURSE-joj repo

All actions are performed as ja.

Clone the course-joj repo
Edit etc/joj-container-config.conf based on needed software/tools
Run joj-container-deploy COURSE

Note. the ssh key, act_runner token, and teapot token are backed up in ~/courses/COURSE

Advanced go-judge setup

Network access:

add -net-share to service file
add /etc/ssl to /etc/go-judge/mount.yaml

Note. used in sfocs-joj

Mount directories:

edit /etc/go-judge/mount.yaml
permissions might need to adjusted, eg. group nogroup with write access if mounting a directory from $HOME

Note. used in sfocs-joj (elm-packages) and ece477-joj (latex font setup)

Trouble shooting

Containers stop automatically

Fixed using loginctl enable-linger frown

Mounting `sysfs` fails

check umask from /proc/xxx/status where xxx is the pid of systemd --user
adjust /etc/pam.d/common-session-noninteractive with content

session optional                        pam_umask.so umask=0002

lxc-copy fails

Reason: apparmor bug in some debian versions

Edit /etc/apparmor.d/usr.bin.lxc-copy

  mount options=(rw,move) -> /home/ja/.local/share/lxc/{,**},

lxc fails to assign CPUs

On the host run:

echo "+cpuset" > /sys/fs/cgroup/user.slice/cgroup.subtree_control
echo "+cpuset" > /sys/fs/cgroup/user.slice/user-1000.slice/cgroup.subtree_control
echo "+cpuset" > /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/cgroup.subtree_control` 
echo "+cpuset" >  /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/cgroup.subtree_control