joj3 setup #32

New Issue

manuel · 2024-09-08T17:23:51+08:00

manuel commented

2024-09-08 17:23:51 +08:00

goal: ease joj deployment through a (ephemeral?) clone of a "template container"

idea:

prepare a generic joj template container (JTC)
prepare external hooks to automatically run "per course" deployment (run on host at container creation)
prepare internal hooks to automatically run "per course" deployment (run inside container on 1st run)
write basic usage doc

goal: ease joj deployment through a (ephemeral?) clone of a "template container" idea: - [x] prepare a generic joj template container (JTC) - [x] prepare external hooks to automatically run "per course" deployment (run on host at container creation) - [x] prepare internal hooks to automatically run "per course" deployment (run inside container on 1st run) - [ ] write basic usage doc

manuel added the

enhancement

component

framework

labels 2024-09-08 17:23:51 +08:00

manuel self-assigned this 2024-09-08 17:23:51 +08:00

周赵嘉程521432910016 was assigned by manuel

2024-09-08 17:23:51 +08:00

manuel commented

2024-09-08 17:31:55 +08:00

@zzjc123 can you please list what software should be installed in JTC (docker, joj3/go, etc.)

for the hooks, we probably want to start with a simple script to

install course specific software
add an ssh key
configure joj3

in the end the hook could simply clone a repo and run the above tasks based on the files in the repo (eg. list of packages to install, ssh keys of TAs, etc.)

@zzjc123 can you please list what software should be installed in JTC (docker, joj3/go, etc.) for the hooks, we probably want to start with a simple script to - install course specific software - add an ssh key - configure joj3 in the end the hook could simply clone a repo and run the above tasks based on the files in the repo (eg. list of packages to install, ssh keys of TAs, etc.)

周赵嘉程521432910016 commented

2024-09-08 17:38:47 +08:00

I think git, golang, docker might be enough

git (clone the latest JOJ repo, or we can just download release with wget)
docker (create images)
golang (compile the joj3, can be ignored if we download release)

I think git, golang, docker might be enough - git (clone the latest JOJ repo, or we can just download release with wget) - docker (create images) - golang (compile the joj3, can be ignored if we download release)

manuel commented

2024-09-08 17:41:13 +08:00

what dependencies are needed for a JOJ release? (containers should ship a joj release)?

周赵嘉程521432910016 commented

2024-09-08 17:43:57 +08:00

you mean what softwares are needed for compiling the job or packages we used? If softwares I think golang itself is enough.

周赵嘉程521432910016 commented

2024-09-08 17:50:08 +08:00

containers should ship a joj release?

We may not need compile JOJ release if basic docker images (containing joj) are prepared.

> containers should ship a joj release? We may not need compile JOJ release if basic docker images (containing joj) are prepared.

manuel commented

2024-09-09 21:48:56 +08:00

new joj container ready for testing: ssh -p 22151 tt@111.186.58.48

\to all pub keys have been imported, so everybody should be able to connect. please now use this one for testing

the container is "minimal", nvim, docker, git are installed. please report here anything that is missing.

notes:

this container should feature everything that is needed to run JOJ and nothing more. the goal is just to have a "secure/isolated" environment to run joj through docker.
this container is unprivileged (prev one was priviledged)

@all

new joj container ready for testing: `ssh -p 22151 tt@111.186.58.48` $\to$ all pub keys have been imported, so everybody should be able to connect. please now use this one for testing the container is "minimal", nvim, docker, git are installed. please report here anything that is missing. notes: - this container should feature everything that is needed to run JOJ and nothing more. the goal is just to have a "secure/isolated" environment to run joj through docker. - this container is unprivileged (prev one was priviledged) @all

manuel commented

2024-09-10 08:23:09 +08:00

tt user is for teaching team to connect, we probably want a joj user to run joj

周赵嘉程521432910016 commented

2024-09-10 12:19:18 +08:00

register docker process

docker pull gitea/act_runner:latest

# generate config to modify if needed
docker run --entrypoint="" --rm -it gitea/act_runner:latest act_runner generate-config > config.yaml

# create image
docker run \
    -v $(pwd)/config.yaml:/config.yaml \
    -v $(pwd)/data:/data \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -e CONFIG_FILE=/config.yaml \
    -e GITEA_INSTANCE_URL=https://focs.ji.sjtu.edu.cn/git \
    -e GITEA_RUNNER_REGISTRATION_TOKEN=GdyxWCm3ga7ITvvgzqJtH1TltE6DhaCInz5fbm85 \
    -e GITEA_RUNNER_NAME=templateRunner \
    --name templateRunner \
    -d gitea/act_runner

# register docker process ```bash docker pull gitea/act_runner:latest # generate config to modify if needed docker run --entrypoint="" --rm -it gitea/act_runner:latest act_runner generate-config > config.yaml # create image docker run \ -v $(pwd)/config.yaml:/config.yaml \ -v $(pwd)/data:/data \ -v /var/run/docker.sock:/var/run/docker.sock \ -e CONFIG_FILE=/config.yaml \ -e GITEA_INSTANCE_URL=https://focs.ji.sjtu.edu.cn/git \ -e GITEA_RUNNER_REGISTRATION_TOKEN=GdyxWCm3ga7ITvvgzqJtH1TltE6DhaCInz5fbm85 \ -e GITEA_RUNNER_NAME=templateRunner \ --name templateRunner \ -d gitea/act_runner ```

周赵嘉程521432910016 commented

2024-09-10 12:20:23 +08:00

we may need to change the ownership of runner-image folder to keep it secure

manuel commented

2024-09-10 20:01:03 +08:00

progress:

pipx, go-judge, acl, tmux, sudo installed in JTC
teapot installed through pipx in joj-test1
initial config repo prepared

@zzjc123

is the GITEA_RUNNER_REGISTRATION_TOKEN different for each course/instance of the runner?
joj3 is a single binary?

we may need to change the ownership of runner-image folder to keep it secure

we'll use acl to ensure things are ok

progress: - pipx, go-judge, acl, tmux, sudo installed in JTC - teapot installed through pipx in joj-test1 - initial config repo prepared @zzjc123 - is the `GITEA_RUNNER_REGISTRATION_TOKEN` different for each course/instance of the runner? - joj3 is a single binary? > we may need to change the ownership of runner-image folder to keep it secure we'll use acl to ensure things are ok

周赵嘉程521432910016 commented

2024-09-10 23:15:24 +08:00

progress:

pipx, go-judge, acl, tmux, sudo installed in JTC

teapot installed through pipx in joj-test1

initial config repo prepared

@zzjc123

is the GITEA_RUNNER_REGISTRATION_TOKEN different for each course/instance of the runner?

joj3 is a single binary?

we may need to change the ownership of runner-image folder to keep it secure

we'll use acl to ensure things are ok

TOKEN can be repo level or higher level, currently repo level

We need joj3, healthcheck binary to run whole process

> progress: > - pipx, go-judge, acl, tmux, sudo installed in JTC > - teapot installed through pipx in joj-test1 > - initial config repo prepared > > @zzjc123 > - is the `GITEA_RUNNER_REGISTRATION_TOKEN` different for each course/instance of the runner? > - joj3 is a single binary? > > > we may need to change the ownership of runner-image folder to keep it secure > > we'll use acl to ensure things are ok > TOKEN can be repo level or higher level, currently repo level We need joj3, healthcheck binary to run whole process

周赵嘉程521432910016 commented

2024-09-11 16:38:58 +08:00

docker run \
    -v $(pwd)/config.yaml:/config.yaml \
    -v $(pwd)/data:/data \
    -v $(pwd)/bin:/joj3-bin \
    -v /usr/bin/pipx:/usr/bin/pipx \
    -v /home/tt/.local/config:/root/.local/config \
    -v /home/tt/.local/bin:/root/.local/bin \
    -v /home/tt/.local/pipx/venvs/joint-teapot:/root/.local/pipx/venvs/joint-teapot \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -e CONFIG_FILE=/config.yaml \
    -e GITEA_INSTANCE_URL=https://focs.ji.sjtu.edu.cn/git \
    -e GITEA_RUNNER_REGISTRATION_TOKEN=GdyxWCm3ga7ITvvgzqJtH1TltE6DhaCInz5fbm85 \
    -e GITEA_RUNNER_NAME=templateRunner \
    -e PATH="/joj3-bin:$PATH" \
    --name templateRunner \
    -d gitea/act_runner

For this file, binary in $(pwd)/bin works, but joint-teapot couldn't work

```bash docker run \ -v $(pwd)/config.yaml:/config.yaml \ -v $(pwd)/data:/data \ -v $(pwd)/bin:/joj3-bin \ -v /usr/bin/pipx:/usr/bin/pipx \ -v /home/tt/.local/config:/root/.local/config \ -v /home/tt/.local/bin:/root/.local/bin \ -v /home/tt/.local/pipx/venvs/joint-teapot:/root/.local/pipx/venvs/joint-teapot \ -v /var/run/docker.sock:/var/run/docker.sock \ -e CONFIG_FILE=/config.yaml \ -e GITEA_INSTANCE_URL=https://focs.ji.sjtu.edu.cn/git \ -e GITEA_RUNNER_REGISTRATION_TOKEN=GdyxWCm3ga7ITvvgzqJtH1TltE6DhaCInz5fbm85 \ -e GITEA_RUNNER_NAME=templateRunner \ -e PATH="/joj3-bin:$PATH" \ --name templateRunner \ -d gitea/act_runner ``` For this file, binary in $(pwd)/bin works, but joint-teapot couldn't work

周赵嘉程521432910016 commented

2024-09-11 16:40:22 +08:00

I think it might related to virtual env but even add it , it returns

bash: joint-teapot: No such file or directory

I think it might related to virtual env but even add it , it returns ```bash bash: joint-teapot: No such file or directory ```

manuel commented

2024-09-11 16:46:47 +08:00

1st line of .local/bin/joint-teapot

#!/home/tt/.local/pipx/venvs/joint-teapot/bin/python

might need to change (at least) this. might be easier to run pipx ./joint-teapot inside docker? then install would be ok i guess

mounting the whole .local/pipx could also help? (instead of the teapot venv)

1st line of `.local/bin/joint-teapot` ```python #!/home/tt/.local/pipx/venvs/joint-teapot/bin/python ``` might need to change (at least) this. might be easier to run `pipx ./joint-teapot` inside docker? then install would be ok i guess mounting the whole `.local/pipx` could also help? (instead of the teapot venv)

周赵嘉程521432910016 commented

2024-09-11 16:50:36 +08:00

how can we run pipx inside docker?

manuel commented

2024-09-11 17:02:14 +08:00

seems python3 is missing from the containter :-D

😆 1

周赵嘉程521432910016 commented

2024-09-11 17:04:01 +08:00

Can we just leave teapot in image?

manuel commented

2024-09-11 17:08:38 +08:00

not sure, having bits and pieces in different places make it harder to maintain.teapot is also likely to receive quite a few updates so more maintenance

notes:

at least some of the paths are correct, try to add python then we can see how far we are from maing it work.
other course specific binary will also have to be imported (eg. clang, clang++, octave, clang-tidy, cppcheck, cpplint) with all their dependencies

not sure, having bits and pieces in different places make it harder to maintain.teapot is also likely to receive quite a few updates so more maintenance notes: - at least some of the paths are correct, try to add python then we can see how far we are from maing it work. - other course specific binary will also have to be imported (eg. clang, clang++, octave, clang-tidy, cppcheck, cpplint) with all their dependencies

周赵嘉程521432910016 commented

2024-09-11 17:21:54 +08:00

Install a python in container or mount python path? I tried to mount all python path I can find, but still couldn't make anythin work.