JOJ/JOJ3
25
6
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Update [TA] JOJ preparation and testing processes

manuel 2025-09-27 12:04:54 +08:00
parent 9e6c3f7f2b
commit 030ff5bc19
2 changed files with 94 additions and 60 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
%5BTA%5D-JOJ-preparation-and-testing-process.md

@ -1,60 +0,0 @@
# JOJ preparation and testing process
## Preparation
**Important.** Never release or make JOJ available before it has been fully tested.
## Testing
The first time JOJ is setup, make sure to run extensive tests in a separate test repository. When all tests are successful , create an orphan branch in `course-joj` with name `test-repoName`, where `repoName` corresponds to the name of the repository used by students. This new orphan branch must have the exact same structure as the repository used by students. For instance, if students have a `homework` repository, then call the branch `test-homework` and its content should have the exact same structure as the `homework` repository, e.g. one directory per homework with similar filenames, etc..
## Known issues
### Gitea actions and LFS
Bug: https://github.com/actions/checkout/issues/1830#issuecomment-2314758792
Solution: adjust the YAML workflow fot checkout LFS object separately.
<details><summary>Sample YAML worklow file.</summary>
```yaml
name: Run JOJ3 on Push
on: [push]
jobs:
run:
runs-on: focs-ubuntu-latest-slim
container:
volumes:
- /home/tt/.config:/home/tt/.config
- /home/tt/.cache:/home/tt/.cache
- /home/tt/.ssh:/home/tt/.ssh
steps:
- name: Check out repository code
uses: actions/checkout@focs
with:
lfs: false
persist-credentials: 'true'
- name: Checkout lfs
run: |
git lfs install --local
AUTH=$(git config --local http.https://focs.ji.sjtu.edu.cn/.extraheader)
git config --local --unset http.https://focs.ji.sjtu.edu.cn/.extraheader
git config --local http.${{ gitea.server_url }}/${{ gitea.repository }}.git/info/lfs/objects/batch.extraheader "$AUTH"
git lfs pull
- name: run joj3
run: |
sudo -E -u tt joj3 -conf-root /home/tt/.config/joj
```
</details>
### Memory sanitizer

94
%5BTA%5D-JOJ-preparation-and-testing-processes.md Normal file

@ -0,0 +1,94 @@
# JOJ preparation and testing processes
## Preparation
**Important.** Never release or make JOJ available before it has been fully tested.
## Testing process
### Initial JOJ setup
If JOJ has already been used in previous semesters, you can likely skip to the next section and only refer to this section if necessary.
The first time JOJ is setup:
- run extensive tests in a separate test repository
- ensure all test cases can be passed
- if different JOJ groups are available, then run them independently to ensure everything works as expected
- test on non working code, i.e. check the behavior if
- some test cases fail
- if the code doesn't compile
- if the commit message is incorrect (non-conventional, wrong scope, missing footer, etc.)
- when all previous tests are fine:
- create an orphan branch in `course-joj` with name `test-repoName`, where `repoName` corresponds to the name of the repository used by students
- the new orphan branch must have the exact same structure as the repository used by students
- switch to the new branch and follow instructions from the next section
**Very important.**
- most of the submitted code will not get full score
- ensure the setup works for both right and **wrong submissions**
### New semester JOJ setup
**Important.** Never blindly reuse the setup from previous semesters.
Only release when ready:
- test the setup using a separate JOJ "conf-root" inaccessible to students, e.g. if students' workflow has `conf-root` setup to `projects` then test using `projects-test`
- if an assignment has several tasks, don't release them one by one
- release a whole assignment at once
Always ensure to test before releasing to students:
- switch to the corresponding branch on `course-joj`
- push an empty commit message `test(task): new semester setup [all]`
- check the issue and ensure everything works as expected
- in case of problem refer to the above "Initial JOJ setup" section and run further tests in another repository
## Known issues
### Gitea actions and LFS
Bug: https://github.com/actions/checkout/issues/1830#issuecomment-2314758792
Solution: adjust the YAML workflow fot checkout LFS object separately.
<details><summary>Sample YAML worklow file.</summary>
```yaml
name: Run JOJ3 on Push
on: [push]
jobs:
run:
runs-on: focs-ubuntu-latest-slim
container:
volumes:
- /home/tt/.config:/home/tt/.config
- /home/tt/.cache:/home/tt/.cache
- /home/tt/.ssh:/home/tt/.ssh
steps:
- name: Check out repository code
uses: actions/checkout@focs
with:
lfs: false
persist-credentials: 'true'
- name: Checkout lfs
run: |
git lfs install --local
AUTH=$(git config --local http.https://focs.ji.sjtu.edu.cn/.extraheader)
git config --local --unset http.https://focs.ji.sjtu.edu.cn/.extraheader
git config --local http.${{ gitea.server_url }}/${{ gitea.repository }}.git/info/lfs/objects/batch.extraheader "$AUTH"
git lfs pull
- name: run joj3
run: |
sudo -E -u tt joj3 -conf-root /home/tt/.config/joj
```
</details>
### Memory sanitizer
According to our current observation, MemorySanitizer with `iostream` linked with uninstrumented libc++ will not work. Not sure about other libraries/headers.
Explanation: https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo