diff --git a/README.md b/README.md index 425cd03..81463f1 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,79 @@ These steps are executed in runner-images. We use `sudo -u tt` to elevate the pe 3. With `joint-teapot joj3-failed-table`, it will update the failed table file in grading repo. 4. With `joint-teapot joj3-create-result-issue`, it create an issue in the submitter's repo to show the results. -## Models +## Components + +### Binaries (under `/cmd` and `/pkg`) + +#### JOJ3 + +JOJ3 itself. Parsers and executors are compiled into the JOJ3 binary. + +#### Sample + +Just a sample on how to write an executable that can be called by the executor. + +#### HealthCheck + +The repohealth check will return a json list to for check result. The structure follows the score-comment pattern. + +HealthCheck currently includes, `reposize`, `forbidden file`, `Metafile existence`, `non-ascii character` in file and message, `release tag`, and `ci files invariance` check. + +The workflow is `joj3` pass cli args to healthcheck binary. See `./cmd/healthcheck/main.go` to view all flags. + +### Executors (under `/internal/executors`) + +#### Dummy + +Do not execute any command. Just return empty `ExecutorResult` slice. + +#### Sandbox + +Run the commands in `go-judge` and output the `ExecutorResult` slice. Note: we communicate with `go-judge` using gRPC, which means `go-judge` can run anywhere as the gRPC connection can be established. In deployment, `go-judge` runs in the host machine of the Gitea runner. + +### Parsers (under `/internal/parsers`) + +#### Clang Tidy + +Parser for `clang-tidy`, check `/examples/clangtidy` on how to call `clang-tidy` with proper parameters. + +#### Cppcheck + +Parser for `cppcheck`, check `/examples/cppcheck` on how to call `cppcheck` with proper parameters. + +#### Cpplint + +Parser for `cpplint`, check `/examples/cpplint` on how to call `cpplint` with proper parameters. + +#### Diff + +Compare the specified output of `ExecutorResult` with the content of the answer file. If they are the same, then score will be given. Just like a normal online judge system. + +#### Dummy + +Does not parse the output of `ExecutorResult`. It just output what is set inside the configuration file as score and comment. Currently it is used to output metadata for `joint-teapot`. + +In `joint-teapot`, it will take the content before `-` of the comment of the first stage with name `metadata` as the exercise name and record in the scoreboard. (e.g. If the comment is `p2-s2-0xdeadbeef`, then the exercise name is `p2`.) + +The comment in `metadata` can also be used to skip teapot commands. With `skip-teapot` in the comment, teapot will not run. And with `skip-scoreboard`, `skip-failed-table`, and `skip-result-issue`, only the corresponding step will be skipped, while the others will be executed. (e.g. If the comment is `p2-s2-0xdeadbeef-skip-scoreboard-skip-result-issue`, then only failed table step in teapot will run.) + +#### Healthcheck + +Parser for the `healthcheck` binary mentioned before. + +#### Keyword + +Match the given keyword from the specified output of `ExecutorResult`. For each match, a deduction of score is given. Can be useful if we do not have a specific parser for a code quality tool. Check `/examples/keyword`. + +#### Result Status + +Only check if all the status of the executor is `StatusAccepted`. Can be used to check whether the command run in the executor exit normally. + +#### Sample + +Parser for the `sample` binary mentioned before. Only used as a sample. + +## Models (for developers only) The program parses the configuration file to run multiple stages. @@ -116,78 +188,3 @@ Check the `Result` at . - `Score int`: score of the stage. - `Comment string`: comment on the stage. - -## Binaries (under `/cmd` and `/pkg`) - -### JOJ3 - -JOJ3 itself. - -#### CLI arguments - -1. `-meta-conf` to specify the meta configuration file path. JOJ3 will try to match the commit message by the `regex`, once matched, the configuration file in the `filename` will be load and run the whole process. -2. `-msg` to specify message to trigger the running. If empty, JOJ3 will use git commit message on HEAD. - -### Sample - -Just a sample on how to write an executable that can be called by the executor. - -### HealthCheck - -The repohealth check will return a json list to for check result. The structure follows the score-comment pattern. - -HealthCheck currently includes, `reposize`, `forbidden file`, `Metafile existence`, `non-ascii character` in file and message, `release tag`, and `ci files invariance` check. - -The workflow is `joj3` pass cli args to healthcheck binary. See `./cmd/healthcheck/main.go` to view all flags. - -## Executors (under `/internal/executors`) - -### Dummy - -Do not execute any command. Just return empty `ExecutorResult` slice. - -### Sandbox - -Run the commands in `go-judge` and output the `ExecutorResult` slice. Note: we communicate with `go-judge` using gRPC, which means `go-judge` can run anywhere as the gRPC connection can be established. In deployment, `go-judge` runs in the host machine of the Gitea runner. - -## Parsers (under `/internal/parsers`) - -### Clang Tidy - -Parser for `clang-tidy`, check `/examples/clangtidy` on how to call `clang-tidy` with proper parameters. - -### Cppcheck - -Parser for `cppcheck`, check `/examples/cppcheck` on how to call `cppcheck` with proper parameters. - -### Cpplint - -Parser for `cpplint`, check `/examples/cpplint` on how to call `cpplint` with proper parameters. - -### Diff - -Compare the specified output of `ExecutorResult` with the content of the answer file. If they are the same, then score will be given. Just like a normal online judge system. - -### Dummy - -Does not parse the output of `ExecutorResult`. It just output what is set inside the configuration file as score and comment. Currently it is used to output metadata for `joint-teapot`. - -In `joint-teapot`, it will take the content before `-` of the comment of the first stage with name `metadata` as the exercise name and record in the scoreboard. (e.g. If the comment is `p2-s2-0xdeadbeef`, then the exercise name is `p2`.) - -The comment in `metadata` can also be used to skip teapot commands. With `skip-teapot` in the comment, teapot will not run. And with `skip-scoreboard`, `skip-failed-table`, and `skip-result-issue`, only the corresponding step will be skipped, while the others will be executed. (e.g. If the comment is `p2-s2-0xdeadbeef-skip-scoreboard-skip-result-issue`, then only failed table step in teapot will run.) - -### Healthcheck - -Parser for the `healthcheck` binary mentioned before. - -### Keyword - -Match the given keyword from the specified output of `ExecutorResult`. For each match, a deduction of score is given. Can be useful if we do not have a specific parser for a code quality tool. Check `/examples/keyword`. - -### Result Status - -Only check if all the status of the executor is `StatusAccepted`. Can be used to check whether the command run in the executor exit normally. - -### Sample - -Parser for the `sample` binary mentioned before. Only used as a sample.