JOJ/JOJ3-config-generator
27
1
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Bug in judge/diff parser. #15

New Issue
Closed
opened 2025-05-22 11:19:16 +08:00 by 姚云翔523370910110 · 4 comments
姚云翔523370910110 commented 2025-05-22 11:19:16 +08:00
Member
Copy Link

Problem 1:

  • delete default stdin field for diff parser.
"stdin": {
    "content": ""
},

This part should be delete when generating .json. The location of this is in:

...

"executor": {
    "name": "sandbox",
    "with": {
        "default": {
            "args": [
                "./students/p1/compile/build/pretest"
            ],
            "env": [
                "PATH=/usr/bin:/bin:/usr/local/bin"
            ],
            "stdin": {
                "content": ""
            },
            "stdout": {
                "name": "stdout",
                "max": 2097152
            },
            "stderr": {
                "name": "stderr",
                "max": 2097152
            },
            "cpuLimit": 2000000000,
            "clockLimit": 4000000000,
            "memoryLimit": 2097152,
            "stackLimit": 0,
            "procLimit": 50,
            "cpuRateLimit": 0,
            "cpuSetLimit": "",
            "copyIn": {
                "p1/build/pretest": {
                    "src": "/home/tt/.config/joj/p1/build/pretest"
                }
            },
            "copyInCached": {
                "students/p1/compile/build/pretest": "students/p1/compile/build/pretest",
                "students/p1/compile/build/compile_commands.json": "students/p1/compile/build/compile_commands.json",
                "students/p1/compile/p1.cpp": "students/p1/compile/p1.cpp"
            },
            "copyInDir": ".",
            "copyOut": [
                "stdout",
                "stderr"
            ],
            "copyOutCached": [],
            "copyOutMax": 0,
            "copyOutDir": "",
            "tty": false,
            "strictMemoryLimit": false,
            "dataSegmentLimit": false,
            "addressSpaceLimit": false
        },
        "cases": [
            {
                "stdin": {
                    "src": "/home/tt/.config/joj/students/p1/case0.in"
                },
                "cpuLimit": 1000000000,
                "clockLimit": 2000000000,
                "memoryLimit": 268435456
            },
            {
                "stdin": {
                    "src": "/home/tt/.config/joj/students/p1/case1.in"
                },
                "cpuLimit": 1000000000,
                "clockLimit": 2000000000,
                "memoryLimit": 268435456
            },
            {
                "stdin": {
                    "src": "/home/tt/.config/joj/students/p1/case2.in"
                },
                "cpuLimit": 1000000000,
                "clockLimit": 2000000000,
                "memoryLimit": 268435456
            },
            {
                "stdin": {
                    "src": "/home/tt/.config/joj/students/p1/case3.in"
                },
                "cpuLimit": 1000000000,
                "clockLimit": 2000000000,
                "memoryLimit": 268435456
            }
        ]
    }
},

...

The problem is that with this field, input of all test cases will become the input of the last test case.

Problem 2

  • fix setting default value of fields for diff parser.

Details:

in .toml:

[[stages]]
name = "[joj] p1 basic cases"
command="./students/p1/compile/build/pretest"
files.import = [ "p1/build/pretest" ]
limit.cpu = "3s"
limit.mem = "10m"
limit.stdout = "10m"
limit.stderr = "10m"

diff.force_quit = false
diff.output.hide = false
diff.output.ignore_spaces = true
diff.score = 15

result-detail.exitstatus = true
result-detail.mem = true
result-detail.stderr = true
result-detail.stdout = true
result-detail.time = true

parsers = [ "diff", "result-detail" ]

case0.score = 15
case1.score = 15
case2.score = 15
case3.score = 15

in .json:

"default": {
    "args": [
        "./students/p1/compile/build/pretest"
    ],
    "env": [
        "PATH=/usr/bin:/bin:/usr/local/bin"
    ],
    "stdin": {
        "content": ""
    },
    "stdout": {
        "name": "stdout",
        "max": 2097152
    },
    "stderr": {
        "name": "stderr",
        "max": 2097152
    },
    "cpuLimit": 2000000000,
    "clockLimit": 4000000000,
    "memoryLimit": 2097152,
    "stackLimit": 0,
    "procLimit": 50,
    "cpuRateLimit": 0,
    "cpuSetLimit": "",
    "copyIn": {
        "p1/build/pretest": {
            "src": "/home/tt/.config/joj/p1/build/pretest"
        }
    },
    "copyInCached": {
        "students/p1/compile/build/pretest": "students/p1/compile/build/pretest",
        "students/p1/compile/build/compile_commands.json": "students/p1/compile/build/compile_commands.json",
        "students/p1/compile/p1.cpp": "students/p1/compile/p1.cpp"
    },
    "copyInDir": ".",
    "copyOut": [
        "stdout",
        "stderr"
    ],
    "copyOutCached": [],
    "copyOutMax": 0,
    "copyOutDir": "",
    "tty": false,
    "strictMemoryLimit": false,
    "dataSegmentLimit": false,
    "addressSpaceLimit": false
},
"cases": [
    {
        "stdin": {
            "src": "/home/tt/.config/joj/students/p1/case0.in"
        },
        "cpuLimit": 1000000000,
        "clockLimit": 2000000000,
        "memoryLimit": 268435456
    },
    {
        "stdin": {
            "src": "/home/tt/.config/joj/students/p1/case1.in"
        },
        "cpuLimit": 1000000000,
        "clockLimit": 2000000000,
        "memoryLimit": 268435456
    },
    {
        "stdin": {
            "src": "/home/tt/.config/joj/students/p1/case2.in"
        },
        "cpuLimit": 1000000000,
        "clockLimit": 2000000000,
        "memoryLimit": 268435456
    },
    {
        "stdin": {
            "src": "/home/tt/.config/joj/students/p1/case3.in"
        },
        "cpuLimit": 1000000000,
        "clockLimit": 2000000000,
        "memoryLimit": 268435456
    }
]

The default value is set for all test cases in .toml. e.g. limit.cpu = "3s" but used neither in default nor in each case.

# Problem 1: - [x] delete default `stdin` field for `diff` parser. <details> ``` "stdin": { "content": "" }, ``` </details> This part should be delete when generating `.json`. The location of this is in: <details> ``` ... "executor": { "name": "sandbox", "with": { "default": { "args": [ "./students/p1/compile/build/pretest" ], "env": [ "PATH=/usr/bin:/bin:/usr/local/bin" ], "stdin": { "content": "" }, "stdout": { "name": "stdout", "max": 2097152 }, "stderr": { "name": "stderr", "max": 2097152 }, "cpuLimit": 2000000000, "clockLimit": 4000000000, "memoryLimit": 2097152, "stackLimit": 0, "procLimit": 50, "cpuRateLimit": 0, "cpuSetLimit": "", "copyIn": { "p1/build/pretest": { "src": "/home/tt/.config/joj/p1/build/pretest" } }, "copyInCached": { "students/p1/compile/build/pretest": "students/p1/compile/build/pretest", "students/p1/compile/build/compile_commands.json": "students/p1/compile/build/compile_commands.json", "students/p1/compile/p1.cpp": "students/p1/compile/p1.cpp" }, "copyInDir": ".", "copyOut": [ "stdout", "stderr" ], "copyOutCached": [], "copyOutMax": 0, "copyOutDir": "", "tty": false, "strictMemoryLimit": false, "dataSegmentLimit": false, "addressSpaceLimit": false }, "cases": [ { "stdin": { "src": "/home/tt/.config/joj/students/p1/case0.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 }, { "stdin": { "src": "/home/tt/.config/joj/students/p1/case1.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 }, { "stdin": { "src": "/home/tt/.config/joj/students/p1/case2.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 }, { "stdin": { "src": "/home/tt/.config/joj/students/p1/case3.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 } ] } }, ... ``` </details> The problem is that with this field, input of all test cases will become the input of the last test case. # Problem 2 - [x] fix setting default value of fields for `diff` parser. Details: in `.toml`: <details> ``` [[stages]] name = "[joj] p1 basic cases" command="./students/p1/compile/build/pretest" files.import = [ "p1/build/pretest" ] limit.cpu = "3s" limit.mem = "10m" limit.stdout = "10m" limit.stderr = "10m" diff.force_quit = false diff.output.hide = false diff.output.ignore_spaces = true diff.score = 15 result-detail.exitstatus = true result-detail.mem = true result-detail.stderr = true result-detail.stdout = true result-detail.time = true parsers = [ "diff", "result-detail" ] case0.score = 15 case1.score = 15 case2.score = 15 case3.score = 15 ``` in `.json`: ``` "default": { "args": [ "./students/p1/compile/build/pretest" ], "env": [ "PATH=/usr/bin:/bin:/usr/local/bin" ], "stdin": { "content": "" }, "stdout": { "name": "stdout", "max": 2097152 }, "stderr": { "name": "stderr", "max": 2097152 }, "cpuLimit": 2000000000, "clockLimit": 4000000000, "memoryLimit": 2097152, "stackLimit": 0, "procLimit": 50, "cpuRateLimit": 0, "cpuSetLimit": "", "copyIn": { "p1/build/pretest": { "src": "/home/tt/.config/joj/p1/build/pretest" } }, "copyInCached": { "students/p1/compile/build/pretest": "students/p1/compile/build/pretest", "students/p1/compile/build/compile_commands.json": "students/p1/compile/build/compile_commands.json", "students/p1/compile/p1.cpp": "students/p1/compile/p1.cpp" }, "copyInDir": ".", "copyOut": [ "stdout", "stderr" ], "copyOutCached": [], "copyOutMax": 0, "copyOutDir": "", "tty": false, "strictMemoryLimit": false, "dataSegmentLimit": false, "addressSpaceLimit": false }, "cases": [ { "stdin": { "src": "/home/tt/.config/joj/students/p1/case0.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 }, { "stdin": { "src": "/home/tt/.config/joj/students/p1/case1.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 }, { "stdin": { "src": "/home/tt/.config/joj/students/p1/case2.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 }, { "stdin": { "src": "/home/tt/.config/joj/students/p1/case3.in" }, "cpuLimit": 1000000000, "clockLimit": 2000000000, "memoryLimit": 268435456 } ] ``` </details> The default value is set for all test cases in `.toml`. e.g. `limit.cpu = "3s"` but used neither in default nor in each case.
姚云翔523370910110 added the
bug
 label 2025-05-22 11:19:16 +08:00
李衍志523370910113 was assigned by 姚云翔523370910110 2025-05-22 11:20:59 +08:00
manuel was assigned by 李衍志523370910113 2025-05-22 11:24:55 +08:00
姚云翔523370910110 was assigned by 李衍志523370910113 2025-05-22 11:24:55 +08:00
李衍志523370910113 commented 2025-05-22 11:26:46 +08:00
Member
Copy Link

@egghead_yao @manuel we will discuss some essential requirements and adjustments here. Currently:

  • add default score field for diff parsers
  • trim off every default coded numbers in the toml file while testing
@egghead_yao @manuel we will discuss some essential requirements and adjustments here. Currently: - [x] add default score field for diff parsers - trim off every default coded numbers in the `toml` file while testing
李衍志523370910113 added the
enhancement
 label 2025-05-22 11:27:14 +08:00
李衍志523370910113 commented 2025-05-22 15:02:36 +08:00
Member
Copy Link

fixed and closing

fixed and closing
张泊明518370910136 commented 2025-05-22 15:44:16 +08:00
Owner
Copy Link

:-( problem 1 is caused by joj3, fixing

:-( problem 1 is caused by joj3, fixing
张泊明518370910136 commented 2025-05-22 16:02:05 +08:00
Owner
Copy Link

Could you check if problem 1 persists in the latest JOJ3 (after runner-images is built)?

Could you check if problem 1 persists in the latest JOJ3 (after runner-images is built)?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
李衍志523370910113
manuel
姚云翔523370910110
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: JOJ/JOJ3-config-generator#15
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?