scoreboard and failed table refactoring and redesign #68

New Issue

manuel · 2024-10-18T17:05:09+08:00

manuel commented

2024-10-18 17:05:09 +08:00

goal: share ideas on howto improve information we store.

At the moment we have

scoreboard (jaccount,last call to joj,total, ex scores)
failed table (who failed at what stage)

requirements

discuss what to store
how to display info (files, format)

**goal:** share ideas on howto improve information we store. At the moment we have - scoreboard (jaccount,last call to joj,total, ex scores) - failed table (who failed at what stage) requirements - [ ] discuss what to store - [ ] how to display info (files, format)

manuel added the

labels 2024-10-18 17:05:23 +08:00

manuel commented

2024-10-18 17:45:54 +08:00

scoreboard:

we should keep csv format for easy processing and grade calculation (no switch to .md)
in 23fa we had codequality grade which was separated from joj grade, ie 2 scoreboards. currently codequality is hard to know (need to check issues)
fields to add;
- repo name (useful for team works)
- student id and/or name (name only is bad as more than 1 student can have the same name)

failed table: current format is roughly similar to 23fa. we probably want to take advantage of md format to log more info (eg. easy links).

general ideas to get started: create a "feature-rich" version of the csv scoreboard, eg.

last update field can be clicked and send use to all issues with the pattern of the assignment in the title (eg. h3)
score field sends us to the issue of that run
score field is replaced by x on failure (link to the issue describing the problem)
clicking repo name sends to repo
clicking a name/jaccount/id sends us contribution tab?
record: number of "joj" commits, number non joj commits, number of failed actions
record memory/cpu usage

please share your thoughts and ideas. the more suggestions the better :-)

then we can decise of the direction we want to go. even if something feels hard to implement please suggest it. then we might simply ignore it if not worth the effort...

scoreboard: - we should keep csv format for easy processing and grade calculation (no switch to .md) - in 23fa we had codequality grade which was separated from joj grade, ie 2 scoreboards. currently codequality is hard to know (need to check issues) - fields to add; - repo name (useful for team works) - student id and/or name (name only is bad as more than 1 student can have the same name) failed table: current format is roughly similar to 23fa. we probably want to take advantage of md format to log more info (eg. easy links). general ideas to get started: create a "feature-rich" version of the csv scoreboard, eg. - `last update` field can be clicked and send use to all issues with the pattern of the assignment in the title (eg. `h3`) - score field sends us to the issue of that run - score field is replaced by `x` on failure (link to the issue describing the problem) - clicking repo name sends to repo - clicking a name/jaccount/id sends us contribution tab? - record: number of "joj" commits, number non joj commits, number of failed actions - record memory/cpu usage please share your thoughts and ideas. the more suggestions the better :-) then we can decise of the direction we want to go. even if something feels hard to implement please suggest it. then we might simply ignore it if not worth the effort...

manuel self-assigned this 2024-10-18 17:46:09 +08:00

王韵晨520370910012 was assigned by manuel

2024-10-18 17:46:09 +08:00

张泊明518370910136 was assigned by manuel

2024-10-18 17:46:09 +08:00

张佳澈520370910044 was assigned by manuel

2024-10-18 17:46:09 +08:00

樊络一521370910049 was assigned by manuel

2024-10-18 17:46:09 +08:00

周赵嘉程521432910016 was assigned by manuel

2024-10-18 17:46:09 +08:00

汪睿522370910169 was assigned by manuel

2024-10-18 17:46:09 +08:00

李衍志523370910113 was assigned by manuel

2024-10-18 17:46:09 +08:00

张泊明518370910136 commented

2024-10-19 04:27:09 +08:00

The architecture of teapot runner needs to change a little since scoreboard, failed table, and issue now run independently and concurrently.

Or we can let teapot run all 3 things in one command, then they can rely on each other.

The architecture of teapot runner needs to change a little since scoreboard, failed table, and issue now run independently and concurrently. Or we can let teapot run all 3 things in one command, then they can rely on each other.

manuel commented

2024-10-19 07:43:04 +08:00

that's why we need this issue: we have to first clarify where we want to go (what features to implement) before we start implementing anythng. this is especially important as it might involve "deep" code refactoring for teapot...

@all as TAs and students this is important that you provide feedback and ideas on what you need to grades others but also on what you would like to be graded on :-)

2 main questions:

we need csv scoreboard joj and cq. any other cvs info you would like to see?
what meta info would make TT life easier (link to jump to issues, code, failures, etc.)?

when we know what to include we can think of the formatting, then will come the implementation

please try to all comment on this issue

that's why we need this issue: we have to first clarify where we want to go (what features to implement) before we start implementing anythng. this is especially important as it might involve "deep" code refactoring for teapot... @all as TAs and students this is important that you provide feedback and ideas on what you need to grades others but also on what you would like to be graded on :-) 2 main questions: - we need csv scoreboard joj and cq. any other cvs info you would like to see? - what meta info would make TT life easier (link to jump to issues, code, failures, etc.)? when we know what to include we can think of the formatting, then will come the implementation please try to all comment on this issue

张泊明518370910136 commented

2025-05-28 08:41:35 +08:00

Do we still need to redesign now?