fix: submit_problem saving history before submission guards - #39015
fix: submit_problem saving history before submission guards#39015shadinaif wants to merge 1 commit into
Conversation
|
Thanks for the pull request, @shadinaif! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. 🔘 Update the status of your PRYour PR is currently marked as a draft. After completing the steps above, update its status by clicking "Ready for Review", or removing "WIP" from the title, as appropriate. Where can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
0410bd4 to
8875ffc
Compare
8875ffc to
01d51e2
Compare
WIP: related work in xblocks-contrib is needed
Description
_BuiltInProblemBlock.submit_problem(akaProblemBlock.submit_problem) records the submitted answer intostudent_answers_historybefore the submission guards run, so rejected submissions are still recorded in history, whileattemptscorrectly stays put. Each of those submissions also saves the block, writing a fullcourseware_studentmodulehistoryrow that carries the whole list, so the history table grows exponentially when too many rejected submissions are posted for the same problem.Only the guards that
returnare affected: the xqueue waittime,submission_wait_seconds, and theStudentInputError/ResponseError/LoncapaProblemErrorbranch.closed()and the unreset guard raise, sonothing was ever persisted through them.
Fix. Move the append inside the
try, immediately aftergrade_answers()returns. Only graded submissionsare recorded, the two lists stay in step, and the two
returnguards now leave the request with no dirty fieldsat all, and no
StudentModulewrite and no history row.Operators benefit; learners see no change, since legitimate submissions are recorded exactly as before. No UI change, no configuration change, no migration.
Supporting information
The defect was introduced in this PR (see
xmodule/capa_block.pyline1755), and there is no explanation why the history append self.student_answers_history.append(answers_without_files)` was made way before the other update statementsIf there is a legit reason for the location of the append statement; then we must find another code fix for the bug. It is a bug to save history of rejected submissions especially when no
reject reasonis saved anywhereReal Example
This is an observation from a production instance. An exam that was configured to require two hours between attempts was targeted by bots re-submitting every five seconds. That produced 1400+ history rows for submissions that were never graded, inflating both
courseware_studentmoduleandcourseware_studentmodulehistoryfor no reason. There is no reason to retain those submissions!On the other hand, detecting the abuse is a separate concern and is unaffected by this change: the submissions still show up in the tracking logs whether or not they are kept as learner state.
Testing instructions
Automated
pytest xmodule/tests/test_capa_block.py -k "answer_history or histories_stay_aligned"pytest xmodule/tests/test_delay_between_attempts.pyManual: submissions rejected by the wait timer
student_answers_historythan the last.student_answers_historyholds one entry per graded attempt.
Manual: submissions that fail to grade
numericalresponseproblem.abc) and confirm the error message is shown.student_answers_history.Regression: unchanged before and after
attemptsstill increments exactly once per graded submission.Deadline
None.
Other information
Back-port for older version is possible, but with a minor change on a patched function name in tests