chvp/phd-thesis
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix some typos

Browse source
This commit is contained in:
Charlotte Van Petegem 2024-04-26 13:51:47 +02:00
parent 7a084fadae
commit 9a4377bcda
No known key found for this signature in database
GPG key ID: 019E764B7184435A
4 changed files with 20 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

18
book.org
View file

@ -2981,7 +2981,7 @@ In addition, [cite/t:@berniusMachineLearningBased2022] introduced a framework ba
The context of our work is the Dodona learning environment, developed at Ghent University\nbsp{}[cite:@vanpetegemDodonaLearnCode2023]. The context of our work is the Dodona learning environment, developed at Ghent University\nbsp{}[cite:@vanpetegemDodonaLearnCode2023].
Dodona gives automated feedback on each submitted solution to programming exercises, but also has a module that allows teachers to give manual feedback on student submissions and assign scores. Dodona gives automated feedback on each submitted solution to programming exercises, but also has a module that allows teachers to give manual feedback on student submissions and assign scores.
The process of giving manual feedback on a submission to a programming exercise in Dodona is very similar to a code review, where errors or suggestions for improvements are annotated on the relevant line(s)\nbsp{}(Figure\nbsp{}[[fig:feedbackintroductionreview]]). The process of giving manual feedback on a submission to a programming exercise in Dodona is very similar to a code review, where errors or suggestions for improvements are annotated on the relevant line(s)\nbsp{}(Figure\nbsp{}[[fig:feedbackintroductionreview]]).
n 2023 alone, 3\thinsp{}663\thinsp{}749 solutions were submitted to Dodona, of which 44\thinsp{}012 were manually assessed. In 2023 alone, 3\thinsp{}663\thinsp{}749 solutions were submitted to Dodona, of which 44\thinsp{}012 were manually assessed.
During manual assessment, 22\thinsp{}888 annotations were added to specific lines of code. During manual assessment, 22\thinsp{}888 annotations were added to specific lines of code.
#+CAPTION: Assessment of a submitted solution in Dodona. #+CAPTION: Assessment of a submitted solution in Dodona.
@ -3023,7 +3023,7 @@ Program code can be represented as an abstract syntax tree (AST), where the node
Recent work has used demonstrated the efficacy of this approach in efficiently identifying frequent patterns in source code\nbsp{}[cite:@phamMiningPatternsSource2019]. Recent work has used demonstrated the efficacy of this approach in efficiently identifying frequent patterns in source code\nbsp{}[cite:@phamMiningPatternsSource2019].
In an educational context, these techniques have already been used to find patterns common to solutions that failed a given exercise\nbsp{}[cite:@mensGoodBadUgly2021]. In an educational context, these techniques have already been used to find patterns common to solutions that failed a given exercise\nbsp{}[cite:@mensGoodBadUgly2021].
Other work has demonstrated the potential of automatically generating unit tests from mined patterns\nbsp{}[cite:@lienard2023extracting]. Other work has demonstrated the potential of automatically generating unit tests from mined patterns\nbsp{}[cite:@lienard2023extracting].
We use tree minig to find commonalities between the lines of code where the same annotation has been added. We use tree mining to find commonalities between the lines of code where the same annotation has been added.
We begin with a general overview of ECHO (Figure\nbsp{}[[fig:feedbackmethodoverview]]). We begin with a general overview of ECHO (Figure\nbsp{}[[fig:feedbackmethodoverview]]).
The first step is to use the tree-sitter library\nbsp{}[cite:@brunsfeldTreesitterTreesitterV02024] to generate ASTs for each submission. The first step is to use the tree-sitter library\nbsp{}[cite:@brunsfeldTreesitterTreesitterV02024] to generate ASTs for each submission.
@ -3105,7 +3105,7 @@ Due to the exponential nature of the number of possible patterns in a tree, we o
ECHO now has a set of patterns corresponding to each annotation. ECHO now has a set of patterns corresponding to each annotation.
However, some patterns are more informative that others. However, some patterns are more informative that others.
So it assigns weights to the patterns it gets from =TreeminerD= So it assigns weights to the patterns it gets from =TreeminerD=.
Weights are assigned using two criteria. Weights are assigned using two criteria.
The first criterion is the size of the pattern (i.e., the number of nodes in the pattern), since a pattern with twenty nodes is much more specific than a pattern with only one node. The first criterion is the size of the pattern (i.e., the number of nodes in the pattern), since a pattern with twenty nodes is much more specific than a pattern with only one node.
@ -3222,7 +3222,7 @@ We test per exercise because that's our main use case for this method, but we al
An overview of some annotation statistics for the data generated by Pylint can be found in Table\nbsp{}[[tab:feedbackresultsdatasetpylint]]. An overview of some annotation statistics for the data generated by Pylint can be found in Table\nbsp{}[[tab:feedbackresultsdatasetpylint]].
#+CAPTION: Statistics of Pylint annotations for the programming exercises used in the benchmark. #+CAPTION: Statistics of Pylint annotations for the programming exercises used in the benchmark.
#+NAME: tab:feedbackresultsdataset #+NAME: tab:feedbackresultsdatasetpylint
| Exercise | subm. | ann. | inst. | max | avg | | Exercise | subm. | ann. | inst. | max | avg |
|-----------------------+-------+------+-------+-----+-------| |-----------------------+-------+------+-------+-----+-------|
| <l> | <r> | <r> | <r> | <r> | <r> | | <l> | <r> | <r> | <r> | <r> | <r> |
@ -3323,7 +3323,7 @@ The number of instances where the true annotation is ranked first is generally h
[[./images/feedbackpredictionrealworldglobal.png]] [[./images/feedbackpredictionrealworldglobal.png]]
In this experiment, training took between 67 milliseconds and 22.4 seconds depending on the exercise. In this experiment, training took between 67 milliseconds and 22.4 seconds depending on the exercise.
The entire testi phase took between 49 milliseconds and 27 seconds, depending on the exercise. The entire test phase took between 49 milliseconds and 27 seconds, depending on the exercise.
These evaluations were run on the same hardware as those for the machine annotations. These evaluations were run on the same hardware as those for the machine annotations.
For one prediction, the average time ranged from 0.1 milliseconds to 150 milliseconds and the maxima from 0.5 milliseconds to 2.8 seconds. For one prediction, the average time ranged from 0.1 milliseconds to 150 milliseconds and the maxima from 0.5 milliseconds to 2.8 seconds.
The explanation for these wide ranges remains the same as for the Pylint predictions: it all depends on the number of patterns found. The explanation for these wide ranges remains the same as for the Pylint predictions: it all depends on the number of patterns found.
@ -3347,28 +3347,28 @@ This means that a lot of time could be saved during the review process when ECHO
The point at which the critical mass is reached will of course depend on the nature of the exercises and the consistency of the reviewer. The point at which the critical mass is reached will of course depend on the nature of the exercises and the consistency of the reviewer.
#+CAPTION: Progression of the prediction accuracy for the "A last goodbye" exercise over the course of the review process. #+CAPTION: Progression of the prediction accuracy for the "A last goodbye" exercise over the course of the review process.
#+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances. #+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances".
#+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns". #+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns".
#+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set. #+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set.
#+NAME: fig:feedbackpredictionrealworldsimulation1 #+NAME: fig:feedbackpredictionrealworldsimulation1
[[./images/feedbackpredictionrealworldsimulation1.png]] [[./images/feedbackpredictionrealworldsimulation1.png]]
#+CAPTION: Progression of the prediction accuracy for the "Narcissus cipher" exercise over the course of the review process. #+CAPTION: Progression of the prediction accuracy for the "Narcissus cipher" exercise over the course of the review process.
#+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances. #+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances".
#+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns". #+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns".
#+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set. #+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set.
#+NAME: fig:feedbackpredictionrealworldsimulation2 #+NAME: fig:feedbackpredictionrealworldsimulation2
[[./images/feedbackpredictionrealworldsimulation2.png]] [[./images/feedbackpredictionrealworldsimulation2.png]]
#+CAPTION: Progression of the prediction accuracy for the "Cocktail bar" exercise over the course of the review process. #+CAPTION: Progression of the prediction accuracy for the "Cocktail bar" exercise over the course of the review process.
#+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances. #+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances".
#+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns". #+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns".
#+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set. #+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set.
#+NAME: fig:feedbackpredictionrealworldsimulation3 #+NAME: fig:feedbackpredictionrealworldsimulation3
[[./images/feedbackpredictionrealworldsimulation3.png]] [[./images/feedbackpredictionrealworldsimulation3.png]]
#+CAPTION: Progression of the prediction accuracy for the "Anthropomorphic emoji" exercise over the course of the review process. #+CAPTION: Progression of the prediction accuracy for the "Anthropomorphic emoji" exercise over the course of the review process.
#+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances. #+CAPTION: Predictions for instances whose annotation had no instances in the training set are classified as "No training instances".
#+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns". #+CAPTION: Predictions for instances whose annotation had no corresponding patterns in the model learned from the training set are classified as "No patterns".
#+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set. #+CAPTION: The graph on the right shows the number of annotations present with at least one instance in the training set.
#+NAME: fig:feedbackpredictionrealworldsimulation4 #+NAME: fig:feedbackpredictionrealworldsimulation4

4
diagrams/feedbackmethodoverview.svg
View file

@ -10,7 +10,7 @@
xml:space="preserve" xml:space="preserve"
inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)" inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)"
sodipodi:docname="feedbackmethodoverview.svg" sodipodi:docname="feedbackmethodoverview.svg"
inkscape:export-filename="../../../../downloads/feedbackmethodoverview.png" inkscape:export-filename="../../../dodona/feedback-prediction-article-tex/feedbackmethodoverview.png"
inkscape:export-xdpi="96" inkscape:export-xdpi="96"
inkscape:export-ydpi="96" inkscape:export-ydpi="96"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
@ -29,7 +29,7 @@
inkscape:zoom="1.8827923" inkscape:zoom="1.8827923"
inkscape:cx="270.87428" inkscape:cx="270.87428"
inkscape:cy="93.212618" inkscape:cy="93.212618"
inkscape:window-width="2302" inkscape:window-width="1534"
inkscape:window-height="2132" inkscape:window-height="2132"
inkscape:window-x="11" inkscape:window-x="11"
inkscape:window-y="11" inkscape:window-y="11"

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Before After
Before After

8
diagrams/feedbackpattern.svg
View file

@ -10,7 +10,7 @@
id="svg12" id="svg12"
sodipodi:docname="feedbackpattern.svg" sodipodi:docname="feedbackpattern.svg"
inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)" inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)"
inkscape:export-filename="../../../../downloads/feedbackpattern.png" inkscape:export-filename="../../../dodona/feedback-prediction-article-tex/feedbackpattern.png"
inkscape:export-xdpi="96" inkscape:export-xdpi="96"
inkscape:export-ydpi="96" inkscape:export-ydpi="96"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
@ -29,10 +29,10 @@
inkscape:pagecheckerboard="0" inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1" inkscape:deskcolor="#d1d1d1"
inkscape:zoom="3.1395349" inkscape:zoom="3.1395349"
inkscape:cx="135.21111" inkscape:cx="135.37037"
inkscape:cy="152.72963" inkscape:cy="152.72963"
inkscape:window-width="2558" inkscape:window-width="1534"
inkscape:window-height="1412" inkscape:window-height="1064"
inkscape:window-x="11" inkscape:window-x="11"
inkscape:window-y="11" inkscape:window-y="11"
inkscape:window-maximized="0" inkscape:window-maximized="0"

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 38 KiB

Before After
Before After

7
diagrams/feedbacksubtree.svg
View file

@ -10,6 +10,9 @@
id="svg12" id="svg12"
sodipodi:docname="feedbacksubtree.svg" sodipodi:docname="feedbacksubtree.svg"
inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)" inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)"
inkscape:export-filename="../../../dodona/feedback-prediction-article-tex/feedbacksubtree.png"
inkscape:export-xdpi="96"
inkscape:export-ydpi="96"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xlink="http://www.w3.org/1999/xlink"
@ -26,10 +29,10 @@
inkscape:pagecheckerboard="0" inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1" inkscape:deskcolor="#d1d1d1"
inkscape:zoom="2.2199864" inkscape:zoom="2.2199864"
inkscape:cx="142.56844" inkscape:cx="142.34321"
inkscape:cy="138.06391" inkscape:cy="138.06391"
inkscape:window-width="1534" inkscape:window-width="1534"
inkscape:window-height="1064" inkscape:window-height="708"
inkscape:window-x="11" inkscape:window-x="11"
inkscape:window-y="11" inkscape:window-y="11"
inkscape:window-maximized="0" inkscape:window-maximized="0"

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 54 KiB

After

Width:  |  Height:  |  Size: 54 KiB

Before After
Before After