chvp/phd-thesis
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Edit pass/fail not to be anonymized

Browse source
This commit is contained in:
Charlotte Van Petegem 2024-01-31 16:08:17 +01:00
parent fb6c6c12f5
commit 1d6fbb1490
No known key found for this signature in database
GPG key ID: 019E764B7184435A
1 changed files with 5 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

19
book.org
View file

@ -124,16 +124,6 @@ I might even wait with this explicitly to do this closer to the deadline, to inc
:CREATED: [2023-11-21 Tue 16:15]
:END:
** Low priority
:PROPERTIES:
:CREATED: [2023-11-20 Mon 17:17]
:END:
*** TODO Edit pass/fail to not be anonymized
:PROPERTIES:
:CREATED: [2023-11-20 Mon 17:18]
:END:
#+LATEX: \begin{dutch}
* Dankwoord
:PROPERTIES:
@ -1672,7 +1662,7 @@ Course B has snapshots for the first ten lab sessions (labelled S1--S10), a snap
It is important to stress that a snapshot of a course edition measures student performance only using the information available at the time of the snapshot.
As a result, the snapshot does not take into account submissions after its timestamp.
The behaviour of a student can then be expressed as a set of features extracted from the raw submission data.
We identified different types of features (see Appendix\nbsp{}[[Feature types]]) that indirectly quantify certain behavioural aspects of students practising their programming skills.
We identified different types of features (see Appendix\nbsp{}[[#chap:featuretypes]]) that indirectly quantify certain behavioural aspects of students practising their programming skills.
When and how long do students work on their exercises?
Can students correctly solve an exercise and how much feedback do they need to accomplish this?
What kinds of mistakes do students make while solving programming exercises?
@ -1690,7 +1680,7 @@ These features of the snapshot can be used to predict whether a student will fin
In addition, the snapshot also contains a label indicating whether the student passed or failed that is used during training and testing of classification algorithms.
Students that did not take part in the final examination, automatically fail the course.
Since course B has no hard deadlines, we left out deadline-related features from its snapshots (=first_dl=, =last_dl= and =nr_dl=; see Appendix\nbsp{}[[Feature types]]).
Since course B has no hard deadlines, we left out deadline-related features from its snapshots (=first_dl=, =last_dl= and =nr_dl=; see Appendix\nbsp{}[[#chap:featuretypes]]).
To investigate the impact of deadline-related features, we also made predictions for course A that ignore these features.
*** Classification algorithms
@ -1730,6 +1720,7 @@ Many studies for pass/fail prediction use accuracy (\((TP+TN)/(TP+TN+FP+FN)\)) a
However, this can yield misleading results.
For example, let's take a dummy classifier that always "predicts" students will pass, no matter what.
This is clearly a bad classifier, but it will nonetheless have an accuracy of 75% for a course where 75% of the students pass.
In our study, we will therefore use two more complex metrics that take these effects into account: balanced accuracy and F_1-score.
Balanced accuracy is the average of sensitivity and specificity.
The F_1-score is the harmonic mean of precision and recall.
@ -1872,7 +1863,7 @@ Features with a positive importance (red colour) will increase the odds with inc
To simulate that we want to make predictions for each course edition included in this study, we trained logistic regression models with data from the remaining two editions of the same course.
A label "edition 18--19" therefore means that we want to make predictions for the 2018--2019 edition of a course with a model built from the 2016--2017 and 2017--2018 editions of the course.
However, in this case we are not interested in the predictions themselves, but in the importance of the features in the models.
The importance of all features for each course edition can be found in Appendix\nbsp{}[[Feature importances]].
The importance of all features for each course edition can be found at https://github.com/dodona-edu/pass-fail-article.
We will restrict our discussion by highlighting the importance of a selection of feature types for the two courses.
For course A, we look into the evaluation scores (Figure\nbsp{}[[fig:passfailfeaturesAevaluation]]) and the feature types =correct_after_15m= (Figure\nbsp{}[[fig:passfailfeaturesAcorrect]]) and =wrong= (Figure\nbsp{}[[fig:passfailfeaturesAwrong]]).
@ -2317,7 +2308,7 @@ for digit in number:
:END:
#+LATEX: \appendix
* Feature types
* Pass/fail prediction feature types
:PROPERTIES:
:CREATED: [2023-10-23 Mon 18:09]
:CUSTOM_ID: chap:featuretypes