Handle most of Peter's feebdack
This commit is contained in:
Charlotte Van Petegem
parent
cd4ef0bd89
commit
7a9a8234b3
2 changed files with 156 additions and 118 deletions
159
book.org
159
book.org
|
|
@ -2,7 +2,7 @@
|
|||
#+AUTHOR: Charlotte Van Petegem
|
||||
#+LANGUAGE: en-gb
|
||||
#+LATEX_CLASS: book
|
||||
#+LATEX_CLASS_OPTIONS: [paper=240mm:170mm,parskip,DIV=11,BCOR=15mm]
|
||||
#+LATEX_CLASS_OPTIONS: [paper=240mm:170mm,parskip,DIV=10,BCOR=10mm]
|
||||
#+LATEX_COMPILER: lualatex
|
||||
#+LATEX_HEADER: \usepackage[inline]{enumitem}
|
||||
#+LATEX_HEADER: \usepackage{shellesc, luacode}
|
||||
|
|
@ -115,6 +115,11 @@ Make sure to mention that the content is based on certain articles (if applicabl
|
|||
:CREATED: [2023-11-20 Mon 17:23]
|
||||
:END:
|
||||
|
||||
*** TODO Update statistics in [[#chap:use]]
|
||||
:PROPERTIES:
|
||||
:CREATED: [2024-01-22 Mon 12:51]
|
||||
:END:
|
||||
|
||||
*** TODO Redo screenshots/visualizations
|
||||
:PROPERTIES:
|
||||
:CREATED: [2023-11-20 Mon 17:19]
|
||||
|
|
@ -238,7 +243,7 @@ However, learning analytics, reports and exports usually only take into account
|
|||
Because of the importance of deadlines and to avoid discussions with students about missed deadlines, series deadlines are not only announced on the course page.
|
||||
The student's home page highlights upcoming deadlines for individual courses and across all courses.
|
||||
While working on a programming assignment, students will also see a clear warning starting from ten minutes before a deadline.
|
||||
Courses also provide an iCalendar link that students can use to publish course deadlines in their personal calendar application.
|
||||
Courses also provide an iCalendar link\nbsp{}[cite:@stenersonInternetCalendaringScheduling1998] that students can use to publish course deadlines in their personal calendar application.
|
||||
|
||||
Because Dodona logs all student submissions and their metadata, including feedback and grades from automated and manual assessment, we use that data to integrate reports and learning analytics in the course page\nbsp{}[cite:@fergusonLearningAnalyticsDrivers2012].
|
||||
We also provide export wizards that enable the extraction of raw and aggregated data in CSV format for downstream processing and educational data mining\nbsp{}[cite:@romeroEducationalDataMining2010; @bakerStateEducationalData2009].
|
||||
|
|
@ -256,7 +261,7 @@ This support for *decentralized authentication* allows users to benefit from sin
|
|||
|
||||
Dodona automatically creates user accounts upon successful authentication and uses the association with external identity providers to assign an institution to users.
|
||||
By default, newly created users are assigned a student role.
|
||||
Teachers and instructors who wish to create content (courses, learning activities and judges), must first request teacher rights using a streamlined form.
|
||||
Teachers and instructors who wish to create content (courses, learning activities and judges), must first request teacher rights using a streamlined form[fn:: https://dodona.be/rights_requests/new/].
|
||||
|
||||
** Automated assessment
|
||||
:PROPERTIES:
|
||||
|
|
@ -317,7 +322,7 @@ The latter is part of the configuration of learning activities, with the option
|
|||
Dodona automatically stores metadata about all learning activities such as content type, natural language, programming language and repository to increase their findability in our large collection.
|
||||
Learning activities may also be tagged with additional labels as part of their configuration.
|
||||
|
||||
Any repository containing learning activities must have a predefined directory structure.
|
||||
Any repository containing learning activities must have a predefined directory structure[[fn:: https://docs.dodona.be/en/references/exercise-directory-structure/].
|
||||
Directories that contain a learning activity also have their own internal directory structure that includes a *description* in HTML or Markdown.
|
||||
Descriptions may reference data files and multimedia content included in the repository, and such content can be shared across all learning activities in the repository.
|
||||
Embedded images are automatically encapsulated in a responsive lightbox to improve readability.
|
||||
|
|
@ -419,7 +424,7 @@ The evaluation tracks which submissions have been manually assessed, so that ana
|
|||
|
||||
Dodona's design decisions have allowed it to spread to more than 1\thinsp{}000 schools, colleges and universities, mainly in Flanders (Belgium) and the Netherlands.
|
||||
The renewed interest in embedding computational thinking in formal education has undoubtedly been an important stimulus for such a wide uptake\nbsp{}[cite:@wingComputationalThinking2006].
|
||||
All other educational institutions use the version of Dodona hosted at Ghent University, which is free to use for educational purposes.
|
||||
All other educational institutions use the instance of Dodona hosted at Ghent University, which is free to use for educational purposes.
|
||||
|
||||
Dodona currently hosts a collection of 15 thousand learning activities that are freely available to all teachers, allowing them to create their own learning paths tailored to their teaching practice.
|
||||
In total, 61 thousand students have submitted more than 15 million solutions to Dodona in the seven years that it has been running (Figures\nbsp{}[[fig:useadoption1]] & [[fig:useadoption2]]).
|
||||
|
|
@ -458,7 +463,7 @@ Other negative feedback was mostly related to individual courses the students we
|
|||
|
||||
Since the academic year 2011--2012 we have organized an introductory Python course at Ghent University (Belgium) with a strong focus on active and online learning.
|
||||
Initially the course was offered twice a year in the first and second term, but from academic year 2014--2015 onwards it was only offered in the first term.
|
||||
The course is taken by a mix of undergraduate, graduate, and postgraduate students enrolled in various study programmes (mainly formal and natural sciences, but not computer science), with 442 students enrolled for the 2021--2022 edition.
|
||||
The course is taken by a mix of undergraduate, graduate, and postgraduate students enrolled in various study programmes (mainly formal and natural sciences, but not computer science), with 442 students enrolled for the 2021--2022 edition[fn:: https://dodona.be/courses/773].
|
||||
|
||||
**** Course structure
|
||||
:PROPERTIES:
|
||||
|
|
@ -494,7 +499,7 @@ Along the same lines, the first assignment for each topic is an ISBN-themed prog
|
|||
As soon as students feel they have enough understanding of the topic, they can start working on the five remaining mandatory assignments.
|
||||
Students can work on their programming assignments during weekly computer labs, where they can collaborate in small groups and ask help from teaching assistants.
|
||||
They can also work on their assignments and submit solutions outside lab sessions.
|
||||
In addition to the mandatory assignments, students can further elaborate on their programming skills by tackling additional programming exercises they select from a pool of over 850 exercises linked to the ten programming topics.
|
||||
In addition to the mandatory assignments, students can further elaborate on their programming skills by tackling additional programming exercises they select from a pool of over 900 exercises linked to the ten programming topics.
|
||||
Submissions for these additional exercises are not taken into account in the final grade.
|
||||
|
||||
**** Assessment, feedback and grading
|
||||
|
|
@ -607,7 +612,7 @@ The entire group of students is only addressed once about plagiarism, without go
|
|||
Every three or four years we see a persistent cluster of students exchanging code for mandatory assignments over multiple weeks.
|
||||
If this is the case, we individually address these students to point them again on their responsibilities, again differentiating between students that share their solution and students that receive solutions from others.
|
||||
|
||||
Tests and exams, on the other hand, are taken on-campus under human surveillance and allow no communication with fellow students or other persons.
|
||||
Tests and exams, on the other hand, are taken on-campus under human surveillance and allow no communication with fellow students or other persons (and more recently, also no generative AI).
|
||||
Students can work on their personal computers and get exactly two hours to solve two programming assignments during a test, and three hours and thirty minutes to solve three programming assignments during an exam.
|
||||
Tests and exams are "open book/open Internet", so any hard copy and digital resources can be consulted while solving test or exam assignments.
|
||||
Students are instructed that they can only be passive users of the Internet: all information available on the Internet at the start of a test or exam can be consulted, but no new information can be added.
|
||||
|
|
@ -636,7 +641,7 @@ After each test and exam, exercises are published and students receive manual re
|
|||
But in contrast to mandatory exercises we do not publish sample solutions for test and exam exercises, so that these exercises can be reused during the next edition of the course.
|
||||
When students ask for sample solutions of test or exam exercises, we explain that we want to give the next generation of students the same learning opportunities they had.
|
||||
|
||||
So far, we have created more than 850 programming assignments for this introductory Python course alone.
|
||||
So far, we have created more than 900 programming assignments for this introductory Python course alone.
|
||||
All these assignments are publicly shared on Dodona as open educational resources\nbsp{}[cite:@hylenOpenEducationalResources2021; @tuomiOpenEducationalResources2013; @wileyOpenEducationalResources2014; @downesModelsSustainableOpen2007; @caswellOpenEducationalResources2008].
|
||||
They are used in many other courses on Dodona (on average 10.8 courses per assignment) and by many students (on average 503.7 students and 4801.5 submitted solutions per assignment).
|
||||
We estimate that it takes about 10 person-hours on average to create a new assignment for a test or an exam: 2 hours for ideation, 30 minutes for implementing and tweaking a sample solution that meets the educational goals of the assignment and can be used to generate a test suite for automated assessment, 4 hours for describing the assignment (including background research), 30 minutes for translating the description from Dutch into English, one hour to configure support for automated assessment, and another 2 hours for reviewing the result by some extra pair of eyes.
|
||||
|
|
@ -716,7 +721,7 @@ Such "deadline hugging" patterns are also a good breeding ground for students to
|
|||
#+NAME: fig:usefweanalyticscorrect
|
||||
[[./images/usefweanalyticscorrect.png]]
|
||||
|
||||
Using educational data mining techniques on historical data exported from several editions of the course, we further investigated what aspects of practising programming skills promote or inhibit learning, or have no or minor effect on the learning process\nbsp{}[cite:@vanpetegemPassFailPrediction2022].
|
||||
Using educational data mining techniques on historical data exported from several editions of the course, we further investigated what aspects of practising programming skills promote or inhibit learning, or have no or minor effect on the learning process\nbsp{}(Chapter [[#chap:passfail]]).
|
||||
It won't come as a surprise that midterm test scores are good predictors for a student's final grade, because tests and exams are both summative assessments that are organized and graded in the same way.
|
||||
However, we found that organizing a final exam end-of-term is still a catalyst of learning, even for courses with a strong focus of active learning during weeks of educational activities.
|
||||
|
||||
|
|
@ -754,7 +759,7 @@ Given that cohort sizes are large enough, historical data from a single course e
|
|||
:CUSTOM_ID: chap:technical
|
||||
:END:
|
||||
|
||||
Dodona and all software related to it comprises a lot of code.
|
||||
Dodona and its ecosystem comprise a lot of code.
|
||||
This chapter discusses the technical background of Dodona itself\nbsp{}[cite:@vanpetegemDodonaLearnCode2023] and a stand-alone online code editor, Papyros (\url{https://papyros.dodona.be}), that was integrated into Dodona\nbsp{}[cite:@deridderPapyrosSchrijvenUitvoeren2022].
|
||||
I will also discuss two judges that I was involved with the development of.
|
||||
The R judge was written entirely by myself\nbsp{}[cite:@nustRockerversePackagesApplications2020].
|
||||
|
|
@ -773,6 +778,7 @@ In addition, a scalable pool of interchangeable worker servers are available to
|
|||
In this section, I will highlight a few of these components.
|
||||
|
||||
#+CAPTION: Diagram of all the servers involved with running and developing Dodona.
|
||||
#+CAPTION: The role of each server in the deployment is listed below its name.
|
||||
#+CAPTION: Every server also has an implicit connection with Phocus (the monitoring server), since metrics are collected on every server such as load, CPU usage, disk usage, ...
|
||||
#+CAPTION: The Pandora server is grayed out because it is not used anymore (see Section\nbsp{}[[Python Tutor]] for more info).
|
||||
#+NAME: fig:technicaldodonaservers
|
||||
|
|
@ -784,18 +790,21 @@ In this section, I will highlight a few of these components.
|
|||
:END:
|
||||
|
||||
The user-facing part of Dodona runs on the main web server, also called Dodona (see Figure\nbsp{}[[fig:technicaldodonaservers]]).
|
||||
Dodona is a Ruby-on-Rails web application, following the Rails-standard way of organizing functionality in models, views and controllers.
|
||||
Dodona is a Ruby-on-Rails web application that follows the Rails-standard way of organizing functionality in models, views and controllers.
|
||||
|
||||
The way we handle complex logic in the frontend has seen a number of changes along the years.
|
||||
These changes were mostly done because of increasing complexity and to eliminate jQuery.
|
||||
When Dodona was started, there were only a few places where JavaScript was used.
|
||||
Dodona also used the Rails-standard way of serving dynamically generated JavaScript to replace parts of pages (e.g. for pagination or search).
|
||||
With the introduction of more complex features like evaluations, we switch to using lightweight web components where this made sense.
|
||||
With the introduction of more complex features like evaluations, we switched to using lightweight web components where this made sense.
|
||||
We also eliminated jQuery, because more and more of its functionality was implemented natively by browsers.
|
||||
And lastly, all JavaScript was rewritten to TypeScript.
|
||||
|
||||
**** Security and performance
|
||||
:PROPERTIES:
|
||||
:CREATED: [2024-01-10 Wed 14:23]
|
||||
:END:
|
||||
|
||||
Another important aspect of running a public web application is its security.
|
||||
Dodona needs to operate in a challenging environment where students simultaneously submit untrusted code to be executed on its servers ("remote code execution as a service") and expect automatically generated feedback, ideally within a few seconds.
|
||||
Many design decisions are therefore aimed at maintaining and improving the reliability and security of its systems.
|
||||
Since Dodona grew from being used to teach mostly by people we knew personally to being used in secondary schools all over Flanders, we went from being able to fully trust exercise authors to having this trust reduced (as it is impossible for a team of our size to vet all the people we give teacher's rights in Dodona).
|
||||
|
|
@ -806,12 +815,12 @@ For user content where this creative freedom is not as necessary (e.g. series or
|
|||
|
||||
One of the most important components of Dodona is the feedback table.
|
||||
It has, therefore, seen a lot of security, optimization and UI work over the years.
|
||||
Since teachers can determine a lot of the content that eventually ends up in the feedback table, the same sanitization that is used for series and course descriptions is used for the messages that are added to the feedback table (since these can contain Markdown and arbitrary HTML as well).
|
||||
Since judge and exercise authors can determine a lot of the content that eventually ends up in the feedback table, the same sanitization that is used for series and course descriptions is used for the messages that are added to the feedback table (since these can contain Markdown and arbitrary HTML as well).
|
||||
The increase in teachers that added exercises to Dodona also meant that the variety in feedback given grew, sometimes resulting in a huge volume of testcases and long output.
|
||||
Optimization work was needed to cope with this volume of feedback.
|
||||
|
||||
When Dodona was first written, the library used creating diffs of the generated and expected results actually shelled out to the GNU =diff= command.
|
||||
This output was parsed and changed into HTML by the library using find and replace operations.
|
||||
Optimization work was needed to cope with this volume of feedback.
|
||||
For example, when Dodona was first written, the library used for creating diffs of the generated and expected results (=diffy=[fn:: https://github.com/samg/diffy]) actually shelled out to the GNU =diff= command.
|
||||
This output was parsed and transformed into HTML by the library using find and replace operations.
|
||||
As one can expect, starting a new process and doing a lot of string operations every time outputs had to be diffed resulted in very slow loading times for the feedback table.
|
||||
The library was replaced with a pure Ruby library (=diff-lcs=), and its outputs were built into HTML using Rails' efficient =Builder= class.
|
||||
This change of diffing method also fixed a number of bugs we were experiencing along the way.
|
||||
|
|
@ -830,7 +839,7 @@ If there are lots of small differences between a very long generated and expecte
|
|||
:CUSTOM_ID: subsec:techdodonajudging
|
||||
:END:
|
||||
|
||||
Student code is run in background jobs by our worker servers (Salmoneus, Sisyphus, Tantalus, Tityos and Ixion, as can be seen in Figure\nbsp{}[[fig:technicaldodonaservers]]).
|
||||
Student submissions are automatically assessed in background jobs by our worker servers (Salmoneus, Sisyphus, Tantalus, Tityos and Ixion; Figure\nbsp{}[[fig:technicaldodonaservers]]).
|
||||
To divide the work over these servers we make use of a job queue, based on =delayed_job=[fn:: https://github.com/collectiveidea/delayed_job].
|
||||
Each worker server has 6 job runners, which regularly poll the job queue when idle.
|
||||
|
||||
|
|
@ -854,8 +863,8 @@ The judge must be robust enough to provide feedback on all possible submissions
|
|||
Following the principles of software reuse, the judge is ideally also a generic framework that can be used to assess submissions for multiple assignments.
|
||||
This is enabled by the submission metadata that is passed when calling the judge, which includes the path to the source code of the submission, the path to the assessment resources of the assignment and other metadata such as programming language, natural language, time limit and memory limit.
|
||||
|
||||
Rather than providing a fixed set of judges, Dodona adopts a minimalistic interface that allows third parties to create new judges: automatic assessment is bootstrapped by launching the judge's run executable that can fetch the JSON formatted submission metadata from standard input and must generate JSON formatted feedback on standard output.
|
||||
The feedback has a standardized hierarchical structure that is specified in a JSON schema.
|
||||
Rather than providing a fixed set of judges, Dodona adopts a minimalistic interface that allows third parties to create new judges: automatic assessment is bootstrapped by launching the judge's =run= executable that can fetch the JSON formatted submission metadata from standard input and must generate JSON formatted feedback on standard output.
|
||||
The feedback has a standardized hierarchical structure that is specified in a JSON schema[fn:: https://github.com/dodona-edu/dodona/tree/main/public/schemas].
|
||||
At the lowest level, /tests/ are a form of structured feedback expressed as a pair of generated and expected results.
|
||||
They typically test some behaviour of the submitted code against expected behaviour.
|
||||
Tests can have a brief description and snippets of unstructured feedback called messages.
|
||||
|
|
@ -896,7 +905,7 @@ This means that the only student that can be impacted by the Python Tutor failin
|
|||
Development of Dodona is done on GitHub.
|
||||
Over the years, Dodona has seen over 16\thinsp{}000 commits by 26 contributors, and there have been 339 releases.
|
||||
All new features and bug fixes are added to the main branch through pull requests, of which there have been almost 3\thinsp{}800.
|
||||
These pull requests are reviewed by (at least) two others on the Dodona team before they are merged.
|
||||
These pull requests are reviewed by (at least) two other developers of the Dodona team before they are merged.
|
||||
We also treat pull requests as a form of documentation by writing an extensive PR description and adding screenshots for all visual changes or additions.
|
||||
The extensive test suite also runs automatically for every pull request, and developers are encouraged to add new tests for each feature or bug fix.
|
||||
We've also made it very easy to deploy to our testing (Mestra) and staging (Naos) environments so that reviewers can test changes without having to spin up their local development instance of Dodona.
|
||||
|
|
@ -906,14 +915,14 @@ There is no production data present and in fact, the database is wiped and resee
|
|||
Naos is much closer to the production setup.
|
||||
It runs on a pseudonymized version of the production database, and has all the judges configured.
|
||||
|
||||
We also make sure that our dependencies are always up-to-date using Dependabot.
|
||||
By updating our dependencies regularly, we make sure that we are not met by surprises when there is an important security update.
|
||||
Since Dodona is a website accessible over the public web, it would be a problem if we could not quickly apply a security update.
|
||||
We also make sure that our dependencies are always up-to-date using Dependabot[fn:: https://docs.github.com/en/code-security/dependabot/working-with-dependabot].
|
||||
By updating our dependencies regularly, we make sure that we are not met by incompatibilities that take a long time to integrate when there is an important security update.
|
||||
Since Dodona is accessible over the public web, it would be problematic if we could not quickly apply security updates.
|
||||
|
||||
The way we release Dodona has seen a few changes over the years.
|
||||
We've gone from a few large releases with bugfix point-releases between them, to lots of smaller releases, to in the end a /release/ per pull request.
|
||||
Since we are the only deployment of Dodona, releasing every pull request immediately after merging makes getting feedback from our users a very quick process.
|
||||
When we did versioned releases we also wrote release notes at that point.
|
||||
We've gone from a few large releases with bugfix point-releases between them, to lots of smaller releases, to now a /release/ per pull request.
|
||||
Since ours is the only instance of Dodona, releasing every pull request immediately after merging makes getting feedback from our users a very quick process.
|
||||
When we did versioned releases we also wrote release notes at the time of release.
|
||||
Because we don't have versioned releases any more, we now bundle the changes into release notes for every month.
|
||||
They are mostly autogenerated from the merged PRs, but bigger features are given more context and explanation.
|
||||
|
||||
|
|
@ -933,8 +942,8 @@ Backups of the database are automatically saved every day and kept for 12 months
|
|||
The backups are taken by dumping a replica database.
|
||||
The replica database is used because dumping the main database write-locks it while it is being dumped, which would result in Dodona being unusable for a significant amount of time.
|
||||
|
||||
We also have an extensive monitoring and alerting system in place.
|
||||
This gives us some light analytics about Dodona usage, but can also tell us if there are problems with one of our servers.
|
||||
We also have an extensive monitoring and alerting system in place, based on Grafana.
|
||||
This gives us some superficial analytics about Dodona usage, but can also tell us if there are problems with one of our servers.
|
||||
The analytics are also calculated using the replica database to avoid putting unnecessary load on our main production database.
|
||||
The web server and worker servers also send notifications when an error occurs in their runtime.
|
||||
This is one of the main ways we discover bugs that got through our tests, since our users don't regularly report bugs themselves.
|
||||
|
|
@ -959,11 +968,11 @@ Students might also be working on devices that they don't own (PCs in the school
|
|||
There are a few reasons why we could not initially offer a simple online IDE.
|
||||
Even though we can use a lot of the infrastructure very graciously offered by Ghent University, these resources are not limitless.
|
||||
The extra (interactive) evaluation of student code was something we did not have the resources for, nor did we have any architectural components in place to easily integrate this into Dodona.
|
||||
The main goal of this work was thus to provide a client-side Python execution environment we could then include in Dodona.
|
||||
The main goal of Papyros was thus to provide a client-side Python execution environment we could then include in Dodona.
|
||||
Note that we don't want to replace the entire execution model with client-side execution, as the client is an untrusted execution environment where debugging tools could be used to manipulate the results.
|
||||
Because the main idea is integration in Dodona, we primarily wanted users to be able to execute entire programs, and not necessarily offer a REPL at first.
|
||||
|
||||
Given that the target audience for this tool is secondary education students, we identified a number of secondary requirements:
|
||||
Given that the target audience for Papyros is secondary education students, we identified a number of secondary requirements:
|
||||
- The editor of our online IDE should have syntax highlighting.
|
||||
Recent literature\nbsp{}[cite:@hannebauerDoesSyntaxHighlighting2018] has shown that this does not necessarily have an impact on students' learning, but as the authors point out, it was the prevailing wisdom for a long time that it does help.
|
||||
- It should also include linting.
|
||||
|
|
@ -1120,7 +1129,7 @@ Another feature that teachers wanted that we had not built into a judge previous
|
|||
The API for the R judge was designed to follow the visual structure of the feedback table as closely as possible, as can be seen in the sample evaluation code in Listing\nbsp{}[[lst:technicalrsample]].
|
||||
Tabs are represented by different evaluation files.
|
||||
In addition to the =testEqual= function demonstrated in Listing\nbsp{}[[lst:technicalrsample]] there are some other functions to specifically support the requested functionality.
|
||||
=testImage= will set up some the R environment so that generated plots (or other images) are sent to the feedback table (in a base 64 encoded string) instead of the filesystem.
|
||||
=testImage= will set up some handlers in the R environment so that generated plots (or other images) are sent to the feedback table (in a base 64 encoded string) instead of the filesystem.
|
||||
It will also by default make the test fail if no image was generated (but does not do any verification of the image contents).
|
||||
An example of what the feedback table looks like when an image is generated can be seen in Figure\nbsp{}[[fig:technicalrplot]].
|
||||
=testDF= has some extra functionality for testing the equality of data frames, where it is possible to ignore row and column order.
|
||||
|
|
@ -1139,7 +1148,7 @@ If some code needs to be executed in the student's environment before the studen
|
|||
#+CAPTION: Sample evaluation code for a simple R exercise.
|
||||
#+CAPTION: The feedback table will contain one context with two testcases in it.
|
||||
#+CAPTION: The first testcase checks whether some t-test was performed correctly, and does this by performing two equality checks.
|
||||
#+CAPTION: The second testcase checks that the \(p\) value calculated by the t-test is correct.
|
||||
#+CAPTION: The second testcase checks that the \(p\)-value calculated by the t-test is correct.
|
||||
#+CAPTION: The =preExec= is executed in the student's environment and here fixes a random seed for the student's execution.
|
||||
#+NAME: lst:technicalrsample
|
||||
#+ATTR_LATEX: :float t
|
||||
|
|
@ -2104,49 +2113,85 @@ Graders only need to write out a detailed and clear message once and can then re
|
|||
|
||||
Given that we now have a system for re-using feedback given earlier, we can ask ourselves if we can do this in a smarter way.
|
||||
Instead of teachers having to search for the annotation they want to use, what if we could predict which annotation they want to use?
|
||||
This is exactly what we will explore in this section.
|
||||
This is exactly what we will explore in this section, which is based on an article that is currently under review.
|
||||
|
||||
The general idea of the method we explored was to find patterns in the syntax trees of submissions that received a certain annotation.
|
||||
When a teacher wants to add an annotation, we can then find the annotation that matches the best by calculating a score for each annotation's pattern set.
|
||||
To validate this method we used two testing sets that both use actual students submissions from an exam; one using messages given by PyLint and one with real-world data of saved annotations and their uses extracted from Dodona.
|
||||
*** Introduction
|
||||
:PROPERTIES:
|
||||
:CREATED: [2024-01-19 Fri 15:47]
|
||||
:END:
|
||||
|
||||
We will first give an overview of the algorithm we use to find patterns and then go over how to match these patterns given a syntax tree.
|
||||
We will also explain some practical issues that we had to consider during implementation.
|
||||
Then, we discuss what we did to rank annotations and then move on to discussing the results for the two datasets.
|
||||
Feedback is a very important element in student learning\nbsp{}[cite:@hattiePowerFeedback2007; @blackAssessmentClassroomLearning1998].
|
||||
In programming education, many steps have been taken to automate feedback using automated assessment systems\nbsp{}[cite:@paivaAutomatedAssessmentComputer2022; @ihantolaReviewRecentSystems2010; @ala-mutkaSurveyAutomatedAssessment2005].
|
||||
These automated assessment systems provide feedback on correctness, and can provide some feedback on style and best practices by using linters but are not able to provide the same high-level feedback on program design that a seasoned programmer can give.
|
||||
In many educational practices, automated assessment is therefore supplemented with manual feedback, especially when grading evaluations or exams.
|
||||
This requires a large time investment from teachers.
|
||||
|
||||
|
||||
Others have therefore tried to improve the process of giving feedback using AI.
|
||||
[cite/t:@vittoriniAIBasedSystemFormative2021] automated grading using natural language processing, and found that students who used this system during the semester were more likely to pass the course at the end of the semester.
|
||||
Others have used AI to enable students to conduct peer and self-evaluation\nbsp{}[cite:@leeSupportingStudentsGeneration2023].
|
||||
[cite/t:@berniusMachineLearningBased2022] introduced a framework based on clustering text segments to reduce the grading overhead.
|
||||
|
||||
Data mining techniques for extracting frequently occurring patterns from data that can be represented as trees were already developed in the early 2000s\nbsp{}[cite:@zakiEfficientlyMiningFrequent2005; @asaiEfficientSubstructureDiscovery2004].
|
||||
Since program code can be represented as an abstract syntax tree, more recent work looked into how these algorithms could be used to efficiently find frequent patterns in source code\nbsp{}[cite:@phamMiningPatternsSource2019].
|
||||
In an educational context, these techniques could then be used to for example find patterns common to solutions that failed a given exercise\nbsp{}[cite:@mensGoodBadUgly2021].
|
||||
Other work looked into generating unit tests from mined patterns\nbsp{}[cite:@lienard2023extracting].
|
||||
|
||||
Dodona is the automated assessment system developed at Ghent University\nbsp{}[cite:@vanpetegemDodonaLearnCode2023].
|
||||
It has a built-in module for giving manual feedback on and grading student submissions.
|
||||
The process of giving feedback on a programming assignment in Dodona is very similar to a code review, where mistakes or suggestions for improvements are annotated at the relevant line(s).
|
||||
There is however a very important difference with a traditional code review: the teacher gives feedback on many implementations of the same problem.
|
||||
Since students often make the same mistakes as other students, it follows that the same feedback is often given by a teacher on many solutions.
|
||||
In Dodona, we have already tried to anticipate this need by allowing teachers to save certain annotations, which can then later be re-used by simply searching for them.
|
||||
This gives us the data we’re using in this study: code submissions, where those submissions have been annotated on specific lines with remarks that are shared over those submissions.
|
||||
This is also the terminology we will use: an annotation is a specific instance of a remark on a line of code, a remark is the text that can be reused over multiple annotations.
|
||||
|
||||
In the following sections we present a machine learning method for suggesting re-use of previously given feedback.
|
||||
We start with an in-depth explanation of the design of the method, and then presents and discusses the experimental results we obtained when testing the method on student submissions.
|
||||
|
||||
*** Methodology
|
||||
:PROPERTIES:
|
||||
:CREATED: [2024-01-08 Mon 13:18]
|
||||
:END:
|
||||
|
||||
The general methodology used by our method is explained visually in Figure [[fig:feedbackmethodoverview]].
|
||||
We start by using tree-sitter to generate ASTs for every submission.
|
||||
For each annotation, we then extract a limited context from the AST around the line where it was placed.
|
||||
We then collect all the subtrees for each remark.
|
||||
Every remark’s forest of subtrees is given to the TreeminerD algorithm which gives us a collection of patterns for each remark.
|
||||
Each pattern is then weighted according to its length and how often it occurs in the entire collection of patterns (for all remarks).
|
||||
The result of these operations is our trained model.
|
||||
A prediction can be made when a teacher selects a line in a given student's submission.
|
||||
This is done by again extracting the limited context around that line.
|
||||
We then compute a similarity score for each remark, using its weighted patterns.
|
||||
This similarity score is used to rank the remarks and this ranking is shown to the teacher.
|
||||
We will now give a more detailed explanation of these steps.
|
||||
|
||||
#+CAPTION: Overview of our machine learning method for predicting feedback re-use.
|
||||
#+NAME: fig:feedbackmethodoverview
|
||||
[[./diagrams/feedbackmethodoverview.svg]]
|
||||
|
||||
**** Extracting a subtree around a line
|
||||
:PROPERTIES:
|
||||
:CREATED: [2024-01-19 Fri 15:44]
|
||||
:END:
|
||||
|
||||
**** TreeminerD
|
||||
:PROPERTIES:
|
||||
:CREATED: [2023-11-20 Mon 13:33]
|
||||
:END:
|
||||
|
||||
To efficiently mine forests for frequent patterns there are two main options: FREQT\nbsp{}[cite:@asaiEfficientSubstructureDiscovery2004] and Treeminer\nbsp{}[cite:@zakiEfficientlyMiningFrequent2005].
|
||||
These two algorithms are in essence the same, and were developed independently and simultaneously.
|
||||
They have been used before to mine patterns in source code\nbsp{}[cite:@phamMiningPatternsSource2019], for example to find differing patterns in code written by passing and failing students\nbsp{}[cite:@mensGoodBadUgly2021].
|
||||
In this work we opted to use the Treeminer algorithm, and more precise the TreeminerD variation on this algorithm.
|
||||
This variation gives only the distinct frequent patterns in a forest instead of all occurrences of all frequent patterns in a forest.
|
||||
This can be done much more efficiently, and in this work we don't use the extra information that the unmodified Treeminer algorithm gives us.
|
||||
|
||||
**** Matching patterns to trees
|
||||
:PROPERTIES:
|
||||
:CREATED: [2023-11-20 Mon 13:33]
|
||||
:END:
|
||||
|
||||
**** Practical considerations
|
||||
**** Assigning weights to patterns
|
||||
:PROPERTIES:
|
||||
:CREATED: [2023-11-22 Wed 14:39]
|
||||
:END:
|
||||
|
||||
**** Ranking annotations
|
||||
**** Determining similarity
|
||||
:PROPERTIES:
|
||||
:CREATED: [2023-11-22 Wed 14:47]
|
||||
:END:
|
||||
|
||||
*** Validation and results
|
||||
*** Results and discussion
|
||||
:PROPERTIES:
|
||||
:CREATED: [2024-01-08 Mon 13:18]
|
||||
:END:
|
||||
|
|
@ -2160,7 +2205,7 @@ This can be done much more efficiently, and in this work we don't use the extra
|
|||
:CREATED: [2023-11-20 Mon 13:33]
|
||||
:END:
|
||||
|
||||
*** Conclusion
|
||||
*** Conclusions and future work
|
||||
:PROPERTIES:
|
||||
:CREATED: [2023-11-20 Mon 13:33]
|
||||
:END:
|
||||
|
|
|
|||
|
|
@ -2,14 +2,17 @@
|
|||
<!-- Created with Inkscape (http://www.inkscape.org/) -->
|
||||
|
||||
<svg
|
||||
width="160mm"
|
||||
width="170mm"
|
||||
height="65mm"
|
||||
viewBox="0 0 604.72438 245.66929"
|
||||
viewBox="0 0 642.51966 245.66929"
|
||||
version="1.1"
|
||||
id="svg1"
|
||||
xml:space="preserve"
|
||||
inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)"
|
||||
sodipodi:docname="feedbackmethodoverview.svg"
|
||||
inkscape:export-filename="../../../../downloads/feedbackmethodoverview.png"
|
||||
inkscape:export-xdpi="96"
|
||||
inkscape:export-ydpi="96"
|
||||
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
|
||||
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
|
|
@ -23,16 +26,22 @@
|
|||
inkscape:pagecheckerboard="0"
|
||||
inkscape:deskcolor="#d1d1d1"
|
||||
inkscape:document-units="mm"
|
||||
inkscape:zoom="1.8827923"
|
||||
inkscape:cx="328.23588"
|
||||
inkscape:cy="23.635108"
|
||||
inkscape:window-width="2302"
|
||||
inkscape:zoom="2.6626704"
|
||||
inkscape:cx="313.21939"
|
||||
inkscape:cy="13.33248"
|
||||
inkscape:window-width="3838"
|
||||
inkscape:window-height="2104"
|
||||
inkscape:window-x="102"
|
||||
inkscape:window-y="102"
|
||||
inkscape:window-maximized="0"
|
||||
inkscape:current-layer="g1" /><defs
|
||||
inkscape:current-layer="layer1"
|
||||
inkscape:export-bgcolor="#ffffffff" /><defs
|
||||
id="defs1"><rect
|
||||
x="23.477077"
|
||||
y="80.721069"
|
||||
width="65.340591"
|
||||
height="30.699326"
|
||||
id="rect1" /><rect
|
||||
x="624.52576"
|
||||
y="193.55714"
|
||||
width="72.878258"
|
||||
|
|
@ -185,12 +194,12 @@
|
|||
inkscape:groupmode="layer"
|
||||
id="layer1"
|
||||
transform="translate(-119.58348,-121.61611)"><path
|
||||
d="m 677.86162,133.03739 a 4,4 0 0 1 4,4 4,4 0 0 1 -4,4 4,4 0 0 1 -4,-4 4,4 0 0 1 4,-4 m 0,10 c 4.42,0 8,1.79 8,4 v 2 h -16 v -2 c 0,-2.21 3.58,-4 8,-4 z"
|
||||
d="m 718.41067,134.14794 a 4,4 0 0 1 4,4 4,4 0 0 1 -4,4 4,4 0 0 1 -4,-4 4,4 0 0 1 4,-4 m 0,10 c 4.42,0 8,1.79 8,4 v 2 h -16 v -2 c 0,-2.21 3.58,-4 8,-4 z"
|
||||
id="path1" /><path
|
||||
d="m 620.72925,220.26075 c -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.45,-0.45 0.75,-1.08 0.75,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.25,0 -0.39,0.04 l -1.63,-3.25 c 0.48,-0.45 0.77,-1.08 0.77,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.29,1.34 0.76,1.79 l -1.62,3.25 c -0.14,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.3,1.34 0.75,1.79 l -1.61,3.25 c -0.13,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-0.7 -0.29,-1.34 -0.76,-1.79 l 1.62,-3.25 c 0.14,0.04 0.26,0.04 0.39,0.04 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.46,-0.45 0.75,-1.08 0.75,-1.79 0,-0.71 -0.29,-1.34 -0.75,-1.79 l 1.61,-3.25 c 0.13,0.04 0.26,0.04 0.39,0.04 0.13,0 0.26,0 0.39,-0.04 l 1.61,3.25 c -0.45,0.45 -0.75,1.09 -0.75,1.79 0,1.38 1.12,2.5 2.5,2.5 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 z"
|
||||
d="m 661.2783,221.3713 c -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.45,-0.45 0.75,-1.08 0.75,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.25,0 -0.39,0.04 l -1.63,-3.25 c 0.48,-0.45 0.77,-1.08 0.77,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.29,1.34 0.76,1.79 l -1.62,3.25 c -0.14,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.3,1.34 0.75,1.79 l -1.61,3.25 c -0.13,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-0.7 -0.29,-1.34 -0.76,-1.79 l 1.62,-3.25 c 0.14,0.04 0.26,0.04 0.39,0.04 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.46,-0.45 0.75,-1.08 0.75,-1.79 0,-0.71 -0.29,-1.34 -0.75,-1.79 l 1.61,-3.25 c 0.13,0.04 0.26,0.04 0.39,0.04 0.13,0 0.26,0 0.39,-0.04 l 1.61,3.25 c -0.45,0.45 -0.75,1.09 -0.75,1.79 0,1.38 1.12,2.5 2.5,2.5 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 z"
|
||||
id="path1-3-2" /><g
|
||||
id="g2"
|
||||
transform="translate(19.303425,11.421284)"><path
|
||||
transform="translate(59.852472,12.531835)"><path
|
||||
d="m 398.62829,208.83947 c -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.45,-0.45 0.75,-1.08 0.75,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.25,0 -0.39,0.04 l -1.63,-3.25 c 0.48,-0.45 0.77,-1.08 0.77,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.29,1.34 0.76,1.79 l -1.62,3.25 c -0.14,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.3,1.34 0.75,1.79 l -1.61,3.25 c -0.13,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-0.7 -0.29,-1.34 -0.76,-1.79 l 1.62,-3.25 c 0.14,0.04 0.26,0.04 0.39,0.04 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.46,-0.45 0.75,-1.08 0.75,-1.79 0,-0.71 -0.29,-1.34 -0.75,-1.79 l 1.61,-3.25 c 0.13,0.04 0.26,0.04 0.39,0.04 0.13,0 0.26,0 0.39,-0.04 l 1.61,3.25 c -0.45,0.45 -0.75,1.09 -0.75,1.79 0,1.38 1.12,2.5 2.5,2.5 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 z"
|
||||
id="path1-3-7" /><path
|
||||
d="m 311.97287,208.83947 c -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.45,-0.45 0.75,-1.08 0.75,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.25,0 -0.39,0.04 l -1.63,-3.25 c 0.48,-0.45 0.77,-1.08 0.77,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.29,1.34 0.76,1.79 l -1.62,3.25 c -0.14,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.3,1.34 0.75,1.79 l -1.61,3.25 c -0.13,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-0.7 -0.29,-1.34 -0.76,-1.79 l 1.62,-3.25 c 0.14,0.04 0.26,0.04 0.39,0.04 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.46,-0.45 0.75,-1.08 0.75,-1.79 0,-0.71 -0.29,-1.34 -0.75,-1.79 l 1.61,-3.25 c 0.13,0.04 0.26,0.04 0.39,0.04 0.13,0 0.26,0 0.39,-0.04 l 1.61,3.25 c -0.45,0.45 -0.75,1.09 -0.75,1.79 0,1.38 1.12,2.5 2.5,2.5 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 z"
|
||||
|
|
@ -200,7 +209,7 @@
|
|||
d="m 138.66202,208.83947 c -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.45,-0.45 0.75,-1.08 0.75,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.25,0 -0.39,0.04 l -1.63,-3.25 c 0.48,-0.45 0.77,-1.08 0.77,-1.79 0,-1.38 -1.12,-2.5 -2.5,-2.5 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.29,1.34 0.76,1.79 l -1.62,3.25 c -0.14,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,0.71 0.3,1.34 0.75,1.79 l -1.61,3.25 c -0.13,-0.04 -0.26,-0.04 -0.39,-0.04 -1.38,0 -2.5,1.12 -2.5,2.5 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-0.7 -0.29,-1.34 -0.76,-1.79 l 1.62,-3.25 c 0.14,0.04 0.26,0.04 0.39,0.04 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 -0.13,0 -0.26,0 -0.39,0.04 l -1.61,-3.25 c 0.46,-0.45 0.75,-1.08 0.75,-1.79 0,-0.71 -0.29,-1.34 -0.75,-1.79 l 1.61,-3.25 c 0.13,0.04 0.26,0.04 0.39,0.04 0.13,0 0.26,0 0.39,-0.04 l 1.61,3.25 c -0.45,0.45 -0.75,1.09 -0.75,1.79 0,1.38 1.12,2.5 2.5,2.5 0.13,0 0.25,0 0.39,-0.04 l 1.63,3.25 c -0.47,0.45 -0.77,1.09 -0.77,1.79 0,1.38 1.12,2.5 2.5,2.5 1.38,0 2.5,-1.12 2.5,-2.5 0,-1.38 -1.12,-2.5 -2.5,-2.5 z"
|
||||
id="path1-3-5" /></g><g
|
||||
id="g3"
|
||||
transform="matrix(0.5418082,0,0,0.5418082,209.66651,160.77376)"><path
|
||||
transform="matrix(0.5418082,0,0,0.5418082,250.21556,161.88431)"><path
|
||||
d="m 128.87841,293.95453 h -1.5 v -4 c 0,-1.11 -0.9,-2 -2,-2 h -4 v -1.5 a 2.5,2.5 0 0 0 -2.5,-2.5 2.5,2.5 0 0 0 -2.5,2.5 v 1.5 h -4 a 2,2 0 0 0 -2,2 v 3.8 h 1.5 c 1.5,0 2.7,1.2 2.7,2.7 0,1.5 -1.2,2.7 -2.7,2.7 h -1.5 v 3.8 a 2,2 0 0 0 2,2 h 3.8 v -1.5 c 0,-1.5 1.2,-2.7 2.7,-2.7 1.5,0 2.7,1.2 2.7,2.7 v 1.5 h 3.8 a 2,2 0 0 0 2,-2 v -4 h 1.5 a 2.5,2.5 0 0 0 2.5,-2.5 2.5,2.5 0 0 0 -2.5,-2.5 z"
|
||||
id="path1-1" /><path
|
||||
d="m 147.78408,293.95453 h -1.5 v -4 c 0,-1.11 -0.9,-2 -2,-2 h -4 v -1.5 a 2.5,2.5 0 0 0 -2.5,-2.5 2.5,2.5 0 0 0 -2.5,2.5 v 1.5 h -4 a 2,2 0 0 0 -2,2 v 3.8 h 1.5 c 1.5,0 2.7,1.2 2.7,2.7 0,1.5 -1.2,2.7 -2.7,2.7 h -1.5 v 3.8 a 2,2 0 0 0 2,2 h 3.8 v -1.5 c 0,-1.5 1.2,-2.7 2.7,-2.7 1.5,0 2.7,1.2 2.7,2.7 v 1.5 h 3.8 a 2,2 0 0 0 2,-2 v -4 h 1.5 a 2.5,2.5 0 0 0 2.5,-2.5 2.5,2.5 0 0 0 -2.5,-2.5 z"
|
||||
|
|
@ -209,17 +218,18 @@
|
|||
id="path1-1-5" /><path
|
||||
d="m 147.78408,312.78657 h -1.5 v -4 c 0,-1.11 -0.9,-2 -2,-2 h -4 v -1.5 a 2.5,2.5 0 0 0 -2.5,-2.5 2.5,2.5 0 0 0 -2.5,2.5 v 1.5 h -4 a 2,2 0 0 0 -2,2 v 3.8 h 1.5 c 1.5,0 2.7,1.2 2.7,2.7 0,1.5 -1.2,2.7 -2.7,2.7 h -1.5 v 3.8 a 2,2 0 0 0 2,2 h 3.8 v -1.5 c 0,-1.5 1.2,-2.7 2.7,-2.7 1.5,0 2.7,1.2 2.7,2.7 v 1.5 h 3.8 a 2,2 0 0 0 2,-2 v -4 h 1.5 a 2.5,2.5 0 0 0 2.5,-2.5 2.5,2.5 0 0 0 -2.5,-2.5 z"
|
||||
id="path1-1-6" /></g><path
|
||||
d="m 674.91307,320.20398 h 12 v 1 h 3 v 2 h -3 v 2 h 3 v 2 h -3 v 2 h 3 v 2 h -3 v 2 h 3 v 2 h -3 v 1 h -12 v -1 h -3 v -2 h 3 v -2 h -3 v -2 h 3 v -2 h -3 v -2 h 3 v -2 h -3 v -2 h 3 v -1 m 5,11 v 3 h 1 v -3 h -1 m 2,0 v 3 h 1 v -3 h -1 m 2,0 v 3 h 1 v -3 z"
|
||||
d="m 715.46212,321.31453 h 12 v 1 h 3 v 2 h -3 v 2 h 3 v 2 h -3 v 2 h 3 v 2 h -3 v 2 h 3 v 2 h -3 v 1 h -12 v -1 h -3 v -2 h 3 v -2 h -3 v -2 h 3 v -2 h -3 v -2 h 3 v -2 h -3 v -2 h 3 v -1 m 5,11 v 3 h 1 v -3 h -1 m 2,0 v 3 h 1 v -3 h -1 m 2,0 v 3 h 1 v -3 z"
|
||||
id="path1-3" /><path
|
||||
d="m 674.81037,155.52639 -6.98397,6.98398 -3.20099,-3.20099 0.82062,-0.82062 2.38037,2.37455 6.16335,-6.15753 z"
|
||||
d="m 715.35942,156.63694 -6.98397,6.98398 -3.20099,-3.20099 0.82062,-0.82062 2.38037,2.37455 6.16335,-6.15753 z"
|
||||
id="path1-2"
|
||||
style="stroke-width:0.581998" /><path
|
||||
d="m 690.85622,155.31885 -0.82061,-0.82062 -3.25337,3.25337 -3.25337,-3.25337 -0.82062,0.82062 3.25337,3.25337 -3.25337,3.25336 0.82062,0.82062 3.25337,-3.25337 3.25337,3.25337 0.82061,-0.82062 -3.25336,-3.25336 z"
|
||||
d="m 731.40527,156.4294 -0.82061,-0.82062 -3.25337,3.25337 -3.25337,-3.25337 -0.82062,0.82062 3.25337,3.25337 -3.25337,3.25336 0.82062,0.82062 3.25337,-3.25337 3.25337,3.25337 0.82061,-0.82062 -3.25336,-3.25336 z"
|
||||
id="path1-4"
|
||||
style="stroke-width:0.581998" /><path
|
||||
d="m 605.22925,133.03739 c -1.11,0 -2,0.89 -2,2 v 14 c 0,1.11 0.89,2 2,2 h 14 c 1.11,0 2,-0.89 2,-2 v -14 c 0,-1.11 -0.89,-2 -2,-2 h -14 m 0,2 h 14 v 14 h -14 v -14 m 2,2 v 2 h 10 v -2 h -10 m 0,4 v 2 h 10 v -2 h -10 m 0,4 v 2 h 7 v -2 z"
|
||||
d="m 645.7783,134.14794 c -1.11,0 -2,0.89 -2,2 v 14 c 0,1.11 0.89,2 2,2 h 14 c 1.11,0 2,-0.89 2,-2 v -14 c 0,-1.11 -0.89,-2 -2,-2 h -14 m 0,2 h 14 v 14 h -14 v -14 m 2,2 v 2 h 10 v -2 h -10 m 0,4 v 2 h 10 v -2 h -10 m 0,4 v 2 h 7 v -2 z"
|
||||
id="path1-6-2" /><g
|
||||
id="g1"><path
|
||||
id="g1"
|
||||
transform="translate(40.549047,1.110551)"><path
|
||||
d="m 229.78754,133.03739 c -1.11,0 -2,0.89 -2,2 v 14 c 0,1.11 0.89,2 2,2 h 14 c 1.11,0 2,-0.89 2,-2 v -14 c 0,-1.11 -0.89,-2 -2,-2 h -14 m 0,2 h 14 v 14 h -14 v -14 m 2,2 v 2 h 10 v -2 h -10 m 0,4 v 2 h 10 v -2 h -10 m 0,4 v 2 h 7 v -2 z"
|
||||
id="path1-6" /><path
|
||||
d="m 142.46545,133.03739 c -1.11,0 -2,0.89 -2,2 v 14 c 0,1.11 0.89,2 2,2 h 14 c 1.11,0 2,-0.89 2,-2 v -14 c 0,-1.11 -0.89,-2 -2,-2 h -14 m 0,2 h 14 v 14 h -14 v -14 m 2,2 v 2 h 10 v -2 h -10 m 0,4 v 2 h 10 v -2 h -10 m 0,4 v 2 h 7 v -2 z"
|
||||
|
|
@ -292,62 +302,45 @@
|
|||
d="m 612.80796,154.06037 v 41.44442"
|
||||
id="path14-30" /></g><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle)"
|
||||
d="m 158.35996,235.83766 103.22843,76.30967"
|
||||
d="m 198.90901,236.94821 103.22843,76.30967"
|
||||
id="path3" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-8)"
|
||||
d="M 401.25721,235.53297 298.02878,311.84264"
|
||||
d="M 441.80626,236.64352 338.57783,312.95319"
|
||||
id="path3-6" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1"
|
||||
d="M 270.36864,309.36435 Z"
|
||||
d="M 310.91769,310.4749 Z"
|
||||
id="path5" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle)"
|
||||
d="m 240.15784,231.2781 28.73056,76.1835"
|
||||
d="m 280.70689,232.38865 28.73056,76.1835"
|
||||
id="path6" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-3)"
|
||||
d="m 320.59208,230.87264 -28.73056,76.1835"
|
||||
d="m 361.14113,231.98319 -28.73056,76.1835"
|
||||
id="path6-2" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#DartArrow)"
|
||||
d="M 300.13199,326.93805 H 653.72144"
|
||||
id="path8" /><text
|
||||
xml:space="preserve"
|
||||
style="fill:#000000;stroke:#000000;stroke-width:0.995291;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#DartArrow)"
|
||||
d="M 340.68104,328.0486 H 689.85076"
|
||||
id="path8" /><path
|
||||
style="font-weight:300;font-size:14.6667px;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light';white-space:pre;stroke:#000000;stroke-width:1.00157"
|
||||
d="m 294.6704,344.06792 q 1.89201,0 2.83067,0.73334 0.93867,0.73333 0.93867,2.22934 0,0.748 -0.24933,1.34933 -0.23467,0.60134 -0.73334,1.02667 -0.484,0.41067 -1.24667,0.63067 -0.748,0.22 -1.77467,0.22 h -1.52533 v 4.28268 h -0.85067 v -10.47203 z m -0.088,0.748 h -1.672 v 4.70802 h 1.43733 q 1.01201,0 1.73067,-0.22 0.73334,-0.23467 1.10001,-0.76267 0.38133,-0.54267 0.38133,-1.46667 0,-1.17334 -0.71867,-1.71601 -0.71867,-0.54267 -2.25867,-0.54267 z m 8.69736,1.78934 q 1.32001,0 1.96534,0.64534 0.64534,0.64533 0.64534,2.05334 v 5.23601 h -0.63067 l -0.14667,-1.32001 h -0.044 q -0.27867,0.44001 -0.64533,0.77734 -0.352,0.32267 -0.86534,0.51333 -0.49867,0.176 -1.232,0.176 -0.73334,0 -1.276,-0.24933 -0.54267,-0.264 -0.83601,-0.748 -0.29333,-0.484 -0.29333,-1.188 0,-1.15867 0.95334,-1.76001 0.95333,-0.616 2.772,-0.704 l 1.42267,-0.0587 v -0.55733 q 0,-1.15867 -0.46933,-1.628 -0.45467,-0.48401 -1.36401,-0.48401 -0.60133,0 -1.15866,0.16134 -0.54267,0.14666 -1.10001,0.42533 l -0.264,-0.66 q 0.54267,-0.27867 1.20267,-0.45467 0.66,-0.176 1.364,-0.176 z m 0.45467,4.04801 q -1.48133,0.0733 -2.21467,0.528 -0.73333,0.44 -0.73333,1.33467 0,0.71867 0.45466,1.10001 0.45467,0.36666 1.23201,0.36666 1.24667,0 1.90667,-0.68933 0.67467,-0.68934 0.68933,-1.93601 v -0.76267 z m 6.84934,3.32934 q 0.32267,0 0.616,-0.044 0.29334,-0.0587 0.51334,-0.132 v 0.67467 q -0.22,0.088 -0.55734,0.14667 -0.32266,0.0587 -0.68933,0.0587 -0.64534,0 -1.1,-0.23466 -0.45467,-0.24934 -0.70401,-0.76267 -0.23466,-0.51334 -0.23466,-1.32 v -4.95735 h -1.14401 v -0.46933 l 1.14401,-0.264 0.308,-1.80401 h 0.528 v 1.86267 h 2.40534 v 0.67467 h -2.40534 v 4.92801 q 0,0.80667 0.32267,1.23201 0.32266,0.41066 0.99733,0.41066 z m 4.98668,0 q 0.32267,0 0.616,-0.044 0.29334,-0.0587 0.51334,-0.132 v 0.67467 q -0.22,0.088 -0.55734,0.14667 -0.32267,0.0587 -0.68933,0.0587 -0.64534,0 -1.10001,-0.23466 -0.45466,-0.24934 -0.704,-0.76267 -0.23466,-0.51334 -0.23466,-1.32 v -4.95735 h -1.14401 v -0.46933 l 1.14401,-0.264 0.308,-1.80401 h 0.528 v 1.86267 h 2.40534 v 0.67467 h -2.40534 v 4.92801 q 0,0.80667 0.32267,1.23201 0.32266,0.41066 0.99733,0.41066 z m 5.73468,-7.39201 q 1.01201,0 1.67201,0.46933 0.67467,0.45467 1.012,1.26134 0.352,0.80666 0.352,1.86267 v 0.55733 h -5.57335 q 0,1.56934 0.71867,2.39067 0.73334,0.82134 2.06801,0.82134 0.73333,0 1.276,-0.11733 0.54267,-0.11734 1.188,-0.41067 v 0.76267 q -0.58667,0.264 -1.15867,0.38133 -0.572,0.11733 -1.33467,0.11733 -1.15867,0 -1.98,-0.484 -0.80667,-0.484 -1.232,-1.37867 -0.41067,-0.90933 -0.41067,-2.112 0,-1.20267 0.396,-2.12667 0.396,-0.92401 1.15867,-1.45201 0.76267,-0.54266 1.848,-0.54266 z m 0,0.704 q -1.08533,0 -1.74533,0.71867 -0.66001,0.704 -0.77734,2.03867 h 4.69335 q 0,-0.82134 -0.23467,-1.43734 -0.23467,-0.616 -0.71867,-0.968 -0.46933,-0.352 -1.21734,-0.352 z m 8.34536,-0.68934 q 0.29333,0 0.55733,0.044 0.27867,0.0293 0.49867,0.088 l -0.11733,0.76267 q -0.23467,-0.0587 -0.46934,-0.088 -0.23466,-0.044 -0.51333,-0.044 -0.528,0 -0.95334,0.20533 -0.42533,0.20534 -0.73333,0.58667 -0.308,0.38134 -0.46934,0.90934 -0.16133,0.528 -0.16133,1.188 v 4.28268 h -0.836 v -7.80269 h 0.704 l 0.088,1.46667 h 0.044 q 0.19067,-0.45466 0.51333,-0.80667 0.33734,-0.36666 0.80667,-0.572 0.46934,-0.22 1.04134,-0.22 z m 6.072,-0.0147 q 1.33467,0 2.03867,0.68933 0.704,0.68934 0.704,2.17067 v 5.08935 h -0.82134 v -5.04535 q 0,-1.12933 -0.51333,-1.65733 -0.51333,-0.52801 -1.51067,-0.52801 -1.276,0 -1.92134,0.74801 -0.64533,0.748 -0.64533,2.22933 v 4.25335 h -0.82134 v -7.80269 h 0.66 l 0.132,1.34934 h 0.044 q 0.22,-0.42534 0.58667,-0.76267 0.36667,-0.33733 0.88,-0.528 0.52801,-0.20533 1.18801,-0.20533 z m 10.14936,5.88134 q 0,0.704 -0.352,1.20267 -0.352,0.49867 -1.02667,0.76267 -0.67466,0.24933 -1.64267,0.24933 -0.80667,0 -1.43733,-0.14666 -0.63067,-0.14667 -1.07067,-0.352 v -0.80667 q 0.528,0.264 1.188,0.44 0.66,0.16133 1.33467,0.16133 1.17334,0 1.68667,-0.38133 0.51333,-0.396 0.51333,-1.07067 0,-0.44 -0.24933,-0.71867 -0.23467,-0.29333 -0.71867,-0.51333 -0.46933,-0.23467 -1.17333,-0.46934 -0.71867,-0.24933 -1.27601,-0.49867 -0.55733,-0.264 -0.86533,-0.66 -0.308,-0.396 -0.308,-1.08533 0,-0.93867 0.76267,-1.46667 0.76266,-0.528 2.03867,-0.528 0.704,0 1.30533,0.132 0.61601,0.132 1.11467,0.352 l -0.29333,0.68933 q -0.45467,-0.20533 -1.02667,-0.33733 -0.572,-0.132 -1.144,-0.132 -0.924,0 -1.43734,0.32266 -0.51333,0.32267 -0.51333,0.93867 0,0.46934 0.23466,0.74801 0.24934,0.264 0.71867,0.45466 0.484,0.19067 1.15867,0.44 0.68934,0.23467 1.24667,0.49867 0.572,0.264 0.89467,0.67467 0.33733,0.41067 0.33733,1.1 z"
|
||||
id="text9"
|
||||
style="font-size:14.6667px;white-space:pre;shape-inside:url(#rect9);fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1"
|
||||
transform="translate(37.782292,9.9028949)"><tspan
|
||||
x="212.26172"
|
||||
y="343.52651"
|
||||
id="tspan3"><tspan
|
||||
style="font-weight:300;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light'"
|
||||
id="tspan1">Patterns</tspan></tspan></text><text
|
||||
xml:space="preserve"
|
||||
aria-label="Patterns" /><path
|
||||
style="font-weight:300;font-size:14.6667px;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light';white-space:pre;stroke:#000000;stroke-width:1.00157"
|
||||
d="m 707.10615,356.08452 -3.76935,-9.57736 h -0.044 q 0.0293,0.264 0.0293,0.60134 0.0147,0.32266 0.0147,0.68933 0.0147,0.352 0.0147,0.73334 v 7.55335 h -0.792 v -10.47203 h 1.276 l 3.60801,9.18136 h 0.0587 l 3.65201,-9.18136 h 1.24666 v 10.47203 h -0.836 v -7.64135 q 0,-0.32267 0,-0.66001 0.0147,-0.33733 0.0293,-0.66 0.0147,-0.32266 0.0293,-0.60133 h -0.044 l -3.78401,9.56269 z m 14.59336,-3.90134 q 0,0.90933 -0.23467,1.65733 -0.22,0.73334 -0.67467,1.27601 -0.44,0.528 -1.1,0.82133 -0.66,0.29333 -1.51067,0.29333 -0.80667,0 -1.452,-0.27866 -0.63067,-0.29334 -1.08534,-0.82134 -0.45467,-0.54267 -0.68933,-1.29067 -0.23467,-0.748 -0.23467,-1.65733 0,-1.27601 0.41067,-2.17068 0.42533,-0.90933 1.21733,-1.39333 0.80667,-0.484 1.90667,-0.484 1.11467,0 1.87734,0.51333 0.77734,0.49867 1.17334,1.408 0.396,0.90934 0.396,2.12668 z m -6.11601,0 q 0,0.99733 0.27866,1.76 0.27867,0.748 0.86534,1.15867 0.58667,0.41067 1.46667,0.41067 0.90933,0 1.496,-0.41067 0.58667,-0.42534 0.86534,-1.17334 0.27866,-0.76267 0.27866,-1.74533 0,-0.98267 -0.264,-1.73068 -0.264,-0.748 -0.836,-1.17333 -0.572,-0.42534 -1.496,-0.42534 -1.33467,0 -1.99467,0.88001 -0.66,0.88 -0.66,2.44934 z m 11.04402,4.048 q -1.56933,0 -2.42,-0.99733 -0.836,-1.012 -0.836,-2.96267 0,-2.02401 0.88,-3.08001 0.89467,-1.056 2.44934,-1.056 0.66,0 1.144,0.20533 0.484,0.19067 0.82133,0.528 0.35201,0.33733 0.54267,0.76267 h 0.0587 q -0.0147,-0.352 -0.044,-0.76267 -0.0147,-0.42533 -0.0147,-0.76267 v -3.168 h 0.836 v 11.14669 h -0.67466 l -0.11734,-1.36401 h -0.044 q -0.20533,0.41067 -0.54267,0.76267 -0.33733,0.33734 -0.836,0.54267 -0.49867,0.20533 -1.20267,0.20533 z m 0.10267,-0.71866 q 1.364,0 1.92134,-0.82134 0.572,-0.82133 0.572,-2.376 v -0.132 q 0,-1.61334 -0.572,-2.46401 -0.55734,-0.86534 -1.86267,-0.86534 -1.26134,0 -1.90667,0.89467 -0.64534,0.88 -0.64534,2.53734 0,1.58401 0.616,2.40534 0.616,0.82134 1.87734,0.82134 z m 8.82937,-7.37735 q 1.012,0 1.672,0.46933 0.67467,0.45467 1.012,1.26134 0.352,0.80666 0.352,1.86267 v 0.55733 h -5.57334 q 0,1.56934 0.71867,2.39067 0.73333,0.82134 2.068,0.82134 0.73333,0 1.276,-0.11733 0.54267,-0.11734 1.18801,-0.41067 v 0.76267 q -0.58667,0.264 -1.15867,0.38133 -0.57201,0.11733 -1.33467,0.11733 -1.15867,0 -1.98001,-0.484 -0.80667,-0.484 -1.232,-1.37867 -0.41067,-0.90933 -0.41067,-2.112 0,-1.20267 0.396,-2.12667 0.396,-0.92401 1.15867,-1.45201 0.76267,-0.54266 1.84801,-0.54266 z m 0,0.704 q -1.08534,0 -1.74534,0.71867 -0.66,0.704 -0.77734,2.03867 h 4.69335 q 0,-0.82134 -0.23467,-1.43734 -0.23467,-0.616 -0.71867,-0.968 -0.46933,-0.352 -1.21733,-0.352 z m 5.98401,7.24535 h -0.836 v -11.14669 h 0.836 z"
|
||||
id="text9-8"
|
||||
style="font-size:14.6667px;white-space:pre;shape-inside:url(#rect9-3);fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1"
|
||||
transform="translate(448.28203,11.447466)"><tspan
|
||||
x="212.26172"
|
||||
y="343.52651"
|
||||
id="tspan5"><tspan
|
||||
style="font-weight:300;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light'"
|
||||
id="tspan4">Model</tspan></tspan></text><text
|
||||
xml:space="preserve"
|
||||
id="text10"
|
||||
style="font-style:normal;font-variant:normal;font-weight:300;font-stretch:normal;font-size:14.6667px;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light';white-space:pre;shape-inside:url(#rect10);fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1"><tspan
|
||||
x="622.09766"
|
||||
y="245.4269"
|
||||
id="tspan7"><tspan
|
||||
dx="0 12.789359"
|
||||
id="tspan6">Ma</tspan></tspan></text><text
|
||||
xml:space="preserve"
|
||||
aria-label="Model" /><path
|
||||
d="m 701.17104,225.77243 -3.76934,-9.57736 h -0.044 q 0.0293,0.264 0.0293,0.60134 0.0147,0.32266 0.0147,0.68933 0.0147,0.352 0.0147,0.73334 v 7.55335 h -0.792 V 215.3004 h 1.276 l 3.60801,9.18136 h 0.0587 l 3.65201,-9.18136 h 1.24667 v 10.47203 h -0.83601 v -7.64135 q 0,-0.32267 0,-0.66001 0.0147,-0.33733 0.0293,-0.66 0.0147,-0.32266 0.0293,-0.60133 h -0.044 l -3.78401,9.56269 z m 10.82402,-7.93469 q 1.32,0 1.96534,0.64534 0.64533,0.64533 0.64533,2.05334 v 5.23601 h -0.63067 l -0.14666,-1.32001 h -0.044 q -0.27867,0.44001 -0.64534,0.77734 -0.352,0.32267 -0.86533,0.51333 -0.49867,0.176 -1.23201,0.176 -0.73333,0 -1.276,-0.24933 -0.54267,-0.264 -0.836,-0.748 -0.29334,-0.484 -0.29334,-1.188 0,-1.15867 0.95334,-1.76001 0.95334,-0.616 2.77201,-0.704 l 1.42267,-0.0587 v -0.55733 q 0,-1.15867 -0.46934,-1.628 -0.45467,-0.48401 -1.364,-0.48401 -0.60134,0 -1.15867,0.16134 -0.54267,0.14666 -1.1,0.42533 l -0.264,-0.66 q 0.54266,-0.27867 1.20267,-0.45467 0.66,-0.176 1.364,-0.176 z m 0.45467,4.04801 q -1.48134,0.0733 -2.21468,0.528 -0.73333,0.44 -0.73333,1.33467 0,0.71867 0.45467,1.10001 0.45467,0.36666 1.232,0.36666 1.24667,0 1.90667,-0.68933 0.67467,-0.68934 0.68934,-1.93601 v -0.76267 z m 6.84933,3.32934 q 0.32267,0 0.61601,-0.044 0.29333,-0.0587 0.51333,-0.132 v 0.67467 q -0.22,0.088 -0.55733,0.14667 -0.32267,0.0587 -0.68934,0.0587 -0.64533,0 -1.1,-0.23466 -0.45467,-0.24934 -0.704,-0.76267 -0.23467,-0.51334 -0.23467,-1.32 v -4.95735 h -1.144 v -0.46933 l 1.144,-0.264 0.308,-1.80401 h 0.528 v 1.86267 h 2.40534 v 0.67467 h -2.40534 v 4.92801 q 0,0.80667 0.32267,1.23201 0.32267,0.41066 0.99733,0.41066 z m 5.91069,0.704 q -1.144,0 -1.95067,-0.46933 -0.792,-0.484 -1.21734,-1.37867 -0.41067,-0.89467 -0.41067,-2.156 0,-1.32001 0.46934,-2.22934 0.484,-0.90934 1.32,-1.37867 0.836,-0.484 1.96534,-0.484 0.55733,0 1.04133,0.11733 0.49867,0.10267 0.88001,0.27867 l -0.22,0.704 q -0.39601,-0.16134 -0.85067,-0.24934 -0.45467,-0.10266 -0.86534,-0.10266 -0.93867,0 -1.584,0.41067 -0.63067,0.396 -0.968,1.144 -0.32267,0.748 -0.32267,1.78934 0,0.968 0.27867,1.716 0.29333,0.73333 0.89466,1.144 0.60134,0.41067 1.54001,0.41067 0.572,0 1.07067,-0.11733 0.51333,-0.11734 0.93867,-0.30801 v 0.76267 q -0.38134,0.16134 -0.88001,0.27867 -0.49866,0.11733 -1.12933,0.11733 z m 4.67867,-7.71468 q 0,0.29333 -0.0147,0.572 -0.0147,0.27867 -0.0293,0.55734 h 0.0587 q 0.20533,-0.44001 0.572,-0.77734 0.36667,-0.33733 0.88,-0.528 0.52801,-0.19067 1.18801,-0.19067 0.89467,0 1.51067,0.308 0.616,0.29334 0.924,0.92401 0.32267,0.616 0.32267,1.61333 v 5.08935 h -0.82134 v -5.04535 q 0,-1.12933 -0.51333,-1.65733 -0.51334,-0.52801 -1.51067,-0.52801 -0.85067,0 -1.42267,0.33734 -0.572,0.32267 -0.86534,0.99733 -0.27867,0.66001 -0.27867,1.64267 v 4.25335 h -0.82133 v -11.14669 h 0.82133 z m 8.71203,4.53201 q 0,-0.64533 0.10267,-1.07067 0.11733,-0.44 0.38134,-0.792 0.27866,-0.36667 0.76266,-0.73334 0.54267,-0.44 0.89467,-0.77733 0.352,-0.33733 0.528,-0.71867 0.176,-0.38133 0.176,-0.95333 0,-0.89467 -0.55733,-1.33467 -0.55734,-0.45467 -1.51067,-0.45467 -0.63067,0 -1.15867,0.14666 -0.51333,0.14667 -1.02667,0.41067 l -0.308,-0.704 q 0.616,-0.29333 1.20267,-0.44 0.60133,-0.16133 1.33467,-0.16133 1.32,0 2.068,0.66 0.76267,0.64533 0.76267,1.86267 0,0.704 -0.23466,1.20267 -0.22001,0.484 -0.63067,0.88 -0.41067,0.396 -0.968,0.82134 -0.44001,0.33733 -0.68934,0.64533 -0.23467,0.29333 -0.33733,0.67467 -0.10267,0.36667 -0.10267,0.90933 v 0.22 h -0.68934 z m -0.24933,2.47867 q 0,-0.36666 0.16133,-0.54267 0.16134,-0.176 0.46934,-0.176 0.32267,0 0.484,0.176 0.16133,0.17601 0.16133,0.54267 0,0.352 -0.16133,0.54267 -0.16133,0.176 -0.484,0.176 -0.308,0 -0.46934,-0.176 -0.16133,-0.19067 -0.16133,-0.54267 z"
|
||||
id="text11"
|
||||
style="font-style:normal;font-variant:normal;font-weight:300;font-stretch:normal;font-size:14.6667px;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light';white-space:pre;shape-inside:url(#rect11);fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1"
|
||||
transform="translate(30.083254,17.760373)"><tspan
|
||||
x="624.52539"
|
||||
y="206.90151"
|
||||
id="tspan8">Match?</tspan></text><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-8)"
|
||||
d="M 681.15807,311.31992 V 234.32221"
|
||||
style="font-weight:300;font-size:14.6667px;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light';white-space:pre;stroke:#000000;stroke-width:1.00157"
|
||||
aria-label="Match?" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:0.955696;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-8)"
|
||||
d="M 721.70712,312.43047 V 242.3245"
|
||||
id="path11" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-8)"
|
||||
d="m 629.88934,219.52269 h 15.96729"
|
||||
style="fill:#000000;stroke:#000000;stroke-width:0.826931;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-8)"
|
||||
d="m 673.15834,220.63324 h 10.88445"
|
||||
id="path12" /><path
|
||||
style="fill:#000000;stroke:#000000;stroke-width:1.00157;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-8)"
|
||||
d="M 677.2396,209.10375 646.77647,153.49258"
|
||||
id="path13" /></g></svg>
|
||||
style="fill:#000000;stroke:#000000;stroke-width:0.928487;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#ConcaveTriangle-8-86)"
|
||||
d="M 719.86857,205.06657 V 169.25636"
|
||||
id="path2" /><path
|
||||
style="font-weight:300;font-size:14.6667px;font-family:'Noto Sans';-inkscape-font-specification:'Noto Sans Light';white-space:pre;stroke:#000000;stroke-width:1.00157"
|
||||
d="m 141.13472,223.67412 -1.39334,-3.608 h -4.35601 l -1.408,3.608 h -0.86534 l 4.12134,-10.50135 h 0.76267 l 4.01868,10.50135 z m -3.06535,-8.15468 q -0.044,-0.14667 -0.132,-0.396 -0.088,-0.24934 -0.176,-0.528 -0.088,-0.27867 -0.176,-0.49867 -0.0733,0.24933 -0.16133,0.51333 -0.0733,0.264 -0.16133,0.49867 -0.0733,0.23467 -0.14667,0.42534 l -1.43734,3.76934 h 3.79868 z m 11.22003,5.44134 q 0,0.93867 -0.46934,1.58401 -0.45466,0.63067 -1.276,0.95333 -0.82134,0.32267 -1.86267,0.32267 -0.64534,0 -1.15867,-0.0733 -0.51333,-0.0587 -0.93867,-0.14667 -0.42533,-0.10266 -0.76267,-0.23466 v -0.85068 q 0.55734,0.22001 1.29067,0.39601 0.73334,0.16133 1.61334,0.16133 0.80667,0 1.408,-0.23467 0.616,-0.23466 0.95334,-0.68933 0.352,-0.45467 0.352,-1.14401 0,-0.63066 -0.29334,-1.02666 -0.27866,-0.39601 -0.88,-0.68934 -0.58667,-0.308 -1.51067,-0.616 -0.66,-0.23467 -1.188,-0.484 -0.51333,-0.24934 -0.88,-0.58667 -0.36667,-0.33733 -0.55734,-0.792 -0.19066,-0.45467 -0.19066,-1.1 0,-0.86534 0.44,-1.45201 0.44,-0.58667 1.188,-0.89467 0.76267,-0.308 1.716,-0.308 0.76267,0 1.45201,0.14667 0.68933,0.132 1.30533,0.41067 l -0.27866,0.73333 q -0.61601,-0.264 -1.24667,-0.396 -0.63067,-0.132 -1.26134,-0.132 -0.73333,0 -1.29067,0.22 -0.54267,0.20534 -0.85067,0.63067 -0.308,0.41067 -0.308,1.02667 0,0.67467 0.29334,1.08534 0.29333,0.41066 0.85066,0.68933 0.55734,0.264 1.36401,0.54267 0.924,0.32267 1.584,0.68933 0.67467,0.352 1.02667,0.88 0.36667,0.52801 0.36667,1.37867 z m 5.06,2.71334 h -0.85067 v -9.70935 h -3.37334 v -0.76267 h 7.59735 v 0.76267 h -3.37334 z m 8.78537,-2.068 q 0,0.704 -0.352,1.20267 -0.352,0.49867 -1.02667,0.76267 -0.67467,0.24933 -1.64267,0.24933 -0.80667,0 -1.43734,-0.14667 -0.63067,-0.14666 -1.07067,-0.352 v -0.80667 q 0.528,0.26401 1.188,0.44001 0.66001,0.16133 1.33467,0.16133 1.17334,0 1.68667,-0.38133 0.51334,-0.39601 0.51334,-1.07067 0,-0.44 -0.24934,-0.71867 -0.23466,-0.29334 -0.71866,-0.51334 -0.46934,-0.23466 -1.17334,-0.46933 -0.71867,-0.24933 -1.276,-0.49867 -0.55734,-0.264 -0.86534,-0.66 -0.308,-0.396 -0.308,-1.08534 0,-0.93866 0.76267,-1.46667 0.76267,-0.528 2.03867,-0.528 0.704,0 1.30534,0.132 0.616,0.132 1.11467,0.352 l -0.29334,0.68934 q -0.45467,-0.20534 -1.02667,-0.33734 -0.572,-0.132 -1.144,-0.132 -0.924,0 -1.43734,0.32267 -0.51333,0.32267 -0.51333,0.93867 0,0.46933 0.23467,0.748 0.24933,0.264 0.71867,0.45467 0.484,0.19067 1.15866,0.44 0.68934,0.23467 1.24667,0.49867 0.57201,0.264 0.89467,0.67467 0.33734,0.41066 0.33734,1.1 z"
|
||||
id="text1"
|
||||
aria-label="ASTs" /></g></svg>
|
||||
|
|
|
|||
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 35 KiB |
Loading…
Add table
Add a link
Reference in a new issue