Plagiarism checks

Introduction

Artemis allows tutors and exercise instructors to check assignment submissions from students for plagiarism. With this feature, different types of assignments can be checked in Artemis, including programming assignments, modeling assignments, and text assignments. Plagiarism checks are available for both course and exam exercises. To conduct a plagiarism check, a tutor or instructor must initiate one of two available procedures: manual plagiarism checks or continuous plagiarism control. These checks are always specific to a particular exercise. First, we give an overview of different available features of the plagiarism check. Next, we explain both plagiarism check workflows from the perspective of various Artemis users, using UML Activity Diagrams to visualize the process.

Plagiarism Check Overview

In this section we give an overview of available features of plagiarism check. We explain different configuration settings, possible user actions and present the plagiarism result views, which are accessible after plagiarism checks run.

Settings

Before starting the plagiarism check, the user can configure different settings to get the best possible results.

1. Similarity Threshold in % (minimum value 0, maximum value 100).

Ignore comparisons whose similarity is below this threshold. A similarity of 0% means that there is no overlap between two submissions, a similarity of 50% means that about half of two submissions are identical. A similarity of 100% means that two submissions are completely identical.

2. Minimum Score in % (minimum value 0, maximum value 100).

Consider only submissions with a score greater than or equal to this value.

3. Minimum Size.

1. Programming exercises: Consider only submissions that have at least as many git diff lines in comparison to the template as the specified value.

2. Modeling exercises: Consider only submissions that have at least as many modeling elements as the specified value.

3. Text exercises: Consider only submissions that have at least as many words as the specified value.

Actions

The user can use different actions to run a plagiarism check, inspect the results or improve the performance.

Note

Plagiarism detection can take a long time for large courses and exams.

1. Detect plagiarism: The user can execute the plagiarism check using this action.

2. Rerun plagiarism detection: The user can rerun the plagiarism check by executing this action. It can be helpful to check for plagiarism using the different settings as described in section Settings.

3. Clean up: The user can clean up the latest plagiarism check results for the selected exercise. It helps to keep the database slim and safe storage capacity. Moreover, it should improve the overall system performance. To execute the clean-up action, the user must approve it by clicking ok on the dialog, as this action deletes all potentially approved or denied plagiarism cases.

4. Download: The user can download the plagiarism results in JSON or CSV format to open them in a selected editor for the further analysis.

Results

After the plagiarism check was executed, the results can be inspected in different views.

1. Overview of the similarity distribution. This statistical overview shows the similarity distribution based on the percentage as a histogram chart. Above the chart, additional metrics show various statistics about the latest check. The user can analyze the distribution quickly and adjust the plagiarism check settings accordingly.

2. Selected submission. When the user selects the submission from the submission list on the left side of the plagiarism run result view, new actions can be selected to initiate the plagiarism check workflow. We will provide further details on the workflow in the next section.

Manual Plagiarism Checks

In this section, we explain the process of the manual plagiarism case detection process from different users’ perspectives.

Tutors

Tutors and instructors can execute plagiarism checks. They carefully review the automatically identified cases and provide a first decision whether to accept or deny the found case. We visualized the process in the following diagram.

1. Open the Exercise via:

1. For course exercises: Course Management → Course → Exercises → Your Exercise.

2. For exam exercises: Course Management → Course → Exams → Exam → Exercise Groups → Your Exercise.

2. Access the plagiarism tool as seen in the picture

3. Run the Plagiarism Check.

1. Tutors and instructors can adjust the similarity threshold and other settings as described in section Settings if deemed necessary (most of the time not, depends on the formulation of the exercise and on the number of possible solution. A similarity of less than 50% typically means that students did not plagiarize).

2. Checking exercises with many students can take some time (sometimes multiple minutes) and might be resource intensive → We recommend to run plagiarism checks at times when only a few users actively use Artemis in such cases

4. Start checking for plagiarism.

1. Review if the presented matches are actual plagiarism cases or not.

2. Depending on your decision, either deny or confirm the match as plagiarism.

3. Continue until the matches start to get “too different”.

Instructors

The instructors can execute the same actions as tutors, additionally they are able to make a final decision on the particular confirmed plagiarism case. We visualized the process in the following diagram.

1. Open the plagiarism cases via:

1. For course exercises: Course Management → Course → Assessment → Plagiarism Cases.

2. For exam exercises: Course Management → Course → Exams → Exam → Plagiarism Cases.

2. Overview the current status of all confirmed plagiarism cases by inspecting the management overview. The overview provides the following information about each confirmed case:

1. In how many comparisons the confirmed case appears.

2. Student was notified or not.

3. Student has responded or not.

4. Final verdict.

3. Export the confirmed cases as CSV file to analyze them in another editor.

4. Select one confirmed case and navigate to the case detail view.

5. Notify the student about the potential plagiarism case and ask him or her for a statement.

6. Make a final verdict by selecting one of 3 available options. The final verdict must be approved by clicking ‘Confirm’ in the dialog.

1. No plagiarism. The instructor can deem the accusation invalid and resolve the plagiarism case.

2. Warning. The instructor can write a warning message to the student and confirm the verdict in the dialog.

3. Point deduction in % (minimum value 0, maximum 100). Deduct exercise points and confirm the verdict in the dialog.

4. Plagiarism. If a grading key exists, the student receives the plagiarism special grade for the exam or course that the corresponding exercise belongs to. The next steps must be made manually by contacting responsible persons in the university to mark the student’s grade as “Unterschleif”.

Students

After the student got a notification that the instructor asked for the statement, he or she has one week to respond. The process is visualized in the following diagram.

1. Open the notified plagiarism confirmed case via:

1. For course exercises: Course Overview → Course → Exercise → Plagiarism Case / Resolved Plagiarism Case.

2. For exam exercises: Course Overview → Course → Exams → Exam → Plagiarism Case / Resolved Plagiarism Case.

2. Reply to the message of the instructor.

3. Wait for the final verdict.

Continuous Plagiarism Control

In this section, we explain the automated flow of the continuous plagiarism control (CPC) and present actions taken by different Artemis users.

Automated Flow

The CPC is an experimental feature designed to shorten the feedback loop for students and reduce the manual work required from tutors and instructors. Continuous feedback on potential plagiarism aims to encourage students to submit their original work and enhance their learning outcomes. The continuous plagiarism control does not accuse students of plagiarism, nor does it affect their scores. Instead, it notifies students of significant similarities detected between their submissions, allowing them to revise their solutions.

The CPC initiates plagiarism checks every night for all onboarded exercises, processing all the latest submissions for each exercise. If a significant similarity is detected, the CPC creates or updates a significant similarity plagiarism case for that submission. It is a special form of plagiarism case, which is marked as a warning and presents a gentle message about detected similarity. If a submission does not include significant similarity, the CPC removes any associated existing significant similarity plagiarism case. Subsequently, the student analyzes the feedback received and, if the submission period is still open, can submit a new solution. If the submission period has ended, the student can either dispute the plagiarism case or choose to take no action. Eventually, the instructor makes a decision about accusing the student of plagiarism and about any further steps.

The following activity diagram depicts the CPC flow described above.

Videos below show the full CPC process from student’s and instructor’s perspective.

Creating a new exercise with the CPC enabled.

Reviewing check’s results by an instructor.

Reviewing check’s results by a student.

Fixing submissions marked as a significant similarity.

Responding to a significant similarity plagiarism case that persisted after the exercise due date and adding a verdict to it.

Instructors

Configuration of the CPC

Instructors can configure various aspects of the CPC for each exercise. The following list describes all configuration options:

• Instructors can enable the CPC. - Instructor can turn the CPC on and off for a particular exercise. Default: OFF.

• Instructors can enable the post-due date checks. - Instructor can configure the CPC to test the exercise one more time the night after the exercise due date. This allows to test submissions posted shortly before the due date. Default: OFF.

• Instructors can set the response period for the significant similarity plagiarism case. - Instructor can specify for how long after the exercise due date a student will be able to submit a complaint. Default: 7 days. Min: 7 days.

• Instructors can set the similarity threshold. - Instructor can specify the similarity threshold for the plagiarism check performed by the CPC. Optimal value for this parameter depends on an individual exercise. Similarity above the threshold does not implicate plagiarism. Default: 90%.

• Instructors can set the minimum score. - Instructor can specify the minimum submission score for the plagiarism check performed by the CPC. Default: 0.

• Instructors can set the minimum size. - Instructor can specify the minimum submission size for the plagiarism check performed by the CPC. Default: 50.

Instructors can adjust the CPC configuration options on the exercise create and update page:

Post exercise due date steps and results verification

After the exercise due date, instructors can manually review all of the significant similarity plagiarism cases and decide on their verdicts. Steps at this point are equivalent to the steps in manual checks flow.

Instructors can check the plagiarism checks results CPC creates during and after the submission period using exactly the same steps and interface like with manual checks.

Students

When a student submits a work for which the CPC identifies significant similarity and creates a plagiarism case, the student receives a notification similar to what is sent when instructors create a classic plagiarism case. If the exercise due date has not passed, the student can submit updated work again. The CPC will check the submission the following night and will either remove or update the significant similarity plagiarism case based on the results of the check. Before the exercise due date, a student cannot see the submissions of other students involved in the significant similarity plagiarism case.

When a significant similarity plagiarism case exists for the latest submission, the student sees a dedicated button redirecting to it:

A significant similarity plagiarism case contains a short message and an overview of similarity in the submission:

In case the exercise due date has passed and student does not agree with the significant similarity plagiarism case, the student can dispute the case within the period configured by the instructor.