Modeling exercise

Note

Artemis uses an npm package called Apollon as its modeling editor.
It has its standalone version which can be accessed via https://apollon.ase.in.tum.de/.
The standalone version is free to use without the necessity of creating an account.
It offers additional features, including but not limited to, sharing and exporting the diagram.
For more information please visit Apollon Standalone.

Overview 

Conducting a Modeling exercise consists of 3 steps:

Instructor prepares exercise: Creates and configures the modeling exercise in Artemis.
Student solves exercise: Student works on the exercise and submits the solution.
Tutors assesses submissions: Reviews the submitted exercises and creates results for the students.

Setup 

The following sections describe the supported features and the process of creating a new modeling exercise.

Open .
Navigate into Exercises of your preferred course.

Create new modeling exercise 

Click on Create new modeling exercise.

The following screenshot illustrates the first section of the form. It consists of:

Title: Title of an exercise.
Categories: Category of an exercise.
Difficulty: Difficulty of an exercise. (No level, Easy, Medium or Hard).
Mode: Solving mode of an exercise. This cannot be changed afterwards (Individual or Team).
Release Date: Date after which students can access the exercise.
Due Date: Date till when students can work on the exercise.
Assessment Due Date: Date after which students can view the feedback of the assessments from the instructors.
Inclusion in course score calculation: Option that determines whether or not to include exercise in course score calculation.
Points: Total points of an exercise.
Bonus Points: Bonus points of an exercise.
Diagram Type: Type of diagram that is used throughout an exercise.

Note

Fields marked with red are mandatory to be filled.

Note

The field Diagram Type determines the components that students/instructors can use while working on the exercise.
This option cannot be changed after creating the exercise.
For example: If the instructor selects class diagram as its diagram type, users (instructors/students) will now only be able to use components of class diagrams throughout the exercise.

The following screenshot illustrates the second section of the form. It consists of:

Enable automatic assessment suggestions: When enabled, Artemis tries to automatically suggest assessments for diagram elements based on previously graded submissions for this exercise.
Enable feedback suggestions from Athena: When enabled, Artemis tries to automatically suggest assessments for diagram elements using the Athena service.
Problem Statement: The task description of the exercise as seen by students.
Assessment Instructions: Instructions for instructors while assessing the submission.

Note

If you are not clear about any of the fields, you can access additional hints by hovering over the hint icon for many of them.

../../_images/create-modeling-exercise-form-hint.png

The following screenshot illustrates the last section of the form. It consists of:

Example Solution: Example solution of an exercise.
Example Solution Explanation: Explanation of the example solution.
Example Solution Publication Date: Date after which the example solution is accessible for students. If you leave this field empty, the solution will only be published to tutors.

Once you are done defining the schema of an exercise, you can now create an exercise by clicking on save button. You will then be redirected to Example Submissions for Assessment Training Page.

In this page, you can either Create Example Submission or Use as Example Submission for Assessment Training. Example submissions can be used to assess the submissions of students semi-automatically. Artemis uses those submissions to automatically apply the known assessment comments to similar model elements in other submissions as well.

Select if you want to create an example submission from scratch.
Alternatively, after the exercise already started, you can also use some submissions submitted by students as an example submission. For that, click on .

Note

Artemis uses semi-automatic grading of modeling exercises using machine learning. You can hence train the model by selecting Use in Assessment Training checkbox while creating an example submission.

../../_images/use-in-assessment-training.png

Import Modeling Exercise 

Alternatively, you can also import modeling exercise from the existing one by clicking on Import Modeling Exercise.
An import modal will prompt up, where you will have an option to select and import previous modeling exercises from the list by clicking on button.
Once you import one of the exercise, you will then be redirected to a form which is similar to Create new modeling exercise form with all the fields filled from imported exercise. You can now modify the fields as per your necessity to create a new Modeling Exercise.

Result 

Click the button of the modeling exercise and adapt the interactive problem statement. There you can also set release and due dates.
Click the button to see the scores achieved by the students.
Click the button to see the list of students participated in the exercise.
Click the button to see the list of submission submitted by students.
Click the button to modify/add example submission of the exercise.
Click the button to delete the exercise.
You can get an overview of the exercise by clicking on the title.

Student Submission 

When the exercise is released students can work on the exercise.
They can start the exercise by clicking the button.
Once they start the exercise, they will now have the option to work on it in an online modeling editor by clicking on the button.
The screenshot below depicts the online modeling exercise interface for students. They can read the Problem Statement, work on the online editor and also provide an explanation to their solutions, if needed.

Assessment 

When the due date is over you can assess the submissions.

To assess the submissions, first click on Assessment Dashboard.
Then click on Submissions of the modeling exercise.
You will then be redirected to Submissions and Assessments Page.
Click on button of specific student. You will then be redirected to the assessment page where you will be able to assess submission of that student.
You can now start assessing the elements of the model by double clicking it. Once you double click, you will get an assessment dialog where you can assign points, feedback and navigate through all other assessable components.
Alternatively, you can also assess the diagram by dragging and dropping assessment instructions from the Assessment Instructions section.
Feedback to the entire submission can also be added by clicking on the button.

Once you’re done assessing the solution, you can either:

Click on to save the incomplete assessment so that you can continue it afterwards.
Click on to submit the assessment.
Click on to cancel and release the lock of the assessment.
Click on to navigate to exercise dashboard page.

Automatic Assessment Suggestions 

If the checkbox Automatic assessment suggestions enabled is checked for a modeling exercise, Artemis generates assessment suggestions for submissions using the Athena service. This section provides insights into how suggestions are retrieved in Artemis and how to apply them in the exercise grading process.

Note

To learn how to set up an instance of the Athena service and configure your Artemis installation accordingly, please refer to the section Athena Service.

After clicking on assess-submission on one of the submission entries on the Submissions and Assessments Page, assessment suggestions are loaded automatically as indicated by the following loading indicator:

Once assessment suggestions have been retrieved, a notice on top of the page indicates that the current submission contains assessment suggestions created via generative AI.

../../_images/assessment-suggestions-notice.png

The suggestions themselves are shown as follows. If a suggestion directly references a diagram element, a dialog showing the suggested grading score for this specific suggestion as well as a suggestion on what could be improved is attached to the corresponding element. In this example, a remark is made that an element is present in the evaluated BPMN diagram without being mentioned in the problem statement.

../../_images/referenced-assessment-suggestion.png

If a suggestion addresses a more general aspect of the diagram, multiple diagram elements at once, or elements that are missing from the diagram, the suggestion is shown in a card overview below the diagram. These unreferenced suggestions can be accepted or discarded via buttons on the individual suggestion cards.

../../_images/unreferenced-assessment-suggestion.png

An demonstration of the automated generation of assessment suggestions for a business process model can be found in the following screencast:

To learn how automatic suggestions are generated and how exercises can be optimized for automatic evaluation, please refer to Generation of Assessment Suggestions for Modeling Exercises.