Skip to content

Latest commit

 

History

History
476 lines (322 loc) · 18.3 KB

Instructor-Guide--Importing-and-Exporting-Data.md

File metadata and controls

476 lines (322 loc) · 18.3 KB
Raw

Bulk Upload and Download Data Feature

When needing to upload or download files, users are able to upload and download a grouped set of data contained in a CSV file(or in some cases also a YML file).

  • Features are listed in alphabetical order.
  • For more information about any file, please check the template files provided.

Annotation Categories

Instructors are able to upload and download annotations (including deductive annotations). Instructors can edit/delete annotations and add/remove annotation categories.

Upload and Download File Format

  • CSV and YAML files are supported for both uploads and downloads.

How to make a valid file of annotations for uploading to MarkUs?

CSV format:

  • Each annotation category has its own row.
  • The annotation category name is the first value in the row.
  • Following the category name, is the name of the associated criterion (if the category is for deductive annotations).
  • If no criterion should be associated with the category, leave an empty column value after the category name.
  • If the criterion is not specified, each value in the row following the empty value should be an annotation content.
  • If the criterion is specified, following the criterion should be deductive annotations, represented by the annotation content value, followed by the deduction value for that content.

An example of this file format:

syntax,,Factor out a helper method,Don't forget a return type,...
assumptions,first proof, Preconditions need to be stated,1.0,preconditions are incorrect,0.5,...

YAML format:

  • All annotation categories should be under one block.
  • For an annotation category without a criterion, each annotation text under it should begin with - and occupy one line.
  • For an annotation category with a criterion, the object notation should be used.
    • The category name should be a key which has two keys as its value.
    • One of the keys should be criterion: which has the criterion name as a value, and the other should be texts: which will have a list of annotation texts.
    • Each annotation text will be a list of the text content followed by the text deduction.

An example of this file format:

Induction:
  - State inductive hypothesis.
  - Remember the base case.
  - Use the inductive hypothesis in the proof.
Complexity:
  criterion: q2
  texts:
  - - "Remember to state the formal definition of big O()"
    - 1.0
  - - "Use the given upper bound in the proof."
    - 2.0

🗒️ Note: Annotations can be blank, annotation category names cannot be blank.

Assignments

Instructors are able to upload and download assignment settings.

Supported formats

Both CSV and YML files are supported.

Fields

The following fields can be specified for both file formats. They are listed with the key names required in the YML format, and in the order that the fields should appear in a CSV file.

  • short_identifier: a unique identifier for the assignment
  • description: a description of the assignment
  • repository_folder: the name of the assignment's repository folder, usually the same as the short identifier
  • due_date: the due date of the assignment

The remaining fields are optional.

  • message
  • group_min
  • group_max
  • tokens_per_period
  • allow_web_submits
  • student_form_groups
  • remark_due_date
  • remark_message
  • assign_graders_to_criteria
  • enable_test
  • enable_student_tests
  • allow_remarks
  • display_grader_names_to_students
  • display_median_to_students
  • group_name_autogenerated
  • is_hidden
  • vcs_submit

🗒️ Note: If the uploaded file contains a short identifier that is already used, the existing assignment is updated with the corresponding uploaded settings.

Assignment Grades

Instructors may download a summary of the grades for each assignment in CSV format.

The CSV file consists of two header rows at the top of the file:

  • The first row contains the Group, User Name, Last Name, First Name, Section, ID Number, Email, Final Grade, and each criterion. The last entry is Bonus/Deductions.
  • The second row contains of 4 dummy entries (it is ignored by MarkUs), followed by the "Out of" total for each column.

Every subsequent row is a student record:

  • The first six entries in the row is the Gruop, User Name, Last Name, First Name, Section, ID Number and Email of the student. If any of these fields are not provided, they are left blank.
  • Every subsequent entry is the grade of the student for that column. This entry can be left blank if no grade is given.
  • The students are ordered by their group. This means all students of the same gruop will appear consecutively.

Every row in the CSV file should have the same number of entries.

Example file

Gruop,User name,Last name,First name,Section,Id number,Email,Final grade,dolores,iusto,Bonus/Deductions
, , , ,Out of,10,6,4
group_001,c5anthei,George,Antheil,LEC0101,353472201,antheil.george@example.com,5,2,3
group_001,c5berkel,Lennox,Berkeley,LEC0201,815161511,berkeley.lennox@example.com,5,2,3
group_002,c5berkel,Alexander,Glazunov,LEC0201,148704361,glazunov.alexander@example.com,9,5,4

Criteria

Instructors can upload all marking criteria as a YML file.

Supported formats

YML files are supported for all criterion.

Fields

The YML file should contain a series of key-value pairs, where each key is the name of the criterion, and the value is an object containing the following fields:

  • type: 'rubric', 'flexible', or 'checkbox'
  • max_mark: A non-negative number with up to one decimal place
  • ta_visible: boolean (default: true)
  • peer_visible: boolean (default: false)

These fields are specific to certain types of criteria:

  • description: string (flexible and checkbox criteria only)

🗒️ Note:

  1. Criteria may not be uploaded if grades have already been released.
  2. Uploading criteria deletes existing criteria and any associated marks.
  3. The order of criteria for the assignment is the same as the order of criteria in the uploaded file.

Rubric Criterion

  1. Users can upload a rubric criterion with levels as many as they want.
  2. For each level, it should have a unique non-empty name and non-empty description.
  3. For each level, it should have its own unique non-negative mark.
  4. If uploading in a yml format, the fields ta_visible and peer_visible are required with a value of True or False.
  5. Uploading a new rubric criterion will replace all pre-existing level data with the data uploaded.

Rubric Criterion Example

criterion_name:
  max_mark: #
  type: Rubric
  levels:
    <level_name>:
      description: level_description
      mark: level_mark
    <level_name>:
      [...]
  ta_visible: true/false
  peer_visible: true/false
criterion_name, <level_name>, <level_description>, <level_mark>, <level_name>, ...

Checkbox/Flexible Criterion Example

criterion_name:
  type: checkbox
  max_mark: 1.0
  description: "Criterion description.
    Use quotation marks to break the description across multiple lines."
  ta_visible: true
  peer_visible: false

Groups

Instructors are able to upload/download a list of groups (with memberships) for an assignment.

Supported formats

Only CSV files are supported.

Fields

Every row of the CSV file should have the following format:

  • The first entry is the name of the group (e.g., group1). This cannot be blank, and must be unique across all assignments.
    • Each group name must be at most 30 characters long, and consist only of alphanumeric characters, hyphens, underscores, and spaces.
  • All subsequent entries are user names of students who are members of this group.

🗒️ Note: A student must belong to at most one group for the assignment. So the same student user name cannot appear in two different rows in your uploaded file, nor can the file contain a student who is already be a member of an existing group.

📼 Deprecation Notice prior to version 1.12.0: The second entry in a row is the the repository name, this cannot be blank, and must also be unique across all assignments. This field was removed in version 1.12.0 as repository names can no longer be set by the user.

Marks Spreadsheet Grades

Instructors are able to upload and download a list of student grades for a mark spreadsheet.

Supported formats

Only CSV files are supported.

File format

The CSV file consists of two header rows at the top of the file:

  • The first row contains the User Name, Last Name, First Name, Section, ID Number and Email (in this order). If the Show Total property is checked, the last column will also contain Total.
  • The second row contains six dummy entries (it is ignored by MarkUs), followed by the "Out of" total for each column.

Every subsequent row is a student record:

  • The first six entries in the row is the User Name, Last Name, First Name, Section, ID Number and Email of the student. If any of these fields are not provided, they are left blank.
  • Every subsequent entry is the grade of the student for that column. This entry can be left blank if no grade is given. The last entry is the total grade of the student and is shown only if the Show Total property is checked.
  • The students are ordered by their user name.

Every row in the CSV file should have the same number of entries.

Example file

User name,Last name,First name,Section,Id number,Email,Q1,Q2,Q3,Total
, , , , , ,Out of,4,5,2,12
c5anthei,George,Antheil,LEC0101,353472201,antheil.george@example.com,3,5,2,10
c5berkel,Lennox,Berkeley,LEC0201,815161511,berkeley.lennox@example.com,2,1,2,5

Handling existing data

By default, uploading data preserves existing columns and grades:

  1. If an existing column does not appear in the uploaded file, that column and all associated grades are preserved.
  2. If an existing column appears in the uploaded file, its "Out Of" field is updated, but any existing grades are not changed.
  3. If a student does not appear in the uploaded file, their grades do not change.

However, you can select "Overwrite existing grades and columns?" when uploading a file to overwrite existing data:

  1. Any existing columns that do not appear in the uploaded file are deleted (as are any grades for those columns).
  2. Any existing grade for a column is overwritten---even when the uploaded entry is blank. That is, it is possible to delete an existing grade by uploading a file with a blank entry for that student and column.

Note: even when "Overwrite existing grades and columns?" is checked, students who do not appear in the uploaded file remain unchanged.

Peer Reviews

Instructors are able to upload and download a list of peer review mappings for reviewers to reviewees.

Supported formats

Only CSV files are supported.

File format

Each line of the CSV file should have the reviewee group name followed by one or more reviewer group names.

Note: when students are working individually, their corresponding group name is simply their user name.

Students

Instructors are able to upload/download a list of students.

Supported formats

Both CSV and YML files are supported for downloading a student list. Only CSV is supported for file upload.

Fields

The following fields are listed with the key names in the YML file, and in the order they must appear in the CSV file.

  • username: the user name of the student
  • last_name: the last name of the student
  • first_name: the first name of the student
  • section_name: the name of the (MarkUs) section the student belongs to
  • id_number: the id number of the student
  • email: the email address of the student

🗒️ Note:

  1. If an uploaded CSV row contains a user name that already exists, then that student's information is updated. Only the student's section_name can be updated in this way.
  2. When updating an existing student, if the section_name, id_number, or email field is blank, then any existing data for that field will be removed!
  3. Uploaded CSV files need only contain data in the the username column. All other columns except section_name will be ignored. Only Admin users may change the last_name, first_name, id_number, and email values for a student.

Upload and Download Feature

Instructors are able to upload and download a list of students. If an uploaded student with the same user_name exists, then that student's information is updated.

Upload and Download Format

  • Only CSV files are supported for uploads.
  • CSV and YML files are supported for downloads.

TAs

Instructors are able to upload/download a list of TAs.

Supported formats

Both CSV and YML files are supported for downloading a TA list. Only CSV is supported for file upload.

Fields

The following fields are listed with the key names in the YML file, and in the order they must appear in the CSV file.

  • username: the user name of the TA (cannot be blank)
  • last_name: the last name of the TA (cannot be blank)
  • first_name: the first name of the TA (cannot be blank)
  • email: the email address of the TA (optional)

Example File

user_name_ta1,last_name_ta1,first_name_ta1
user_name_ta2,last_name_ta2,first_name_ta2
user_name_ta3,last_name_ta3,first_name_ta3

🗒️ Note:

  1. If an uploaded CSV row contains a user name that already exists, then that TA's information is updated.
  2. When updating an existing TA, if the email field is blank, then any existing email address for that TA will be removed!

Tags

Instructors are able to upload and download a list of tags used to label submissions when grading.

Supported formats

Both CSV and YML files are supported.

Fields

The following fields are specified for each tag.

  • name: the name of the tag. The name cannot be blank, and must be unique.
  • description: the description of the tag.
  • user: the user name of an instructor user who is the creator of the tag.

The CSV file format consists of one row per tag, with the above three fields in the given order. The YML file format is a list of objects, where each object contains the three above fields.

Examples

CSV format:

Sample tag,This is a sample description.,instructor_user1
Second look,This submission should be reviewed by an instructor.,instructor_user1

YML format:

- name: Sample tag
  description: This is a sample description.
  user: instructor_user1
- name: Second look
  description: This submission should be reviewed by an instructor.
  user: instructor_user2

Graders

For assignments, instructors can assign graders to:

  • Student groups
  • Marking criteria

For marks spreadsheets, instructors can assign graders to:

  • Individual students

Supported formats

Only CSV files are supported.

Fields

(Assignments) Assigning graders to groups

Each CSV row should consist of the group name followed by a list of TA user names.

Example:

group_0001,grader_user_1,grader_user_2
group_0002,grader_user_2,grader_user_3
group_0003,grader_user_1,grader_user_3
group_0004,grader_user_1

(Assignments) Assigning graders to marking criteria

Each CSV row should consist of the criterion name followed by a list of TA user names.

Question1_proof_structure,grader_user_1
Question1_proof_correctness,grader_user_2
Question2_runtime_partA,grader_user_1,grader_user_2
Question2_runtime_partB,grader_user_3
Question3_proof_by_contradiction,grader_user_4

(Marks Spreadsheets) Assigning graders to students

Each CSV row should consist of the student user name followed by a list of TA user names.

student_user_1,grader_user_1
student_user_2,grader_user_2
student_user_1,grader_user_1,grader_user_2

🗒️ Note:

  1. Any row with non-existent grader usernames will be ignored.
  2. Any row with non-existent group name/criterion name/student user name will be ignored.

Assignment Configuration

Instructors are able to upload/download a group of files that contain all the settings and files required to configure an assignment (that is, it's properties, tags, criteria, annotations, starter files and automated tests).

This upload/download DOES NOT copy assignment settings related to students or graders (i.e. specific settings, group information, etc.). Hence, after copying an assignment over, it is recommended that users check the assignment's settings to make sure it is configured as they desire.

Any peer reviews for an assignment are also NOT copied over. If you wish to also copy peer review assignments, you must upload/download each one separately. Note that when uploading a peer review assignment, please ensure that you create or upload its source assignment first, otherwise, the upload will fail.

Supported formats

A zip file that contains the following yml files for an assignment:

In addition, the zip file has three folders that contain:

  • An assignment's starter files and starter file settings (starter_file_config_files) which includes:
    • A starter file rules yml file which contains:
      • The name of the default starter file group within the zip file.
      • Information about each starter file group:
        • The name of the starter file group within the zip file.
        • The actual name of the starter group.
        • Whether it uses a different display name and if so, what that display name is.
    • Every uploaded starter file located in folders corresponding to which starter file group each file belongs to.
      • Every starter file group does not need a corresponding folder. In such a case, the group is assumed to have no starter files.
  • An assignment's automated test settings (automated_tests_config_files) which includes:
    • An automated test specs json file.
    • A folder containing every uploaded test file.
      • This is an optional folder. If it does not exist, that means there are no test files.

Important: While the contents of the yml files and folders can be extracted and modified for offline configuration, this is NOT recommended and may result in the assignment being unable to be copied over.