WIP: Improve usefulness of remote gradebook data

[pwilkins]

[This issue is a work in progress. When it is ready for review, I'll remove the WIP tag.]

edX supports sending grade data to a remote gradebook. Currently it sends students' calculated grades for a single assignment. This feature integrates edX with an institution's enterprise grading system. Improving the quality of this integration means reducing the manual effort to incorporate edX grades into the Stellar Gradebook grade calculation.

Background

Currently edX's remote gradebook sends this CSV data to MIT's enterprise gradebook[1].

+-----------------+----------------+
| External email  | Ex 01          |
+-----------------+----------------+
| [email protected] | 0.0            |
+-----------------+----------------+
| [email protected] | 0.272727272727 |
+-----------------+----------------+

Note that the grade for assignment Ex 01 is a floating point number between 0 and 1. While this grade is meaningful in the context of edX, it is less useful in a remote grading system. The problem arises because edX is not the only source of assessment in residential course grading. There are usually other gradable events such as written assignments, lab work, and final exams. In practice, grade calculation can't be divided among multiple systems.

In the data example above, edX sends a calculated grade rather than a "raw" grade. The raw grade would be more useful because course graders must manually convert the edX grade to a raw score in Stellar Gradebook.

Mapping between edX and Learning Modules Gradebook
One of the challenges of exporting grades is aligning grading policies in the two systems. In the interest of making this clearer, here is a mapping between the terms used on each side.

+-----------------------+-------------------+
| edX                   | LMod              |
+-----------------------+-------------------+
| Assignment Type       | Category          |
+-----------------------+-------------------+
| Assignment Weight (?) | Category Weight   |
+-----------------------+-------------------+
| Graded Subsection     | Assignment        |
+-----------------------+-------------------+
| Problem Weight (?)    | Assignment Weight |
+-----------------------+-------------------+
| Problem               | n/a               |
+-----------------------+-------------------+
| Problem Weight        | (?)               |
+-----------------------+-------------------+

Our PyLmod library has no support for LMod categories.

Proposals for next steps

edX has already merged a PR (https://github.com/edx/edx-platform/pull/10395) that makes edX's "raw grade" available to the remote gradebook code. edX's "raw grade" is a tuple containing the number of correct answers and the total number of questions.

Evaluating these proposals includes considering what needs changing. There are up to three codebases involved: edx-platform, pylmod, and lmod-proxy. I'll note this as I go through the proposals.

Proposal 1

The simplest proposal is to send the raw grade rather than the calculated grade to the remote gradebook. This would be done by replacing the data above to this:

+-----------------+----------------+
| External email  | Ex 01          |
+-----------------+----------------+
| [email protected] | 0              |
+-----------------+----------------+
| [email protected] | 3              |
+-----------------+----------------+

Since the raw grade here is the number of correct answers, whether it is sent as an integer or as a floating point value is an implementation detail.

This change is better for course staff because they can set the max points manually on the Stellar "assignment" and Stellar will perform the proper calculation.

This proposal requires changes to edx-platform only.

Proposal 2

A second proposal is sending both the raw score and the max points to the remote gradebook by adding a column to the CSV data. I don't believe there is much value in the additional data provided in this way. The max points are really an attribute of the whole assignment so each cell in the added column would contain the same value.

The appeal of this proposal is that it isn't more work than the first proposal and requires changes to edx-platform only.

+-----------------+----------------+------------+
| External email  | Ex 01          | max points |
+-----------------+----------------+------------+
| [email protected] | 0.0            | 5          |
+-----------------+----------------+------------+
| [email protected] | 0.272727272727 | 5          |
+-----------------+----------------+------------+

Proposal 3

The third proposal provides better integration with the remote gradebook with commensurately greater effort. In addition to sending raw scores, it sends assignment weights as well as assignment type weights relieving course staff from performing these steps manually. It will require changes to edx-platform, PyLmod, and lmod_proxy. PyLmod is in the list because we would have to add support for Stellar Gradebook categories.

[1] Actually edX sends the CSV data to a proxy that uses the CSV data to make Learning Module Gradebook API calls.

[pdpinch]

@pwilkins when we spoke you had mentioned also sending the Assignment Type and its weight for each grade as well. Is that unnecessary, or just out of scope?

[pwilkins]

I'm still fleshing this out. I will add that proposal but I want to provide enough context so that the proposal will make sense.

[pwilkins]

@pdpinch this is close enough for your review. There is still one issue I need to resolve about mapping edX to LMod but it shouldn't stop you from looking at this.

[pdpinch]

I thought that one of the advantages of Proposal 2 is that we can use the max_points to set the corresponding value in the Learning Modules gradebook, so that course staff wouldn't have to set it by hand?

Going back to the example from Math, they would like an assignment with 50 possible points to be recorded in the Learning Modules gradebook as "out of 50"

[pwilkins]

I'm working with Eric Heubel to understand what "use max_points to set the corresponding value in..." means. There is some question what the "corresponding value" is. The mapping of values between edX and LMod isn't as straightforward as I thought.

[pdpinch]

I think "Max Points" is the field that Jen & Prof. Edelman have in mind:

Or am I missing some nuance?