DesignWIKI

Fil Salustri's Design Site

User Tools

Site Tools


teaching:salustri_s_grading_policy

Salustri's Grading Policy

This page describes my fundamental grading practises.

General Remarks

Typically, an assignment is graded on the basis of a number of aspects. These aspects will vary from assignment to assignment and your instructor will provide information about them to you.

Each aspect will typically have a weight associated with it, such that more important aspects have a heavier weight than other aspects. Your instructor will provide weights for each aspect on a per-assignment basis. In the absence of such information, assume every aspect is equally weighted.

Assessment is done as follows:

  • Each aspect is score on a scale.
  • The score for each aspect is multiplied by the weight of that aspect.
  • The weighted scores are summed.
  • The sum is normalized to some overall base (e.g., out of 10, or 100, or whatever).

There are two scales used: a 10-point scale, and a 5-point scale. Each is described below.

Grading on a 10-Point Scale

For each aspect, you grade an assignment on the following scale:

0
Absent, did not submit at all, with or without excuse.
1 - 4
Woefully inadequate; does not meet even the most basic requirements.
5
Minimum passing grade, roughly equivalent to a GPA of 1 or a letter grade of D-.
6
Average (statistically and performatively); acceptable but nothing to write home about; roughly equivalent to C.
7
Good / above average; roughly equivalent to B.
8 - 10
Unequivocally excellent; awesome; exceeds all reasonable expectations with respect to technical detail, diligence, attention to detail, precision, and conciseness; equivalent to the “A's.”

To calculate a grade on an assignment, multiply each raw grade by its weight, sum the results, and divide by the sum of the weights.

Example

Assume the following aspects and weights for a hypothetical assignment:

ASPECT WEIGHT RAW SCORE WEIGHTED SCORE
Principal views 2 6 12
Auxiliary view 3 8 24
Dimensions 1 5 5
General format 1 9 9
TOTAL 50
MARK (TOTAL/SUM OF WGTS) 7.14 (out of 10)

The mark reported by the grader in this case would be 7.14.

Grading on a 5-Point Scale

Using a 5-point scale is actually easier and more intuitive because the 5-point scale maps well to both CEAB guidelines and typical GPA measures.

0
Does not meet expectations.
Work was not submitted, or was so comprehensively poor as to be without merit at all.
Work either was not submitted, or was submitted but was so comprehensively poor that it was entirely inadequate.
1
Does not meet expectations.
Work fails to meet the absolute minimal requirements.
Work was submitted, and a very few things were done acceptably, but the vast majority of the work was unacceptable.
2
Marginally meets expectations; this is the lowest passing grade.
Work meets the absolute minimal requirements, but continued performance at this level will likely result in ultimate failure due to “snowballing”.
That is, as subsequent work becomes more challenging but leverages past work, the student will likely be unable to keep up.
3
Meets expectations.
This is an “average” score, and therefore also the most common.
Work is imperfect but reliably well-done in most regards. Sufficient mastery is shown in the subject to a sufficient degree.
4
Exceeds expectations.
Exemplary/excellent work, clearly above-average, showing greater degree of mastery, precision, accuracy, attention to detail, and diligence.
ASPECT WEIGHT RAW SCORE WEIGHTED SCORE
Principal views 2 3 6/8
Auxiliary view 3 3 9/12
Dimensions 1 2 2/4
General format 1 1 1/4
TOTAL 7 9 18/28
MARK (TOTAL/SUM OF WGTS) 2.6 (out of 4)

The mark reported by the grader in this case would be 2.6 (out of 4) or 6.4 (out of 10).

Grading team reports

This is so complex that it deserves a page of its own.

Deductions and penalties

Here is a list of deductions and penalties.

Since online submissions can be made any day (including holidays), then calculating lateness for online submissions includes every day.

Since hardcopy submissions can only be done during days when the University is open, calculating lateness for hardcopy submissions only includes “business days”.

The “late clock” starts at the due time and runs in 24 hour blocks. For instance, if an item is due by 16:00 on a Thursday, and you submit anytime between 16:00 Thursday and 16:00 Friday, you will incur a one-day late penalty.

All deductions are taken from the total value of the work. Example: If an assignment is graded out of 10, then a 10% deduction means a deduction of 1/10.

Late Assignments
10% per day including weekends
This applies to the whole item; i.e., if you hand in half of an assignment late, the whole assignment is late.
Late PTI
2% per day including weekends
If you have to do the Personality Temperament Indicator for your course, the late penalty will be applied to your final course grade.
Late Peer Review
1% per day including weekends
If peer reviews are expected, this penalty will be applied for each late peer review submission to your final course grade.
Submitting Corrupted Files
10% per day after a grace period
Your TA will inform you, and provide a 24 hour grace period to re-submit uncorrupted files. Thereafter, the penalty will apply to the entire assignment.
Document-naming errors
2% per document.
This applies to naming conventions for reports, assignments, and any other documents submitted for grading, where such conventions are defined by the instructors.
Not following submission instructions
10%
This applies to all other submission requirements, excluding file corruption and naming errors (above) and any other rules covered in the Course Outline.
Examples: Incorrect document template use; incomplete sets of files.
WDF-related penalties
5% of project grade if WDF not shared with instructor and TAs by deadlines
2% of project grade if WDF is incorrectly filled out
No project grades will be processed until all team members sign (i.e., agree to) WDF content
Changes made to a WDF after all team members have signed will be interpreted as malicious attempts to take academic advantage of teammates, and will be regarded as Academic Misconduct.

End of Semester Adjustments

In my courses, if your grade is adjusted, it will always be adjusted upward.

Policy requires us to inform students of the grades they get in individual assignments, and of the overall breakdown of marks. There is an implication here that we always just do a straight weighted-sum of individual assignments to get your final mark.

That implication is, however, incorrect.

There are many reasons why instructors might perform adjustments to final grades before sending them to the Registrar. We are allowed and encouraged to make such adjustments to ensure fairness to all students.

However, such adjustments are not public information; they are confidential and no one will ever tell you specifically what adjustments they may have made.

I cannot speak to what other instructors may do, but here are some of the problems I've encountered that might in my courses:

  • Projects are poorly defined. I try to make sure our design briefs are fair and accurate. However, we sometimes notice a problem with a design brief too late. It's not the students' fault, so we will consider an adjustment.
  • TA variations. TAs aren't just clones; they're individuals with different skills and abilities. This may lead to noticeable grading variations between TAs. We check for such variations. If we find them, we adjust for them.
  • Variations over a week. Sometimes we notice effects that run from Mondays to Fridays. If these effects are consistent, we will adjust for them, because it's not fair - all else being equal - that students with classes on Mondays should get lower marks than students with classes on Fridays.

There are literally dozens of possible things that can go wrong, and we check for all of them by applying statistical methods to the marks. When we find a trend that suggests an artificial skew to marks that was not under the students' control, we will do our best to adjust for it.

But we will never disclose the specifics of adjustments made to a given student.

Getting a "boost"

It is generally very hard to get a boost (an increase in a letter grade).

Consider the following example.

  • Say all assignments are marked out of 10, then weighted to get a contribution to your final grade.
  • Say you got 7/10 on an assignment, but you think it should have been 8/10. Let $D$ be the mark difference (in this case, $D = 8-7 = 1$).
  • The difference must be weighted to see how much the change would impact the final mark.
  • The weighted change in your mark is difference times weight $W$.
  • But there are only very few ranges of marks where the change $DW$ will result in a letter grade boost.
    • Indeed, you must be within $DW$ marks of the border between letter grades to get a boost.
    • For a specific letter grade, the odds are only $DW \over 100$ that your final grade will be in the right range.
  • There are 13 letter grades (at Ryerson), so there are 13 ranges in which a boost can result. So the odds of getting boost are $13DW \over 100$ total. While it is true that the range of the letter grades is variable, the resulting effect is insignificant.

The following table indicates actual numbers.

Difference Weight Odds (%)
1 1 1.67 0.2
2 5 1.67 1
3 1 10 1.3
4 5 10 6.5
5 1 20 2.6
6 5 20 13
7 1 30 3.9
8 5 30 19.5

Of particular note for MEC222 is the following:

  • The odds of getting a boost by increasing the grade on a homework assignment by 50% (!!) is just 1%.
  • To have a 4% chance of getting a boost based on changes to the final exam, you would need 10 extra marks on the exam itself.

In other words, don't worry about getting boosts.

The 5-pile grading method

Design and drafting assignments are very difficult to grade, because of the acceptable level of variability inherent in this kind of work. The 5-pile method is meant to provide consistency without overly burdening those doing the grading. These instructions are for instructors and teaching assistants, not for students. The 5-pile method is particularly useful for grading homework assignments, and not for major design projects.

This method is based on the observation that assignments with different kinds of errors can still end up with the same grade; that is, that many different kinds of errors are of identical severity.

  1. Make sure you have a good understanding of what is expected for the assignment, and where variability in solutions is expected and acceptable.
  2. Go through all the assignments once to:
    • review the quality of the work;
    • correct any errors;
    • write feedback comments on the assignment for the student's benefit; but
    • do not assign a grade to the assignment yet.
  3. Go through all the assignments a second time, sorting them into three piles:
    • a “best” pile for those assignments that were at the upper end of the group;
    • a “worst” pile for those assignments where the most improvement is needed; and
    • an “average” pile for all other assignments.
  4. Set aside the “best” and “worst” piles.
  5. Go through the remaining pile one more time, again sorting into three piles for “average,” “below average,” and “above average.”
    • Once this is done, you will have five piles total.
    • Every assignment in each pile will receive the same grade.
  6. All else being equal, the middle pile is assigned a grade of 6/10. Of course, if most students is doing extremely well, then the middle pile should be given a higher grade; conversely, if most students are completely borking the assignment, then the middle pile should be given a lower grade.
  7. Examine the “best” pile. Determine what grade those assignments should all get. 10/10 is reserved for absolutely flawless assignments.
  8. Examine the “worst” pile. Determine what grade those assignments should all get. 0/10 is reserved for absolute failures to capture any sensible result in the assignment.
    • You have now established the bounds of the grading – the best, the worst, and the average.
  9. For the remaining piles, assign grades to each pile between the best and worst.

Here's a simple example:

  • You have divided the assignments into five piles.
  • You examine the “average” pile and determine that each assignment in that pile should get 6/10.
  • You examine the “best” pile and determine that each assignment in that pile should get 9/10.
  • You examine the “worst” pile and determine that each assignment in that pile should get 4/10.
  • The other piles would get (roughly) 5/10 and 7/10.
    • If you very evenly split the grades between 4/10 and 9/10, you would assign 5.25/10, 6.5/10, and 7.75/10.
    • However, three significant digits of accuracy in grading is not reasonable for this kind of grading.
    • Therefore, you are advised to round, but recall that the distribution will usually be close to normal; that's why second best pile should get 7/10 rather than 8/10.

For small sets of assignments, there is a variation of the 5-pile method that can be quicker. This can apply to team-based assignments that are only a few pages long, but for which you might have only four or five assignments to grade.

  1. Make sure you have a good understanding of what is expected for the assignment, and where variability in solutions is expected and acceptable.
  2. Go through all the assignments once to:
    • review the quality of the work;
    • correct any errors;
    • write feedback comments on the assignment for the student's benefit; but
    • do not assign a grade to the assignment yet.
  3. Set one assignment down on a table.
  4. Consider the next assignment. Is it better than the one on the table, or worse?
    • If it's better, place it on the table to the right of the first assignment.
    • If it's worse, place it on the table to the left of the first assignment.
  5. Consider the next assignment.
    • Place it to the left of the two assignments if it is the worst so far.
    • Place it to the right of the two assignments if it is the best so far.
    • Place it between the other two assignments if it's better than one and worse than the other.
  6. Continue with the other assignments, arranging them in a row from worst to best.
  7. Examine the “best” assignment. Determine what grade it should get. 10/10 is reserved for absolutely flawless assignments.
  8. Examine the “worst” assignment. Determine what grade it should get. 0/10 is reserved for absolute failures to capture any sensible result in the assignment.
  9. You have now established the bounds of the grading – the best and the worst.
  10. For the remaining assignments, assign grades to each between the best and worst.
teaching/salustri_s_grading_policy.txt · Last modified: 2020.08.07 19:24 by Fil Salustri