Assessment
Designing Lab Report Assessment Rubrics With Examples
Well-designed assessment instruments will help instructors inform their expectations to students and assess student lab reports fairly and efficiently. A rubric can be an excellent assessment instrument for engineering lab reports. It identifies the instructor’s expectations from an assigned lab report and then explicitly states the possible levels of achievement along a continuum (poor to excellent or novice to expert).
The rubric can be constructed for individual labs or an entire course.
Step 1: Define the purpose of the lab report assignment/assessment for which you are creating a rubric
The first step is to clarify the purpose of the assignment and identify the student’s learning outcome(s) from lab report writing.
Example:
Assume an instructor focuses on data presentation on one lab report assignment. The expected student outcome can be "Students can construct well-formatted tables, graphs, and photographs to present lab data."
Step 2: Choose a Rubric Type: Analytical vs. Holistic
Instructors need to select one out of two types of rubrics: analytical vs. holistic.
Analytical rubrics break down the components of a learning task, giving students feedback on each component. An analytic rubric offers feedback on areas of strength or weakness. Instructors can control each criterion’s relative importance. Not that if students are not experience lab report writers, it is best to not overwhelm them with too many rubrics. Using a only few writing rubrics (perhaps 3 or so) along with a few technical rubrics would be appropriate. As they gain experience, additional expectations can be introduced.
Holistic rubrics provide a broad overview of student performance and allow you to assess a learning task as a whole. A holistic rubric provides an overall sense of student performance. Instructors can save evaluation time by minimizing the number of decisions made.
Example:
In the context of entry-level engineering lab courses, an analytic rubric was chosen for further rubric development on lab data presentation.
Step 3: Define the Criteria
Instructors need to define grading criteria, which are the individual elements of a learning outcome that will be evaluated. Make a list of knowledge and skills required for the lab report assignment. They must be unambiguous, measurable. Before finalizing, eliminate any that are not critical.
Example:
The expected student outcome is "Students can construct well-formatted tables, graphs, and photographs to present lab data."
Here is the list of knowledge and skills required: 1) Use software such as Excel to construct graphs and tables; 2) Tables, graphs, and photographs are “stand-alone” with useful captions; 3) Indicate the units of measure (e.g. newtons) for the quantitative data.
Step 4: Design the Rating Scale
Instructors need to define the levels of quality for student performance. Most rating scales include: 3 levels (below; meet; above), 4 levels (fail; fair; pass; exceed), 5 levels (never; sometimes; usually; mostly; always), or 6 levels (limited-low; limited-high; acceptable-low; acceptable-high; proficient-low; proficient-high). More rating points can provide more detail; however, more rating points can also make grading more difficult and time-consuming.
Example:
Three levels are chosen for the rating scale. Score 1: Below expectations. Score 2: Meets expectations. Score 3: Exceeds expectations.
Step 5: Write Performance Descriptors for Each Rating (Step 3 + Step 4)
This step is basically to mix the outcomes of Steps 3 and 4. Write descriptions of expected performance at each level to finalize the rubric. Your descriptions need to be 1) observable and measurable behaviors, 2) use parallel language across the scale, and 3) indicate the degree to which the standards are met. It is recommended to write the performance descriptor of the highest rate first. Writing those for lower rates will become easy.
Example:
For the expected student outcome related to "well-formatted tables, graphs, and photographs," there are three criteria identified. Each criterion needs three levels of performance descriptors.
1) Use software such as Excel to construct graphs and tables;
Exceed expectations (3): Construct all figures and tables using software such as Excel.
Meet expectations (2): Use software such as Excel for at least one figure or table.
Below expectations (1): Rely on handwriting to construct graphs and tables.
2) Tables, graphs, and photographs are “stand-alone” with useful captions;
Exceed expectations (3): Use completed captions for all figures (figure number, title, axis title, legend, etc.), tables (table number, title, etc.), and photographs (figure number, title, size bar, etc.).
Meet expectations (2): Use captions for tables, graphs, and photographs, but they are incomplete.
Below expectations (1): Fail to use any captions for tables, graphs, and photographs.
3) Indicate the units of measure (e.g. newtons) for the quantitative data.
Exceed expectations (3): Indicate units for all the quantitative data in tables, graphs, and photographs.
Meet expectations (2): Indicate units occasionally for the quantitative data in tables, graphs, and photographs.
Below expectations (1): Fail to indicate any units for the quantitative data in tables, graphs, and photographs.
Step 6: Build and Revise the Rubric
Steps 1 to 5 can be repeated for other student outcomes. Construct a table-formatted rubric for single or multiple student outcomes to a single page for reading and grading ease. After getting feedback from colleagues, teaching assistants, or students, consider the effectiveness of the rubric and revise accordingly.
Example:
The expected student outcomes are expanded to "Students can analyze lab data using appropriate methods" and "Students can establish solid and consistent control of conventions for a technical audience."
Holistic Engineering Lab Writing Rubric Example
A rubric with holistic description of high, medium, and low performance is provided here as another approach you might take.
A holistic rubric from a 300-level electrical engineering lab (the Digital Devices and Logic Design lab of the Mississippi State University) by Powe and Moorehead (2006)
Conducting assessment (prepared for teaching assistants or group grading)
When the labs have graders separately from lab instructors who provide the assessment pieces (rubrics), the lab report graders need to use them accurately. The following "norming" process will assist the lab report graders in conducting grading more accurately.
Step 1: Before the first lab begins, the lab instructor explains the lab assignment and the lab assessment rubric to the lab graders to ensure they understand them well.
Step 2: The lab instructor provides a few (three to five) student samples to the grader. The samples need to be carefully chosen with a wide range. It is recommended to provide one best kind, one worst kind, and one average, at least.
Step 2: The lab instructor and the grader conduct sample lab report grading and feedback using the rubric.
Step 3: The grading results from the lab instructor and the grader are compared. The similarities and differences can be discussed prior to deliberations.
Step 4: It is preferred that the lab instructor and the grader agree on scores and feedback for the samples. If the lab report grader's assessment results are close to those from the lab instructor, the grader is ready for grading.
Step 5: If the scores and feedback from the lab report grader are significantly different from the lab instructor's, the lab report grader did not understand or rely on the rubric during grading. Often, the lab report grader might understand an assignment's expectations differently. Alternatively, it could be that the rubric is not adequately clear. The lab instructor can update the rubric. Repeat Steps 1 to 4.
Resources
Designing Grading Rubrics, Brown University, https://www.brown.edu/sheridan/teaching-learning-resources/teaching-resources/course-design/classroom-assessment/grading-criteria/designing-rubrics
Assessment: What is a Rubric?, DePaul University’s Office for Teaching, Learning and Assessment, https://resources.depaul.edu/teaching-commons/teaching-guides/feedback-grading/rubrics/Pages/default.aspx
Dannelle D. Stevens & Antonia J. Levi, An Introduction to Rubrics (Sterling, VA: Stylus, 2005).
Powe, A., & Moorhead, J. (2006, June), Grading Lab Reports Effectively: Using Rubrics Developed Collaboratively By Ece And Technical Writing Instructors Paper presented at 2006 Annual Conference & Exposition, Chicago, Illinois. 10.18260/1-2--856
Running a Grade-Norming Session, https://wac.umn.edu/tww-program/teaching-resources/grade-norming-session#:~:text=During%20the%20Norming%20Session%3A,minutes%20or%20so%20per%20sample.
CoP Meeting Feedback - Assessment Rubric Design:
1. Images are small
2. Comments for tables are misplaced
3. It might be good to provide features to implement tables/information/rubrics directly into Canvas
4. It is known that some other instructors are not comfortable using rubrics as provided in the module, with the thought that some students may use the rubric to complete assignments that provide only the details listed in the rubric, without taking other requirements onto account
5. Introduction could be improved by addressing common problems in addressing expectations to students
6. The challenge of the courses should not be figuring out the task, but instead completing the task
7. The four levels of performance in rubrics are better when compared to the three performance category rubrics
8. One of the links does not work in the module, faculty development reference link
9. Links should be avoided in the module, as links change and information could be lost
10. IMRDC is used in the module, but not initially defined
11. More rubric examples, specifically a holistic rubric example would be nice
The following comments were moved from Assignment Design - JL
Images of example rubrics in Step 2 are very small and also don’t contain alt text. Would be better implemented as an embedded table and not a picture of a table, if emphasis was able to be retained.
This link on this page “Creating a Rubric: an Online Tutorial for Faculty, the Center for Faculty Development at the University of Colorado Denver, https://www.ucdenver.edu/centers/cfda“ does not take you to the online tutorial, nor is it obvious from this page how to find the tutorial.
I have found four levels of performance to be preferable to three. A “nearly meets expectations” tells me more than simply “does not meet”
Step 4 has a display error—heading is showing up below feedback.
Higher level issue—this discusses implementing a print-oriented rubric that would be on a piece of paper or a word document. Could this discuss how to implement a rubric directly into a CMS? (it can be a pain to move a Word document rubric into Canvas)
Also high level—introduction could be improved by addressing common objections or concerns about using rubrics, and doing a little more persuasive work on why these are a good idea. (Address students min-maxing grade and why transparent and concrete expectations are good. The more consistent your grading system is, the clearer a path student has to meeting or exceeding your expectations.)
Note that you don’t have to strictly grade inside your boxes—you can give people halfway marks
Link #3 is not useful and can be omitted (see note above)
Define “IMRDC” acronym upon first use on the website or hyperlink it to a more detailed explanation
Can there be more rubric examples, including analytical and holistic
Is it worth including blank templates of several sizes? A generic template plus a couple examples of it being filled out?
Some pages have a “download this as a pdf” button—should this site have one?
An example with 9 outcomes may be misleading – this would be overwhelming in sophomore labs ---- but this is a comprehensive list, so it is good to include, but not as an example for a single assignment (making it clear that actual rubrics should simpler/fewer)
Include example of 2-level rubric (good enough or not) - there is no “excellent” way to flip a light switch – either you can or cannot achieve it.