Assessment
Designing Lab Report Assessment Rubrics With Examples
Well-designed assessment instruments will help instructors inform their expectations to students and assess student lab reports fairly and efficiently. A rubric can be an excellent assessment instrument for engineering lab reports. It identifies the instructor’s expectations from an assigned lab report and then explicitly states the possible levels of achievement along a continuum (poor to excellent or novice to expert).
The rubric can be constructed for individual labs or an entire course.
Step 1: Define the purpose of the lab report assignment/assessment for which you are creating a rubric
The first step is to clarify the purpose of the assignment and identify the student’s learning outcome(s) from lab report writing.
Example:
Assume an instructor focuses on data presentation on one lab report assignment. The expected student outcome can be "Students can construct well-formatted tables, graphs, and photographs to present lab data."
Step 2: Choose a Rubric Type: Analytical vs. Holistic
Instructors need to select one out of two types of rubrics: analytical vs. holistic.
Analytical rubrics break down the components of a learning task, giving students feedback on each component. An analytic rubric offers feedback on areas of strength or weakness. Instructors can control each criterion’s relative importance. Not that if students are not experience lab report writers, it is best to not overwhelm them with too many rubrics. Using a only few writing rubrics (perhaps 3 or so) along with a few technical rubrics would be appropriate. As they gain experience, additional expectations can be introduced.
Holistic rubrics provide a broad overview of student performance and allow you to assess a learning task as a whole. A holistic rubric provides an overall sense of student performance. Instructors can save evaluation time by minimizing the number of decisions made.
Example:
In the context of entry-level engineering lab courses, an analytic rubric was chosen for further rubric development on lab data presentation.
Step 3: Define the Criteria
Instructors need to define grading criteria, which are the individual elements of a learning outcome that will be evaluated. Make a list of knowledge and skills required for the lab report assignment. They must be unambiguous, measurable. Before finalizing, eliminate any that are not critical.
Example:
The expected student outcome is "Students can construct well-formatted tables, graphs, and photographs to present lab data."
Here is the list of knowledge and skills required: 1) Use software such as Excel to construct graphs and tables; 2) Tables, graphs, and photographs are “stand-alone” with useful captions; 3) Indicate the units of measure (e.g. newtons) for the quantitative data.
Step 4: Design the Rating Scale
Instructors need to define the levels of quality for student performance. Most rating scales include: 3 levels (below; meet; above), 4 levels (fail; fair; pass; exceed), 5 levels (never; sometimes; usually; mostly; always), or 6 levels (limited-low; limited-high; acceptable-low; acceptable-high; proficient-low; proficient-high). More rating points can provide more detail; however, more rating points can also make grading more difficult and time-consuming.
Example:
Three levels are chosen for the rating scale. Score 1: Below expectations. Score 2: Meets expectations. Score 3: Exceeds expectations.
Step 5: Write Performance Descriptors for Each Rating (Step 3 + Step 4)
This step is basically to mix the outcomes of Steps 3 and 4. Write descriptions of expected performance at each level to finalize the rubric. Your descriptions need to be 1) observable and measurable behaviors, 2) use parallel language across the scale, and 3) indicate the degree to which the standards are met. It is recommended to write the performance descriptor of the highest rate first. Writing those for lower rates will become easy.
Example:
For the expected student outcome related to "well-formatted tables, graphs, and photographs," there are three criteria identified. Each criterion needs three levels of performance descriptors.
1) Use software such as Excel to construct graphs and tables;
Exceed expectations (3): Construct all figures and tables using software such as Excel.
Meet expectations (2): Use software such as Excel for at least one figure or table.
Below expectations (1): Rely on handwriting to construct graphs and tables.
2) Tables, graphs, and photographs are “stand-alone” with useful captions;
Exceed expectations (3): Use completed captions for all figures (figure number, title, axis title, legend, etc.), tables (table number, title, etc.), and photographs (figure number, title, size bar, etc.).
Meet expectations (2): Use captions for tables, graphs, and photographs, but they are incomplete.
Below expectations (1): Fail to use any captions for tables, graphs, and photographs.
3) Indicate the units of measure (e.g. newtons) for the quantitative data.
Exceed expectations (3): Indicate units for all the quantitative data in tables, graphs, and photographs.
Meet expectations (2): Indicate units occasionally for the quantitative data in tables, graphs, and photographs.
Below expectations (1): Fail to indicate any units for the quantitative data in tables, graphs, and photographs.
Step 6: Build and Revise the Rubric
Steps 1 to 5 can be repeated for other student outcomes. Construct a table-formatted rubric for single or multiple student outcomes to a single page for reading and grading ease. After getting feedback from colleagues, teaching assistants, or students, consider the effectiveness of the rubric and revise accordingly.
Example:
The expected student outcomes are expanded to "Students can analyze lab data using appropriate methods" and "Students can establish solid and consistent control of conventions for a technical audience."
Holistic Engineering Lab Writing Rubric Example
A rubric with holistic description of high, medium, and low performance is provided here as another approach you might take.
A holistic rubric from a 300-level electrical engineering lab (the Digital Devices and Logic Design lab of the Mississippi State University) by Powe and Moorehead (2006)
Conducting assessment (prepared for teaching assistants or group grading)
When the labs have graders separately from lab instructors who provide the assessment pieces (rubrics), the lab report graders need to use them accurately. The following "norming" process will assist the lab report graders in conducting grading more accurately.
Step 1: Before the first lab begins, the lab instructor explains the lab assignment and the lab assessment rubric to the lab graders to ensure they understand them well.
Step 2: The lab instructor provides a few (three to five) student samples to the grader. The samples need to be carefully chosen with a wide range. It is recommended to provide one best kind, one worst kind, and one average, at least.
Step 2: The lab instructor and the grader conduct sample lab report grading and feedback using the rubric.
Step 3: The grading results from the lab instructor and the grader are compared. The similarities and differences can be discussed prior to deliberations.
Step 4: It is preferred that the lab instructor and the grader agree on scores and feedback for the samples. If the lab report grader's assessment results are close to those from the lab instructor, the grader is ready for grading.
Step 5: If the scores and feedback from the lab report grader are significantly different from the lab instructor's, the lab report grader did not understand or rely on the rubric during grading. Often, the lab report grader might understand an assignment's expectations differently. Alternatively, it could be that the rubric is not adequately clear. The lab instructor can update the rubric. Repeat Steps 1 to 4.
Resources
Designing Grading Rubrics, Brown University, https://www.brown.edu/sheridan/teaching-learning-resources/teaching-resources/course-design/classroom-assessment/grading-criteria/designing-rubrics
Assessment: What is a Rubric?, DePaul University’s Office for Teaching, Learning and Assessment, https://resources.depaul.edu/teaching-commons/teaching-guides/feedback-grading/rubrics/Pages/default.aspx
Dannelle D. Stevens & Antonia J. Levi, An Introduction to Rubrics (Sterling, VA: Stylus, 2005).
Powe, A., & Moorhead, J. (2006, June), Grading Lab Reports Effectively: Using Rubrics Developed Collaboratively By Ece And Technical Writing Instructors Paper presented at 2006 Annual Conference & Exposition, Chicago, Illinois. 10.18260/1-2--856
Running a Grade-Norming Session, https://wac.umn.edu/tww-program/teaching-resources/grade-norming-session#:~:text=During%20the%20Norming%20Session%3A,minutes%20or%20so%20per%20sample.