Introduction
Capturing instructional practices in Science, Technology, Mathematics, and Engineering (STEM) classrooms is critical to advance our understanding of effective teaching, identify paths to support faculty in improving their instructional practices, and to measure the impact of professional development.
In this website, we provide a new analytical tool – the COPUS Analyzer - aimed at characterizing instructional practices. This tool is based on an established observation protocol, the Classroom Observation Protocol for Undergraduate STEM (COPUS) . This observation protocol captures instructor’s and students’ behaviors that take place during 2-minute time intervals throughout a lecture. The COPUS Analyzer permits the visualization of occurrence and co-occurrence of these behaviors throughout a single or multiple classroom observations for one or several instructors. More importantly, the COPUS Analyzer automatically classifies classroom observations into specific instructional styles, called COPUS Profiles. The profiles represent a range of instructional styles, from teacher-centered to student-centered. They were developed from a data set that comprised 2,008 individual class periods collected from over 500 STEM instructors across at least 25 institutions in the United States.
Get Started!
Follow the following steps to start taking advantage of this tool:
Disclaimer: Any data submitted within this app will be saved by the research team on a secure server and used for research purposes. As such, it is critical that data uploaded be de-identified. The Institutional Review Board at Auburn University and University of Nebraska - Lincoln have approved this project.
If you have any questions regarding this tool or this project, please email either Marilyne Stains or Jordan Harshman
This material is based upon work supported by the National Science Foundation under Grant No. CAREER 1552448. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Instructions
Please watch on the video tutorial:Data Format
This tool will accept data in three formats (you can download each template at the top of this page). All are .xlsx files.
The format of the required variables are as follows:
It is very important to note is that a given instructor can have multiple classes observed on the same day or the same class observed on multiple days or semesters. In order for the COPUS Analyzer to properly account for this, the instructor names needs to be exactly the same as it appears throughout the data.
Data Upload
Bear in mind that once you upload your data, we will save a copy on our secure server for use in our research project. For this reason, make sure that the data you upload is de-identified.
Step 2On the Demographics page, fill out all the required demographics and whether you are uploading data for single or multiple observations and which format (minute-by-minute template or summary template) you have. Make sure that you are using the correct template for each format.
Step 3Once the demographics survey is completed, you will be able to click the 'Submit' button, which will reveal a new tab. By clicking the submit button, you also agree to have us use your submitted data for our studies.
Step 4After clicking the 'Submit' button, click on the 'COPUS Analyzer' tab (or the 'IRR & Training' tab if you are doing inter-rater reliability). Once on the COPUS Analyzer page, click the 'Browse' button and navigate to the file that you want to upload (excel files only).
Data Analysis
Below is a list of the analyses that you can do with this tool. Please note that depending the format of the data you upload, not all of these options will be available (i.e. if you don’t have minute-by-minute data, we can't produce a minute-by-minute heat map).
Options for minute-by-minute filesDownload Summary
button to download the summary file.
Download HM Behaviors
button to download a .png image file of the heatmap currently displayed.
Download HM Categories
button to download a .png image file of the heatmap currently displayed.
Download Report
button.
Download Aggregate
button to download a .png image file of the heatmap currently displayed.
Calculate inter-rater reliability
If you have a data set with multiple coders for the same instructor's observation(s), you can access inter-rater reliability information. Be sure to answer the question on the demographics that you are using the tool for inter-rater reliability/training. Once you upload your data, you will see two tables that look similar:
Both of these tables can be exported in a .csv format (opens in Excel).
Finally, this program will calculate percent agreement and Fleiss' kappa for the selected observation. The values will show at the bottom of the second table. These measures are good for determining how similar all the coders rated the same video. If the graph isn't big enough to fit the numbers (this can happen for classes that are longer than 50 minutes), use the slider to increase the height of the plot.
Troubleshooting
In order for this tool work effectively, please ensure the following:
If after taking these steps you find errors or wish to provide feedback regarding its functions, please contact Marilyne Stains or Jordan Harshman . If you cannot get the tool to work, please include the data you are trying to upload in your email.
Demographics
Please fill out the following fields. All fields are required before you are allowed to use the COPUS analysis tool. Click 'Submit' below when you are finished.
You may now click on 'COPUS Analyzer' tab (or the 'IRR & Training' tab if you are doing inter-rater reliability) to anlayze your COPUS data!
Abbreviations
The following codes are from the Classroom Observation Protocol for Undergraduate STEM (COPUS) paper previously published
Lec = Lecturing (presenting content, deriving mathematical results, presenting a problem solution, etc.
RtW = Real-time writing on board, doc. projector, etc. (often checked off along with Lec)
FUp = Follow-up/feedback on clicker question or activity to entire class.
PQ = Posing a non-clicker question to students (non-rhetorical).
CQ = Asking a clicker question (mark the entire time the instructor is using a clicker question, not just when first asked).
TAnQ = Listening to and answering student questions with entire class listening.
MG = Moving through class guiding ongoing student work during active learning task.
OoO = One-on-one extended discussion with one or a few individuals, not paying attention to the rest of the class (can be along with MG or TAnQ).
DV = Showing or conducting a demo, experiment, simulation, video, or animation.
Adm = Administration (assign homework, return tests, etc.).
TW = Waiting when there is an opportunity for an instructor to be interacting with or observing/listening to student or group activities and the instructor is not doing so.
TO = Other - explain in comments.
L = Listening to instructor/taking notes, etc.
Ind = Individual thinking/problem solving. Only mark when an instructor explicitly asks students to think about a clicker question or another question/problem on their own.
CG = Discuss clicker question in groups of 2 or more students.
WG = Working in groups on worksheet activity.
OG = Other assigned group activity, such as responding to indstructor question.
SAnQ = Student answering a question posed by the instructor with the rest of class listening.
SQ = Student asks question.
WC Engaged in whole class discussion by offering explanations, opinion, judgment, et. to whole class, often facilitated by instructor.
Prd = Making a predication about the outcome of a demo or experiment.
SP = Presentation by student(s)
TQ = Test or quiz
SW = Waiting (instructor late, working on fixing AV problems, instructor otherwise occupied.
SO = Other - explain in comments.
Presenting = Lec, RtW, or DV
Guiding = FUp, PQ, CQ, TAnQ, MG, OoO
Administrative = Adm
Instructor other = TW, TO
Receiving = L
Talking to class = SAnQ, SQ, WC, SP
Students working = Ind, CG, WG, OG, Prd, TQ
Student other = SW, SO