Difference between pages "Evaluation Instruments" and "Evaluation"

From TeachingOpenSource
(Difference between pages)
m (5 revisions imported)
 
(Created page with "This page contains notes about the evaluation effort associated with foss2serve. == Student Learning == Thus far we have some indirect measures of student learning. For exa...")
 
Line 1: Line 1:
This page contains notes about the evaluation effort associated with foss2serve.


=== Evaluation of Student Participation in HFOSS ===
== Student Learning ==
If you are planning on teaching a course that involves HFOSS, we would be most appreciative if you would do some evaluation of the impact of HFOSS.  You don’t have to be teaching an entire course that includes HFOSS. Feedback would be helpful for ALL courses, including those that only include an assignment or module that includes HFOSS.


The form of evaluation that we have been using is a combination of pre and post course surveys. The most recent versions are linked to this page. Given that we are using HFOSS in different courses, some of the questions need to vary between courses. Therefore, we have provided two sets of documents; two Template documents that contain questions common to all courses and two Content Question documents that contain sets of questions specific to courses:
Thus far we have some indirect measures of student learning. For example:


* [[media: StudentSurveyPreTemplateV2.1.docx | Student Survey – Pre Template V2.1]] – This document is intended to be given at the beginning of the term and contains common questions and two Likkert scale evaluation sets. These items are likely common for all courses. This document also contains a line titled: < Content Likert Question Set Here >.  This section is intended to be replaced by questions that are specific to your course from the set provided in the Content Questions – Pre V2.1 document.
<refs needed>


* [[media: StudentSurveyPostTemplateV2.1.docx | Student Survey – Post Template V2.1]] -  This document is intended to be given at the end of the term and contains the same set of questions in the Student Survey – Pre Template V2.1 document except the questions are posed in the past tense. This document also contains a line titled: < Content Likert Question Set Here >.  This section is intended to be replaced by questions that are specific to your course from the set provided in the Content Questions – Post V2.1 document.
We are currently working on obtaining direct measures of learningBroadly, these measures target:


* [[media: ContentQuestionsPreV2.1.docx | Content Questions – Pre V2.1]]  - Contains two sets of content questions for CS2 and Software courses that are intended to be inserted in the appropriate location in the Student Survey – Pre Template V2.1 document.


* [[media: ContentQuestionsPostV2.1.docx | Content Questions – Post V2.1]]  - Contains two sets of content questions for CS2 and Software courses that are intended to be inserted in the appropriate location in the Student Survey – Post Template V2.1 document.
a) learning of factual content including
* Programming
* Software development tools and processes
* Professional behavior
* Team skills


To set up the pre-course survey:
b) development of higher order thinking skills
# Open the Student Survey – Pre Template V2.1.doc
# Open the Content Questions – Pre V2.1.doc
# Copy either the “CS 2 Statement” or the “Software Engineering Statement”  from the Content Questions document to replace the “< Content Likert Question Set Here >” section in the Student Survey – Pre Template V2.1 document.


Obviously you would go through a similar process for the post-course survey.  If you feel that you need different content questions for a course that you are teaching (e.g., databases, etc.), we would be happy to work with you to define those questions.
== Student Motivation ==


All the instruments discussed here have IRB approval via the Western New England University IRB.  If you define additional content questions, we would be happy to get IRB approval at WNE for them and add them to the shared template set.  While you will still need to satisfy your own IRB, at many institutions approval by another IRB can simplify the approval process. If you are able to share your survey data, please send email it to [mailto:ellis@wne.edu Heidi Ellis]. Please also fill out the [[media: Institution_Context_Survey_V1.1.pdf| Institution Context Survey]] so that we know how to process your data.
a) Student self-report measures of attitude and motivation related to  
* Studying computing
* Majoring or minoring in computing
* Computing careers
* Use and importance of computing in other careers


Please feel free to drop the foss2serve team an email if you’d like help with the surveys.  Also, if you are able to provide examples of student work, that would also be most appreciated!
b) Student retention in computing studies
 
Examples of assembled pre and post surveys are available here:
 
* [[Media:StudentSurvey_PreV2.2.docx | Pre-course survey V 2.2]]
* [[Media:StudentSurvey_Post_V2.2.docx | Post-course survey V 2.2]]
 
=== Evaluation of Faculty Participation in POSSE ===
In addition to assessing student participation in HFOSS, the foss2serve team is also assessing faculty opinion and learning in POSSE workshops. We have two surveys
 
* [[media: POSSE_Immediate_Post-Stage-2_Participant_Survey.pdf | POSSE Immediate Post Stage 2 Survey]]  - Contains survey items to determine faculty's immediate opinions of the usefulness of POSSE after the face-to-face POSSE meeting. 
 
* [[media: POSSE_One_Year_Post-Stage-2_Participant_Survey_-_IRB_Approved.pdf | POSSE One-Year Post Stage 2 Survey]]  - Contains survey items to determine faculty's opinions of the usefulness of the POSSE experience at least one year after the POSSE face-to-face meeting. 
 
[[Category : Evaluation ]]

Revision as of 22:52, 5 February 2013

This page contains notes about the evaluation effort associated with foss2serve.

Student Learning

Thus far we have some indirect measures of student learning. For example:

<refs needed>

We are currently working on obtaining direct measures of learning. Broadly, these measures target:


a) learning of factual content including

  • Programming
  • Software development tools and processes
  • Professional behavior
  • Team skills

b) development of higher order thinking skills

Student Motivation

a) Student self-report measures of attitude and motivation related to

  • Studying computing
  • Majoring or minoring in computing
  • Computing careers
  • Use and importance of computing in other careers

b) Student retention in computing studies