From Teaching Open Source

Extremely rough, rambly, informal, repetitive, etc. Release early, release often; what's confusing, what holds gems of promise, what should be cut out, what's too complicated, what would you do? Mel Chua 22:29, 14 October 2011 (UTC)

This proposal is for the NSF Graduate Research Fellowship, and I will be submitting it for the discipline of Engineering Education. It may be useful to look at my research statement and statement of purpose from graduate school to get an idea of where I'm coming from with this.

Contents 1 Framing questions 2 Research preferences 3 Current ideas 3.1 Proposal: Faculty reactions to public course ethnographies 3.2 Proposal: Academic perceptions of open source culture 3.3 Proposal: A comparison of hackerspaces and innovation centers 4 More contextual notes 5 Lit review in progress 5.1 Frameworks 5.2 Open source in academic disciplines 5.3 Research methods 6 Thank you

[edit] Framing questions

To give you an idea of the intellectual space I'm wandering in, here are some questions I've been asking and some thoughts I've been thinking:

• The idea of "exposing discourse" - conversations between engineers about working on engineering projects is typically not something available to non-engineers to overhear, which has implications for...
  • ...access, because non-privileged groups (especially groups already underrepresented in engineering) don't get exposure to the "language" of engineering before they begin formal studies and are immediately expected to "speak" it fluently.
  • ...cross-disciplinary work, because even within the realm of engineering, engineers from one discipline have little opportunity to "overhear" conversations from another discipline (and thus intuit how to work successfully with those other engineers). We don't give people a chance to watch the swimmers from the shore before dumping them in the deep end of the ocean.
  • ...introspection and self-awareness, because engineers do not accurately remember their own conversations later and therefore cannot analyze them successfully.
• Academics can be taught how to quickly get a sense (legitimate peripheral participation) of particular open source communities by finding and analyzing artifacts produced by those communities. How can we get a similar sense of "what's happening" for an academic community (in particular, an active course or research group)?
• How do the cultures of academia (specifically undergraduate education) and open source differ? How are they similar?
• What emotional and psychological processes do professors go through while working to bridge an academic and an open source community? What makes that bridging action scary? Motivating?

[edit] Research preferences

I know that I prefer...

• Qualitative research: ethnography, narratives, in-depth interviews with emergent procedures.
• A focus on faculty stories and perceptions. (Interviewing faculty as opposed to students.)
• Working across multiple institutions. (Working with professors from multiple schools at the same time.)
• Not focusing only on software engineering and computing (electrical, mechanical, chemical, etc. are also great!). However, the tools and practices I draw on to work across disciplines tend to have roots in the software engineering world.

[edit] Current ideas

Each of these is a potential research project. I'm trying to settle down on one by Halloween 2011.

[edit] Proposal: Faculty reactions to public course ethnographies

When we've worked with faculty in teaching open source, they exhibit a high degree of excitement -- but also reluctance and fear, because the open source culture of realtime radical transparency runs counter to accepted academic culture. This project is an examination of faculty interpretations of public realtime ethnography on their classes. What's a "public realtime ethnography"? Think "embedded journalist" or "liveblogger" -- the intent is not for the ethnography to be scholarly or comprehensive, but for it to be an artifact that enables discussion about the activities and discourse of an engineering classroom in a timely enough manner to affect the discourse and activities being discussed. CART, a standard transcription service frequently used as an accessibility aid at colleges and universities for deaf and hard of hearing students, will be employed as an aid in ethnography to reduce the data-collection burden on the researcher.

The main data collected will be in-depth (2-hour) interviews with a small number (~6) of faculty members, 5 interviews per professor (pre-semester, start/middle/end of semester, post-semester). Each faculty member will choose either a single class or a single research project/group of theirs to be publicly chronicled during the course of a semester. For IRB reasons, this might be a small elective class where students specifically volunteer for the "liveblogged" section, or a class from a state university that is already mandated to be open to the public. In these interviews, we will examine the feelings and motivations of professors regarding this heightened level of transparency throughout the course of the project. What is surprising? What is uncomfortable? What are their perceptions of how this practice affected and revealed the discourse within, around, and about their classroom or project?

The final scholarly product would likely look like a collection of interwoven narratives, though as a side-effect we would have a large public body of "exposed discourse" from multiple engineering institutions that could be used as future analysis. Big IRB questions exist for this project, which is a worry of mine, but we already have 2 professors (out of 6 participant slots) volunteering themselves as subjects for this research, so that's a good sign.

[edit] Proposal: Academic perceptions of open source culture

This project consists of several steps.

1. Using existing scholarly analyses of academic culture as a contrasting case, build a comparable synthesis of the elements of open source culture from existing scholarly work (in economics, anthropology, law, etc.) One early attempt at concisely describing "the open source way" is radically cross-functional, collaboratively constructed realtime transparency, but this needs to be refined and validated further.
2. Interview professors who've never participated in an open source/hardware/content/culture community (group A) about their perceptions of these open communities as a learning/project environment for undergraduate engineering students.
3. Interview professors who have participated in open communities and/or guided their students through participation in these communities (group B) about their perceptions of these open communities as a learning/project environment for undergraduate engineering students. Use these interviews to construct case studies of "experienced-with-open-source" professors and the strategies and practices they have employed to bridge an academic community with an open source one.
4. Take these group B case studies back to group A, and interview their analyses and reactions. What do they make of the case studies? Would they be interested in trying any of the techniques described? If not, why not? If so, what sort of support and scaffolding would they need?

This is good foundational work and could also serve as a recruitment mechanism for future projects.

[edit] Proposal: A comparison of hackerspaces and innovation centers

"Innovation centers" are springing up at an increasing number of college and university campuses. The concept of hackerspaces is experiencing similar growth, particularly in urban areas. Both are rapidly emerging and spreading concepts for physical locations designed to inspire opportunistic cross-disciplinary collaboration, but one is embedded in an academic culture, the other in open source culture... which gives us an opportunity to use the contrast between the two as a way to understand the differences between the open source and academic cultures, and potential collaboration points and practices that can be shared across both.

One possible probe for this is to teach 2 off-campus sections of the same undergraduate engineering course (a variant of Physical Computing). One section would be taught using the hackerspace as a classroom; the other section would be taught using the "innovation lab" of another university as a classroom. A third control section would be taught on campus as normal. In all three classes, artifacts of student work would be left in the classroom space outside of class hours. We would be looking at:

• The type of student work produced
• The level of innovation (using existing frameworks for measuring innovation) of student work produced
• Who the students collaborate with in order to produce their work - how many people help, how much do they help, and what kind of help do they give?
• Student self-efficacy, confidence, and self-assessment

There are some initial hypotheses we can test (hackerspaces will cause students to engage a larger number of people in deeper and more frequent interactions, resulting in faster iterations and a larger number of more innovative products), but other things will likely emerge during the course of gathering ethnographies of all three groups.

This research requires a participating institution that is roughly equidistant from a hackerspace and a second host university. RIT is one potential and interested candidate (with InterLOC as the hackerspace and the University of Rochester as the host university). NYU might be another (with NYCResistor as the hackerspace and Columbia as host university), or Tufts (with Artisan's Asylum as the hackerspace and MIT as host university) -- these are just examples to show the sorts of site combinations that might take place.

[edit] More contextual notes

Originally written for the first proposal, but not specific to it.

Engineering is a black box to those outside it -- and to some extent, those inside. For those involved in an engineering project, the process of creation involves a rich and delightfully messy discourse, a conversation between teammates and technology, components and constraints. We are involved in a particular sort of conversation, and you can't learn "engineering conversation" through textbook memorization or reading university brochures any more than you can learn "Italian conversation" through vocabulary memorization or reading tourist guides.

However, for those not already involved in the creative process, that's the equivalent of what they're stuck with. And even for those who are, it's easy to get so caught up in the doing of the discourse that it becomes difficult to step back and reflect upon the nature of our conversations -- do we unconsciously assume gender roles in our teams, do we remember that the design we're praising this week was the one we vehemently disagreed with last month? What happens when we start allowing ourselves and others to eavesdrop on our "ordinary" conversations -- and what are the barriers and benefits to doing so?

The barriers and benefits are actually somewhat characterized already. On the benefits side, it's been shown that rapid feedback, early exposure of and iteraton on work, a variety of perspectives, etc. lead to faster and more vivid learning, higher skill levels, and higher quality product output. It also appears to improve the confidence, self-efficacy, and communication skillsof students (though these results are more preliminary). On the barriers side, the most frequently cited is a lack of resources; I don't have time to blog or take these notes, we're too busy teaching and publishing, etc. But there's a (usually unstated) psychological discomfort as well; what happens if our discourse goes public? What does it mean for my role as a teacher when I explicitly make myself not the only source of information and feedback? How much control should I exert over the situation, and is it possible to lose it? It's a scary thing. Time and again, I've seen faculty show up during the summer all excited about "opening up" their classrooms, then lapse into silence during the school year, then reappear at conferences still excited but tempered with a wistful hope that "someday, we could do this..."

If we took away the resource barriers (i.e. someone else drove the production of that realtime ethnography), I think we'd have an easier time identifying the psychological ones. I already have experience in working with faculty to transfer practices used in open source software communities to (not necessarily software-based) classrooms; these practices, cultural elements, and tools support what we characterize as "radically pervasive, collaboratively constructed realtime transparency" in a low-resource way that is minimally disruptive to ordinary classroom activities. What we haven't looked at yet is the faculty experience through the semester once these sorts of techniques have been adapted. What does it feel like to an academic to have your work or learning conversations exposed in near-realtime?

[edit] Lit review in progress

[edit] Frameworks

• Communities of Practice (esp. Legitimate Peripheral Participation and Zone of Proximal Development) -- Wenger, Lave
• Situated Cognition -- Vygotsky, Seely Brown
• Language learning as an understanding and acculturation practice -- Krashen and others in the International Journal of Language Teaching
• Informal Learning -- readings from Monica Cardella's class
• Privilege -- bit of a blank spot here, but Alice Pawley's work along with the Zastavker-Stein-Chachra-Lynch paper as a starting point of exploration
• The history of the design thinking/studies movement as a parallel for how the open source paradigm shift might take place -- Adams

[edit] Open source in academic disciplines

• Anthropology -- Coleman, Krafft, and Martin
• Law -- Lessig and Benkler
• Economics -- Von Hippel, Hill
• Business -- Hammel (non-scholarly; "business writing")
• Education -- Wiley (and the MIT Press book "Open Education")
• Technology development -- Tiemann

[edit] Research methods

• Small number of interwoven narratives -- like "Composing A Life"
• Active interviewing -- reading some methods books and Alice Pawley's dissertation
• Ethnography -- all the methods books assigned by Susan Silbey in her Qualitative Research Methods class, Monica Cardella's dissertation, plus look at Anthropology of Open Source methods
• The use of case studies (based on ethnography) as an artifact in interviews -- business education research? (not really investigated yet)

[edit] Thank you

• Caryn Park of Tufts for urging me to identify my frameworks and for making me realize that I really, really wanted to use qualitative and exploratory methods for this research -- but was framing it very quantitatively and experimentally (thank you, engineering background) and needed to revise.
• Steve Jacobs of RIT for offering himself as a subject, for the idea to use CART/note-taking services as a supplement, and for more insight on how IRB might work (short version: it's going to be tricky) as well as drawing the connection between innovation centers (in academia) and hackerspaces (in the open source world).
• Heidi Ellis of Western New England University for insights on the fundamental work needed before ethnography takes place, as well as discourse on diversity, why some research ideas appealed more to me than others (people-connections), how to use one stage of research to recruit for another, and for offering herself as a subject as well.
• Greg Hislop of Drexel University for feedback on the relative solidity of each of the three research designs. (But Greg, hackerspaces are so shiny!)