2.27 Qualitative methods

We covered creating transcripts, participatory action research, and focus groups this evening.  It was primarily an interactive class where you looked at primary data in light of the different practices we discussed.

For next week

Read:  Review - ethnographic, case study + phenomenological approaches in Mertens, Ch 8 Qualitative Methods; Perl (1976), 17;   Heath (1983) pdf;  Young, pdf

Write: code the sample transcript 2 (to the left).  Name features of the context, the actor(s), actions, interactions, conditions and outcomes - and then see what you can come up with in terms of patterns and relationships that you might spin out into theory.  

An Allegory for Writing Studies Majors

The Fantastic Flying Books of Mr. Morris Lessmore

2.13 Surveys + voice, discourse and authority

Introduction to surveys
You did a great job applying Merten's survey discussion to a particular problem: how to gather information about who comes to the writing center and who does not.

As illustrated by your three different approaches to this task = the kind of survey you design (and how it will work) depends on how you define your purpose, and the objectives you set for yourself  Each group defined a purpose relevant to discovering who came to the writing center and who didn't:  group one set forward to characterize who came and who didn't;  group two sought to differentiate among and compare participants reasons for attending/not attending; and group three sought to categorize reasons for attending/not attending.  Each of these studies would provide information relevant to the general task (gathering information), and each survey would produce different information relevant to the goals of figuring out who and how to reach out to in terms of  writing center support.

Studies that characterize populations can often serve as the basis for the design of larger surveys or other kinds of research.  For example, group one planned to use a small, open-ended survey to develop questions for a more multiple-choice based survey. Also - conducting a pilot with the purpose for the second study - could help create the "final' version for a larger survey.  Comparative survey would draw from design principles from causal comparison/correlation studies, and it might take several "tests" to ensure the information meets both criteria. This survey might - in some ways - work as a "test" for the classification study.

Anderson et al.
As you talked about the purpose & objectives, and thought about how to make decisions about sampling and delivery method - you encountered versions of the problems described in Anderson et al.  As this essay made clear, it is important to have a strong theoriteical & practical ground for your study.  This means establishing a clear purpose and objectives - and conducting a cycle of proposing and testing sampling and delivery methods that meet that purpose and objectives.  Should you decide to use a survey as a primary method, the form of Anderson et al could work as a useful model for your essay.  It includes sections for theoretical review of literature relevant to the study's design, a clear statement of the purpose and objectives, discussion of shortcomings in design and execution,  presentation of findings; discussion of findings, and conclusions.

I  felt more extensive use of tables to represent data would have made this essay a little less difficult to read.  The narrative presentation of numbers (both percentages & numbers) required readers to create + hold correlations in their heads - rather than "seeing" them in a table.  I think use of tables would also have strengthened the essays discussion of correlations across categores - which is weak or mostly missing except in the case of differences in demographics (teaching rank) and practices.

Uses for surveys even when it is not your primary method
While most of you indicated that you survey research was not at the top of your list as a method - you might find a short, exploratory survey as a usful tool for locating participants, getting participant input on how to design your study;  receiving feedback from participants on your proposed plan for your study; receiving feedback from participants on their experience of your study; and so on.  

Discussion of Bartholomae, Royster, and Elbow
This talk presented three different perspectives on issues associated with voice, discourse and authority in writing.

Bartholomae's essay emphasizes the larger discursive features of written work - their basis for authority (in personal experience versus positioning the author as "arguing against" a "weaker" - though established - idea); their level of abstraction + generalization (successful academic essays are idea based); what "counts" as evidence (while personal experience is not excluded, connections to the network of scholars and ideas of the discipline is valued more highly), and how evidence is presented (through fully supported, logical movement between particular (authoritative) examples  and appropriate generalization).  He points out that freshman writers are expected to write in this form BEFORE they have sufficient experience (with disciplinary knowledge, with the "forms" and with voice) to do so.  He points out that this results in essays that are approximations of the appropriate 'voice' and form, and that surface features (grammar, syntax, spelling) are not necessarily signals of student writer's movement toward mastery of academic discourse.

Royster's contempation of subjectivity took on additional issues confronted by nonmainstream writers as they step into academic discourse.  Her point was that nonmainstream subjects are "theorized" or interpreted in terms of a set of assumptions that limit mainstream readers interpretation of nonmainstream subjectivities.  Her three scenes show mainstream "readers" with assumptions that empower them (mainstream readers) to: 1) speak on behalf of nonmainstream writers/readers; 2) theorize nonmainstream writers/speakers/thinkers in ways that erases and overwrites historical evidence conflicting with histories that place mainstream culture at history's center; 3) essentialize nonmainstream writers as having a single, usually "outsider" voice, rather than allowing that nonmainstream writers connect to multiple nonmainstream and mainstream identities.

Elbow's discussion takes up the task of teaching "the language of power" in classrooms.  In contrast to Bartholomae, his focus was on the different surface issues that shape SWE and home discourse, and on teaching "copyediting"  so that students can move from home language to SWE.  His intent was to provide a respectful and "safe" place for students to think in home language, without ignoring the fact that unless students can step into the forms of SWE, they will be denied access to many opportunities where SWE is required.

Taken together, these three essays present questions about authority, discourse and power that the discipline has not yet fully answered - and which we didn't have enough time to take up.  We will certainly come back to these ideas.

As noted on the calendar, there is no class next week.
For February 27 

Read:  Ch 8 Qualitative Methods with attention to grounded theory, participatory research, focus groups;  Castillo & Chandler (in press), pdf;  Kean University Writing Center Focus group reports.

The pdfs for focus group reports and the Castillo & Chandler article  will be posted by February 18.

See you in a couple weeks>

 .

2.6 Causal Comparative and Correlational studies

Causal comparison and correlation studies. As discussed in Mertens, causal comparative studies are necessary when there is a need to characteristics which it is either not possible, or unethical to manipulate.  Her examples in the book focused on studies of learning and behavioral differences associated with gender, disability and ethnicity.  In writing studies - or in studies where we want to learn about teaching and learning, it may be necessary to use causal comparative or correlational methods to study groups of writers or certain practices or experiences associated with writing or teaching of writing.

I drew attention to the list of reasons for misperceptions of differences among groups ( p. 154) - but re-cast it as a list of caveats for the design and implementation of comparative studies in general.  

We then walked through design of a causal comparative study to look for correlations between feedback = independent variable (in terms of grades or conferencing/comments) and revision = dependent variable.  We ran into lots of difficulties with too many variables and figured out strategies to eliminate some of them (such as only comparing courses with the same level students + same focus). 

Applying this method.  You then used  a modified/hypothetical version of Merten's "steps"(see below) to design causal comparative/correlational studies for (possible) projects by Tim, Neiha, and Wayne.

1. Identify the research problem.

2. Identify the independent or predictor, dependent or criterion variables, and other relevant factors.

3. Identify participants (groups).

4. Describe methods for collecting data 

5. Reflect on possible sources for bias.
6. Pose (anticipated) competing explanations of (anticipated) findings.

I think I kind of sabotaged these discussions - turning everything into ethnographic/case study research - but then I guess that just goes to show the "methods" bias I carry with me.  I think - as you developed them - there were grounds for doing each study as causal comparative, though you would want to put some more time into the organization + design. So put those ideas into your bag of tricks = and keep them in mind as we work our way through the other methods.  

  

Sommers & Brodkey
Simone's and Musheerah's essays are posted to the right.  While our discussion did not focus on the study design - these two particular studies were selected because they "compared" two groups. In fact, each was a case study, though Sommers' piece comes closest to being causal comparative.

Nancy Sommers:
Using our list above, we might think about Sommers' essay as causal comparative in the following ways.
1. Research problem = how does revision correlate with experience writing?
2. Independent variable = writer's experience, dependent variable = revision process
3. Participants = inexperienced + experienced writers from similar geographic  areas (defined in essay) = (also  similar in terms of first language, socioeconoic, ethnic, gender, class distribution? what other similarities might be important.  In other words, what differences within the groups could confuse the data?)
4. Method: Participants wrote three papers (expressive, explanitory, persuasive),and revised each essay twice.  Each participant "was interviewed three times after the final revision of each essay" Essays were coded to identify process + writers' concerns.
5 & 6. Possible sources for bias + competing explanations: "post hoc" fallacy = "proving" the assumption that writers need to be "taught' to revise (novice writers do what they are "taught") => an alternative explanation might be that the younger writers were "developmentally" at a place where thinking processes might not favor revision (think about Bizzell's essay about Perry); this study is especially vulnerable to that since it does not control for age.  This is a much a study of older v younger writers as it is of experienced v inexperienced writers.  Also - as pointed out in class - we noted that the experienced writers were not "taught" to revise - since they (probably) had similar experiences in school - so the conclusion about changing teaching methods - while it may be true - is not relevant to how the experienced writers learned how to revise, since the study does not really examine the process through which experienced writers learned to revise.

Linda Brodkey .  
Brodkey's study does not fit quite so well as a causal comparative study - though it does look at positioning by identity (teacher-v student) within academic discourse = where teachers control the talk and have the power, and students receive talk and have considerably less power.

The points that most interested me were raised in our discussion of Brodkey's conclusions - that the power differential in the literacy letters exchanged between teachers and returning adult students could be accounted for both in terms of the teacherly discourse -and teachers ( unconsciously) not wanting to give up their power - and class.  In our discussion, we raised questions about whether this dynamic was "only" from class. We raised questions about student agency (especially because these were adult students) in terms of determining/following through with topics, and wondered about alternative explanations for why the exchanges turned out as they did.  We speculated about whether there might have been very different explanations about how/why the letter exchanges went as they did had Brodkey ASKED the participants what happened, or why they responded as they did.

We also noticed that though Brodky presented this as about equalizing power relationships in teacher discourse, she did not make any moves to equalize power within researcher discourse.  That is, she held full power in terms of interpretive authority.  Neither student nor teacher participants were asked for their input or interpretations.

Great class tonight.  Your applications of the methods opened new questions and - hopefully- posed possibilities for you as you approach your research.

For next week:
Read: Mertens, Ch 6: Survey methods; Anderson et al (2006, pdf on Course Blog;  Bartholomae (1985), 523;  Royster (1996), 555; Elbow (1999), 641.

As I mentioned in class, the Anderson piece will illustrate many issues that come up with surveys, and the Bartholomae, Royster and Elbow will take up issues in teaching writing as they overlap with discourse.

Wrong email and article on group brainstorming

In case you haven't already noticed, the course email posted on the syllabus is incorrect.  I transposed the numbers.

It should be:  ENG5002.01@gmail.com

Also - the essay I mentioned the first night of class which discussed research on group brainstorming was "Groupthink: The brainstorming myth," by Jonah Lehrer and it was in the January 30, 2012 issue of The New Yorker, on page22.