2.25 Surveys

We had a quick talk-through Mertens chapter on surveys, and as we talked through her points - we reflected on the design of the survey to measure student growth in terms of the learning outcomes posted on the previous blog.

Mertens points out that surveys strengths are that they can handle large #s of responses, and their weakness is that they provide "self reporting" = indirect rather than direct evidence.

She identified 3 kinds of surveys(177), simple descriptive (one shot) cross-sectional (several different groups at one point in time); and longitudinal (one cohort at different points in time).  This seems straight forward enough, but when we tried to classify our study it became confusing because it measure BOTH the same group at several points in time (a cohort of writing majors when they enter and when they graduate the major) - and different groups at several points in time (students entering + exiting the major on a yearly basis).  So perhaps our data is both cross-sectional and longitudinal? And then there is the problem that it is unclear whether we are collecting data on precisely the same group.  So definitions seem so straightforward - but the devil is in the details.

In her discussion of collection methods, Mertons reviews a number of phone, electronic, and f2f approaches - and indicated that usually initial contacts with follow-ups provided the best response.  In terms of our survey = you suggested that there be direct collection (through a student worker present in classes) of data - rather than relying on instructor participation or a request for students to do the survey on their own time.  I think we will follow through on that.

Factors that Mertens identified as affecting response rates included:

• topic salience
• incentives
• length
• and timing of request

For the major survey, the last factor will probably be the most important - as we will make the request when students are in a computer lab and provide class time. 70% is generally the recomended as the acceptable response rate.

We did not discuss sampling plans in depth (183-187) but spent some time on survey instruments in terms of the major survey. Your suggestins included:

• to revise/simplify the response choices
• group questions by headings 
• changing 13 to reflect the standards
• providing pop-ups for technical language or some other means for explaining terms
• revising the directions to set up the survey's purpose more clearly, to increase student engagement, to explain/assure anonymity re the request for the ID
• revise language to reflect students' word choices

This discussion did not touch on many of the concerns raised by Mertens about sensitive questions, requests for different kinds of information, or special types of questions = but we covered a wide range of the kinds of problems that arise when the survey's designers are "different" from survey participants.

The instructions for testing your survey instrument + delivery system, and the questions for critically analyzing survey research will be particularly helpful if you choose to use this method.

Anderson et al
After the break we talked through Anderson et al's essay on Multimodality.
The essay reviewed the uses for surveys in composition studies, described methods for creating a survey for their project, reported their data, and discussed conclusions based on their data. They also   acknowledgd the study's limitations.

As pointed out in class discussion, this survey - and who chose to participate in it- suggests a snapshot of who taught multimodal composing in 2005 (mainly fulltime, tenure-track faculty at universities), how they taught and assessed it (in terms of their own assignments & their own - as opposed to programmatic or disciplinary standards), what kinds of departmental and institutional support these instructors had (pretty good at the level of making computers available - not so great in terms of professional development).  We wondered how accurate this picture was in light of their response rate + distribution (and so did they) - and in some ways the response rate + the profile of responders seems in and of itself suggests something about "what it took" to teach multimodal composing 8 years ago. 

In terms of desiging an online survey - while they outlines a useful process - we agreed that their choice to give such a long survey may have limited the survey's distribution, and in that way - its usefulness.  We observed that a series of shorter sureveys or a tree structure might have helped.

So there was much to think about here.

Royster
Heather's essay sums up Royster's main points and our discussion of her questions at the end of class could have gone on much longer.  She points out three "limitations"of dominant discursive (unconscious) assumptions about voive.  These assumptions deal with who has authority to speak, who has authority to interpret, and what constitutes "authenticity" (or indeed whether authenticity is a useful term).  These are important concepts for teachers to be mindful of as the work the boundaries that define reader-reality-audience-language within student teacher relationships.

We also talked briefly about what kind of 'research' Royster is doing - and how/where it falls within the purview of this course.  Keep that one in mind.

For next week
Read: Ch 8 Mertens = qualitative methods with a focus on grounded theory and participatory research
Perl (1976) p 17, and Castillo & Chandler (pdf)

We will begin with Vanessa's discussion of Elbow and a little more discussion of the "research methods" in these essays on teaching and difference.

Thanks for your good participation tonight - and see you next week.

Writing option student learning outcomes

Student Learning Outcome #1: students will produce essays through a series of drafts that include exploratory writing or talk, as well as revisions that include addition, deletion, substitution and rearrangement.

Student Learning Outcome #2: students will identify central ideas/themes of a text through class discussion and writing.

Student Learning Outcome #3: students will use two or more methodologies from English Studies to develop original research or creative products.

Student Learning Outcome #4: students will demonstrate ability to give a compelling oral presentation.

Student Learning Outcome #5: students will connect ideas from classroom assignments to contemporary issues in class discussion and presentations.

2.11 Finding a research topic and Causal Comparative Research

 Note: Each of you should have received an email from me with one or two suggestions for where to start looking for references for your proposed topics.

Finding a focus for a research project
You started class by discussing possibilities for your thesis projects.  You talked in partners and then we opened up ideas to the whole class.  To the right - I have posted the list of possible topics - with some suggestions for references (I do not guarantee that these are the "best" = but they are a place to start so we have something to build on at your conference.

We then looked through the writing prompts to narrow and focus your topic, and begin to think about how you might actually "do" it.  The questions are listed below. 

What exactly do you want to study with respect to this topic?

What makes this a writing studies project?

What ideas are already "out there" in terms of your idea? (At this point, don't worry about whether what you think is actually true)=> prepare to start checking out references.

What might your study add to the ideas that are already "out there"?

How and where might you gather real world "data" that could add information about your topic?

Make a quick timeline to identify the tasks you would have to accomplish and to set up a schedule for doing your project.

The conference schedule is posted below.
Tuesday, Feb 19:
2:00 Lewis; 4:00 Vanessa

Wednesday, Feb 20
2:00 Nikki, 3:00 Luis; 3:30 Maria

Thursday, Feb 21
12:30 Heather; 2:00 Jay; 3:45 Tobey; 4:00 Marie; 6:00 Rafael

I'm a little worried about Tobey and Marie's schedul (I teach at 4:30 - and between the two of them there is only 45 minutes - but we'll see.  Send me an email if you think you will need more time).

Causal Comparative and Correlational Studies.
We then did a kind of "flyover" of Merten's chapter, with me aspiring to draw attention to the most useful/important points.

Mertons introduces the reason for choosing causal comparative/correlational methosd in her introduction.
When you want to study

• inherent characteristics (that cannot be changed/manipulated)
• characteristics that should not be changed for ethical reasons
• or characteristics that could be manipulated but usually aren't (class assignments, participatory choices etc)

She also points out the difference between causal comparative (which notes whether there are connections between variables) and correlational (which notes HOW MUCH connection there is between variables) studies (152).

We came up with a list of possible comparative studies that included studies of:
Writing practices/achievment by class, economic background

Teacher feedback to gifted vs developmental writersStudent responses to reflecive writig prompts by ageWriting topics of interest (by  gender, class ethnicityLanguage patterns of liars v nonliars

She then pointed out 4 design issues that researchers need to take into consideration if they want to create reliable studies.

1. issues associated with assumptions about group differences (154-5)
2. how you define your group (156-7)
3. issues associated with homogeneity (159 => this includes a discussion of analysis of variance ANOVA as a way to develop more nuanced interpretatios of results)
4. Post hoc fallacy (the mistake of assuming that correlation implies or is coexistent with causality (161-2)
At this point we went back to the list of possible studies, and examined the kinds of issues/problems we might run into in desiging the study of teacher feedback to "gifted" vs. "developmental" writers.  This example made clear the kind of careful thinking and planning that must go into a study if you hope to gather data that can be interpreted in a reliable way.

On the Subjects of Class and Gender in the "Literacy Letters."
We then had a too short to do the essay justice discussion of Linda Brodkey's essay.  I spent some time at the beginning working out what she meant by her statemennts that "we are constituted and unified as subjects in language and discourse" and that "teachers need to learn to read relationships between writer, reader, reality and language," and it seemed to me that we had a good understanding of what she was getting at => that our identities are made out of languge, and that our identities (the Discourse that is an identity) sets us up to say particular things in particular ways.  The notion of "teacher" Discourse and the priviledge and power with respect to being the aribitor of what will and will not be said, (and whether it was said correctly), of monitoring/directing discussion by overseeing turn-taking, choice of subject, the "discourses" that will be respected (or not) and so on. Brodkey's essay explored how teachers in her study remained not only in the "subjects" defined by their profession, but also by their gender and class.  She pointed out the importance of becoming aware that class, gender, and other discourses will BE in the classroom - just as they are everywhere else, and that by virtue of the "teacher" discourse => there will be choices made about what discourses are expected of students which may or may not be justified.

We did not take a close read of the very excellent reation paper for this essay.  It is there for you as a model.

For next class (no class February 18)=> Feb 25 Read:  Mertens, Ch 6: Survey methods; Anderson et al (2006, pdf on Course Blog);  Royster (1996), 555; Elbow (1999), 641 in CT.

I will talk about surveys.  Jay will talk about the Anderson et. al study, and Heather and Vanessa will talk about "writing and difference" by responding to Royster and Elbow, respectively.

2.4 Literature review

Note:  Last week I asked to you do some writing about your research interests (prompts posted to the right), and I said we would start with it this week -but I forgot, and actually that works out just fine.  We will start next week with a discussion of some of the research interests that came up through that writing - and it can lead into a discussion of the Research Proposal. 

Reaction papers.

We started class with a presentation of my reaction paper (more on that after I go over the assignment).  My presentation was certainly a little more "teachy" than yours will be.  The approach you take to your presentations should reflect what you see as important for us to learn from your reading.  The assignments are all far enough ahead so that you should have time to run them by me if you choose.

Reacion papers are due by Saturday morning the Monday before your presentation.  This will allow me to post your paper and give your classmates enough time to read your summary + reaction as a way to help then think abou tthe reading.   The list of readings and presenters is below.  The assignment sheet for the paper + the presentation is posted to the right.

Readings for ENG 5002; Spring 2013

2/4 Berlin (1982), 235 (Chandler)

2/11 Brodkey (1989) p 621 (Chandler)

2/25 Anderson et al (2006, pdf on Course Blog); (Jay)

Royster (1996), 555 (Heather)

Elbow (1999), 641 (Vanessa)

3/4 Perl (1976), 17; (Sally)

Castillo& Chandler (2013), pdf (Nikki)

3/18 Bartholomae (1985),523 (Rafael )

Heath (1983) pdf; (Lewis)

3/25 no presentation

4/1 Hawisher& Selfe (2004), pdf (Tobey)

Brandt, pdf. (Semramis)

Exam I

4/8 Bruffee, 395; Luis

Breuch, 97 Marie

4/15 Selfe& Selfe, 739; (Maria)

Wysocki, 717  Sally

Review of Contemporary Composition: The Major Pedagogical Theories.   I set up my reaction paper by pointing out connections to the discussion of research paradigms discussed by Mertens.  I also pointed out that the essay is a literature review (of sorts) in that it discusses and critiques the major textbooks for teaching writing using each paradigm. As pointed out by Berlin, choosing a text from a particular paradigm is not about emphasizing one element of writing over another - but rather - it is about stepping in to a particular set of assumptions about "the way the world is."

I began with a review of the writer-reality-reader-language relationships for each of the four teaching paradigms: NeoAristotelian or Classisist, Positivist or Current-Traditional, Neo-Platonian or Expressionist, and New Rhetoricial.  After some discussion, I asked you to do some writing to think into teaching practices or assignments associated with each paradigm.  Unfortunately, I did not save this list - but it clearly demonstrated that you are definitely "getting" the features of each paradigm.  We then did some talking about connections between the research paradigm features and the teaching paradigm features.  We correlated the paradigms as follows:

Classicist = I can't remember what we said for this?  Marie and Heather made the point but I can't remember what they said+> was it a connection to the early pragmatists, especially the language philosophers (Peirce) and the assumption that reality must be perceived indirectly and through the social constructst that manifest it - rather than directly? 
Current traditional (post-positivist)
Expressivist = Constructivist (knowledge is assumed to be created within the idividual through the interpretation of larger social constructs)
New Rhetorical -  Transformative (especially when language is assumed as "not neutral")

The idea is not that these are "the right answers" - but that it is important to think about ideological assumptions, and to consider how they shape what we can and cannot see about our teaching and our research,

Literature Reviews:
We spent the rest of class talking about how literature review fits into research methods.  We considered both purposes for doing a research review set up in Mertens: 1) as a method for research in and of itself; 2) to explore and frame a research project. 

I won't dwell on the points of this discussion.  The book is well organized and her presentation is certainly less random that whatever I might write here, but I do want to review some of what we went over in course about strategies for finding sources.
You all know how to use data bases, and how to find the "best (most important)" sources by checking out an edited collection on your topic and then raiding the bibliography.  In addition to these foundational methods, we added these (especially since Mertens methods/lists were more relevant to psychology + education that writing studies:
1.  Check out journals relevant to your field.
http://wpacouncil.org/rcjournals
http://wac.colostate.edu/journals/
Journals on literacy and education

2. Check out web sites of appropriate professional organizations
NCTE  National Council of Teachers of English
CCCC  College Conference on Composition and Communication
IWCA  International Writing Centers Associatiojn
WPA  Writing Program Administrators

3. Cruise bibliographies/reviews compiled in your area of interest
Rebecca Moore Howard's bibliographies

4. Attend professional conferences
See post for Jan 28 = a group will definitely be going to this.

5. Talk to your peers

6.  Search Amazon as if it were you library

7. Use search specialized engines (such as google scholar, comppile)

We then did some writing/talking to think about what you might do for your thesis research - and that was about it.

For next week:
Read: Mertens, CH 5: Causal Comparative and Correlational Research (do not get overwhelmed by the math); Mertens, Appendix: Research Proposals.  CT=> Brodkey, (1989) p 621.
Finish the NIH training and send me a copy of your certificate.

Some of the readings on the list are not yet posted as links.  I will try to have them posted by next week.  If I don't make it - remind me.

Thanks for the good class and see you on Monday.