Monday, April 25, 2011

The Paradox of Inquiry: Searching for Methodological Innovations


Fisher Fuller has posted another comment.  He is certainly persistent in by demanding that I specify the criteria we used to define and select methodological innovations.  That’s challenging but fair.  It’s challenging because, as I suspect he already guessed, we did not know at the outset exactly what we meant by an “innovation in social research methods.”  I copy and paste his comment below.

Vogt did not reveal any criteria for deciding what is and is not an innovation in research methods.  He only told us that he talked to some people and informally polled others. Consensus among friends and vote counting do not add up to principles of decision making on this important issue.  I agree that Williams and Vogt discovered what seem to be some important innovations, but I disagree that they have a reliable method other researchers could use in the future to discover more.


Our method for studying research methods was more ethnographic and exploratory rather than confirmatory.  We were studying, so to speak, the culture of researchers, specifically what they believed about developing and using new research methods.  If one of our informants suggested that a particular method was an important innovation, we recorded it, reflected on it, collected it, and juxtaposed it with the beliefs other researchers shared.  Then we thought about it quite a bit about our collection of beliefs and made our choices.  That was the nub of our method.

I doubt that Fuller will be satisfied with this approach.  We asked people who we thought would know, and we paid careful attention to what they told us.  Fuller is right that we don’t really explain how we knew whom to ask and what to ask them and what kinds of answers would be satisfactory.  Obviously, Malcolm Williams and I didn’t know exactly what kinds of innovation we were going to find, but we did have some ideas—some sensitizing concepts in Herbert Blumer’s terms.

Fuller’s questions returns us to Plato’s paradox of inquiry.   Meno asks Socrates, “How will you inquire into that which you do not know?  What will you put forth as the subject of your inquiry?  And if you find what you want, how will you ever know that this is the thing which you did not know?”  While this view is sometimes described as one of Plato’s principles, Plato/Socrates clearly rejected it.  Socrates replies to Meno that we ought not to listen to this sophistical argument about the impossibility of inquiry because it will keep us from actively seeking knowledge.  Still, while Socrates dismisses the sophistical argument, he spends the rest of the dialogue trying to answer it. 

And exploratory researchers are still working on how to deal with the paradox of inquiry.  It is a perennial question, one that is always worth asking and trying to answer.  But any answer will be incomplete and temporary.  Rejecting the paradox of inquiry, as Plato points out, is necessary if we are to be active seekers after knowledge.  Plato’s argument, I think, can be read as ultimately being pragmatic.  Rejecting the paradox and forging ahead—all the while remembering the kernel of truth it contains—makes critical inquiry and the growth of knowledge possible.  Simply accepting the paradox chokes off further inquiry.  The paradox’s threat to the vigorous pursuit of knowledge is especially great in empirical and therefore partly inductive fields of inquiry such as social research. 




Monday, April 11, 2011

What is an innovation in research methods?

Fisher Fuller raised some challenging questions in a comment, copied here:

I'm suspicious of the whole project. What is a methodological innovation anyway? How do you presume to be able to identify them? Are you trying to create another set of restrictive canonical guidelines? If you can describe an innovation in a summary chapter, is it really an innovation any longer? 


These are serious questions.  Fuller is surely right that the idea of an innovative canon in research methods is almost a contradiction in terms.  If it’s canonical it chokes off innovation.

The short answer to the question of how we identified innovations in the Handbook is that we asked our colleagues, both informally in chats and more formally in some fairly extensive e-mail polling.  We had discussions with anyone who would talk with us, starting with Chris Rojek, our editor at Sage Publications who originally suggested the idea for the Handbook. 

We tried to be inclusive rather than restrictive in our understanding of what constitutes a methodological innovation in social research and we uncovered several broad classes of innovation.  Here are some examples:
  • Some innovations arise because new developments in social life and the need to develop novel ways of studying them.  An example is Mike Thelwall’s studies (in chapter 9) of human communication on the Web.  Another is Elizabeth Griffiths’ discussion of geographic information systems (in chapter 21).
  • A fertile area of innovatory growth occurs in fields that are not brand new but that are undergoing rapid development, often by generating links between separately developed methods.  One such pair of methods, developing separately and together, is multilevel modeling and structural equation modeling.  For discussions of their joint development see chapters 26 by Rex Kline and 27 by Keenan Pituch & Laura Stapleton. 
  • Combining methods is a common source of new ideas about old methods.  An illustration is chapter 13 by John Hitchcock & Bonnie Nastasi on using mixed methods for construct validation.
  • A last example is applying a new method to an area of research where it had not been used before.  For instance, Qualitative Narrative Analysis (see Ragin & Schneider’s chapter 8) is applied to program evaluation (see Vogt et al., chapter 15).
 We won’t be surprised to hear from readers who wonder how we could have missed this or that innovation or how we could have mistaken a short-lived fad for a new approach.  We welcome such discussions, questions, and criticisms because they are a key source of further innovation.

Tuesday, April 5, 2011

New Book on Innovation in Research Methods

The Sage Handbook of Innovation in Social Research Methods is now available (Sage Publications, 2011).  Edited by Malcolm Williams and Paul Vogt, it contains 27 chapters exploring innovatory methodological approaches from conceptualizing research problems through data analysis.  Further details are available at www.sagepublications.com.

My chapter* is on "Innovations in Program Evaluation: Comparative Case Studies as an Alternative to RCTs."  We argue that randomized controlled trials (RCTs) are seldom feasible when evaluating large-scale social and educational interventions.  Further, "gold-standard thinking," which assumes there is one best method for all research problems, is a mistake because it discourages thought and restricts methodological innovation. Building on our research evaluating educational programs, we show how a comparative case study approach can be an effective alternative to RCTs.  We illustrate by applying Charles Ragin's Qualitative Comparative Analysis (QCA) to the evaluation of projects in a large educational program.
      *Written with colleagues Dianne Gardener, Lynne Haeffele, and Paul Baker.

Sunday, April 3, 2011

Overview


Posts and discussions of important topics in research methods in the social and behavioral sciences

Helpful for users of Paul Vogt’s research methods texts, reference books, and other publications
Contains discussions of and supplements to those works, including
  • Dictionary of Statistics and Methodology (Sage Publications; 1st edition, 1993; 4th edition [with Burke Johnson], 2011]).
  • Quantitative Research Methods for Professionals (Allyn & Bacon, 2007).
  • When to Use What: Guidelines for Selecting Research Methods in the Social Sciences, vol 1 (Guilford Press, 2011, forthcoming).

Source for insights and new information about research methods