Chapter+9+Notes

Agnes (541-551)

Enhancing the quality and Credibility of Qualitative Analysis Chapter 9 Pages 541-551


 * Alternative Criteria for Judging Quality_** the five framework discuss in chapter 9 pages 541- 551 displays the different range of criteria that can be brought to bear in judging a qualitative study. Can be also viewed as angles of vision or alternative lenses, not just for critiquing inquiry but for understanding it. Depending on the criteria. Judging quality criteria and credibility constitutes the foundation for perceptions of credibility. It is important to acknowledge at the onset of particular philosophical foundations or theoretical orientation and special purposes for qualitative inquiry generating different criteria for judging quality and credibility in research. The five contrasting sets of criteria for judging of qualitative inquiry from different perspectives and within different philosophical frameworks are as followed:


 * Traditional scientific research _** included postpositivist and realist approaches to qualitative inquiry.
 * 1) Emphasis on objectivity of the inquirer to minimizing investigator bias.
 * 2) Emphasizes rigorous systematic data collection procedures
 * 3) Cross-checking and cross-validation of sources during fieldwork to establish validity and reliability.
 * 4) Variables and hypothesis testing and striving for causal explanations and generalizability for external validity


 * Social construction and constructivist_** highlight elements of the detailed discussion and takes into account biases.
 * 1) View criteria as addressing trustworthiness
 * 2) Naturalistic inquiry should be judged by a systemic process and authenticity (reflexive awareness about one’s own perspective, appreciation for multiple perspectives of others and fairness in constructions of values)
 * 3) Doing justice to the integrity for unique cases
 * 4) Proposed that constructivist inquiry demanded different criteria from those inherited from traditional social science. (ie internal validity, transferability, external validity, dependability, reliability and conformability to objectivity).


 * Artistic and evocative _** are forms of inquiry for creative analytic practice ethnography.
 * 1) Emphasize the scientific nature of qualitative inquiry as both science and art, and mix the two designs.
 * 2) Emphasizes the artistic and evocative aspects drawn to highlight contrasts, and not equally exclusive or pure types.
 * 3) Artistic criteria focus on aesthetics, creativity, interpretive vitality, and expressive voice
 * 4) Artistically oriented qualitative analysts seek to engage those receiving the work, to connect with them, move them, provoke, and stimulate.


 * Critical change_** critical theorists set out to use research to critique society, raise consciousness, and change the balance of power in favor of those less powerful. Flow from critical theory, feminist inquiry, activist research, and participatory research a process aimed at empowerment (values-based perspective).
 * 1) Critical perspective informed by critical change, ranging from largely intellectual and research-oriented approaches that aim to expose the injustices to move activist forms of inquiry that actually engage in bringing about social change
 * 2) Identifies nature and source of inequalities and injustices
 * 3) Represents the perspectives of the less powerful
 * 4) Clear historical and values context


 * Evaluation standards and principles_** provide the foundation for the extended discussion of the qualitative evaluation application. Calls for research evaluations to be useful, practical, ethical, and accurate.
 * 1) The standards for essentially called for evaluators that have four features: utility/feasibility/ propriety/ and accuracy
 * 2) Validity/ reliability, measurability, and generalizability are dimensions that received the greatest attention in judging evaluations
 * 3) Evaluators were asked for accountable (balance)
 * 4) Systematic inquiry
 * 5) Evaluator competence
 * 6) Integrity/honesty
 * 7) Fairness
 * 8) Responsibility to the general public


 * Clouds and cotton: mixing and changing perspectives_** Criteria one choose to emphasize in their work will depend on the purpose of the inquiry, the values and perspectives of the audiences for which ones work, philosophical and methodological orientation are. Operating within any particular framework and using any specific set of criteria will invite criticism from those who judge your work from a different framework with different criteria. Mixing and combining criteria mean dealing with tensions between them.

--- Jose (552-562)

The portion of my reading from chapter 9 is on credibility, rigorous techniques for increasing the quality of data collected, the integrity of analysis, and design checks, which means to keep analysis within its context (keeping finding in context is consider a cardinal principle of qualitative analysis. But for this summary I will highlight the critical role of “triangulation.” The reason why I chose triangulation as the highlight of my summary is because it allows a researcher to overcome biases. The logic of triangulation is based on the premise that no single method ever adequately solves the problem of rival explanations. Because each method on its own reveals different aspects of pragmatic, verifiable and observed realities. In other words, in other words multiple methods, data collection and analysis provide more grain to the research mill. It has been proven that a study that uses only one method are prone to mistakes.

There are different types of triangulation but for the summary I will highlight four, and they are: All of these methods offer strategies for reducing systematic bias and distortion during data analysis. But the most important element of triangulation from my perspective is that triangulation in what ever its form, increases credibility and quality by countering concerns or accusations that its findings arrive from a single source - Martie (563-573)
 * 1) Reconciling Qualitative and Quantitative Data
 * 2) Triangulation of Qualitative data Sources
 * 3) Triangulation with Multiple Analysis
 * 4) Theory Triangulation

Design Checks: Keeping Methods and Data in Context
 * Possible source of distortion - how design decisions affect results
 * Design constraints may have affected the data
 * Sampling limitations: the situation; time periods; selectivity
 * Keeping findings in context is a cardinal principle of qualitative analysis.

High-Quality Lessons Learned
 * Lessons learned and best practices popular purposes of cross-case analysis (see Exhibit 9.3)
 * Persuasion directly related to action

The Credibility of the Researcher
 * Researcher as instrument in inquiry should include some information about researcher: experience, training, perspective, who funded the study
 * Principle to report any personal and professional information that may have affected data collection, analysis and interpretation.
 * In the Caribbean while interviewing farmers had to increase tolerance to rum since some interviews in rum shops

Considering Investigator Effects: Varieties of Reactivity
 * Report how presence of observer may have affected what was observed
 * Observers can go native and become absorbed into local culture
 * Predispositions of evaluator may affect data analysis and interpretation
 * Neutrality and impartiality expected; in contrast, constructive analysts expect to deal with reflexivity

Intellectual Rigor
 * Thread runs through discussion of credibility is intellectual rigor

The Paradigms Debate and Credibility
 * Credibility triangle: rigorous methods, credibility of researcher, philosophical belief in the value of qualitative inquiry
 * Use of qualitative methods can be controversial

Beyond the Numbers Game
 * Quantitative predictions are preferred over qualitative ones
 * Hard data is precision of statistics
 * Soft data is qualitative data
 * Must consider most appropriate design for the type of evaluation needed

- Gene (574-584) Chapter 9 notes
 * Page 574 Beyond Objectivity and subjectivity: New concepts, new language.


 * Page 574 Objectivity is traditionally considered the sine qua of the scientific method.
 * Page 575 Distance does not guarantee objectivity: It merely guarantees distance.
 * Page 576 Trustworthiness of the inquirer is one dimension of rigor.
 * Page 577 Reflections on truth and utility as criteria of quality.
 * Page 577 One paradigm- related belief that affects how people react to qualitative data involves how they think about the idea of “truth”.
 * Page 578 Nature of truth affects how one views the finding of research and evaluations.
 * Page 579 Decision makers did not expect evaluation reports to produce “truth” in any sense. Viewed findings as additional information.


 * Page 581 From Generalizations to Extrapolations and Transferability:


 * Page 581 Core principles of generalization apply to 5 principles:
 * 1) 1 The principle of proximal similarity.
 * 2) 2 The principle of heterogencity of ireelevancies.
 * 3) 3 The principle of discriminant validity.
 * 4) 4 The principle of empirical interpolation and evirapolation.
 * 5) 5 The principle of explanation.
 * Page 583 Three things to look at”
 * 1) 1 Generalizability is a chimera.
 * 2) 2 Generalizability continues to be important.
 * 3) 3 Fragile concept.


 * Page 584 The Credibility Issue in retrospect: Increased legitimaey for qualitative methods.


 * There are three concerns:
 * 1) 1 Rigorous methods
 * 2) 2 The credibility of the researcher
 * 3) 3 Phlosophical belief in the value of qualitative inquiry

Claims:


 * Page 587 All 5 support different kinds of claims.
 * Page 588 Utility of findings as a point of entry for determining what’s at stake in the claim.

Irene (589-598)--VERY ENJOYABLE reading; insightful.

Appendix 9.1 Case Study: A Documenter’s Perspective

Beth Alberty, an __internal, formative program evaluator__ of an innovative school art program, provides a reflective case study on her experience as a novice in analyzing data, a “process of moving from a mass of documentary material to a unified, holistic document.”

1. Alberty defines **//__documentation__//** as, “the interpretive reconstitution of a focal event, setting, project, or other phenomenon, based on observation and on descriptive records set in the context of guiding purposes and commitments” (p. 589). [I totally dig the concept of reconstitution—wish I could do that accurately with my own reflections.]

2. Alberty contends that the most meaningful evaluation of a program’s goals and commitments is one that is planned and carried out by the staff and that such an evaluation contributes to the program as well as to external needs for information. [Makes me think about evaluating my work experience or our own WDA program.]

3. The fact that observing (and record-keeping) //generates questions, insight, and matters for discussion// is one of the many reasons why records for any documentation should be //gathered by those who actually work in the setting//.

4. During the initial stage of documentation (collection of voluminous records), connections are made (with things she knew, with other observations, or questions she had)—New perceptions and new questions began to form. Collecting so much was a way of getting to know a new setting, of orienting herself. Even though much of the material gathered may have remained apparently unused, the first connections, the first patterns emerging from the accumulating records were a valuable aspect of the documenting process.

5. Next phase: Making sense of all the massive amounts of data collected. Alberty reread everything, revisited her purposes, and sought out original resources on documentation. She began to imagine a shape she could give her records that would make a coherent representation of the program to an outside audience.

6. Her observations and records began to take on a life of their own—having meaning through their primary relationship to the setting in which they were made, but they also began to have meaning through secondary relationships to each other (“thickened observations”). These secondary relationships also emerged from observation as a process of reflecting—reflectiveness takes time. Beginning to see the records as a body and the setting through thickened observation is a process of integrating data.

7. Integration of the documentary material becomes apparent when the documenter begins to evince a broad perspective about what is being documented, a perspective that makes what has been gathered available to others without precluding their own perceptions. The perspective the documenter offers others must evoke the constancy, coherence, and integrity of the landscape (setting), and its possibilities for changing its appearance. The process of forming a perspective in which the data gathered are integrated into an organic configuration is obviously a process of interpretation.

8. Alberty locates the documenter as participant, internal to the program or setting, gathering and shaping (interpreting) data in ways that make them available to participants and potentially to an external audience. The documenter’s obligation to interpret his or her observations and those reflected in the records being collected becomes increasingly urgent, and the interpretations become increasingly significant, as all observers in the setting become more knowledgeable about it and thus more capable of bringing range and depth to the interpretation.