Glossary of Mixed Methods Terms/Concepts

The following terms are adopted from Tashakkori and Teddlie's (2003, Handbook of mixed methods in the social and behavioral research) Glossary. Please refer to the original for more details and for further references. Cited references with a date of 2003 are chapters in that Handbook.

Complementary inference
Conceptual (or Inferential) Consistency
Conceptual Framework
Concurrent Mixed Method Design
Concurrent Mixed Model Design
Concurrent Nested Design
Concurrent Triangulation Design
Contrast Principle
Convergent Inference
Conversion Mixed Model Design
Data Consolidation
Data Conversion/Transformation
Data Quality
Deductive inference (in research cycle)
Deductive logic
Design quality
Dialectical position
Divergent inference
Ecological transferability
Ecological validity (or ecological external validity)
Effect size
External validity
Fully integrated mixed model design
Fundamental principle of mixed methods research
Generalizability
Gestalt principle
Inductive inference (in research cycle)
Inference
Inference quality
Inference transferability
Interactive model
Internal validity
Interpretive agreement (or interpretive consistency)
Interpretive distinctiveness
Measurement validity
Meta-inference (or integrated mixed inference)
Meta-interpretation
Methodological triangulated design
Mixed Method Design
Mixed methods sampling
Mixed Model Design
Monomethod design
Monostrand design
Multilevel mixed methods design
Multilevel mixed model design
Multilevel mixed sampling
Multimethods design
Multimethods QUAL study
Multimethods QUAN study
Multiple methods design
Multistrands design
Multitrait-multimethod matrix
Operational transferability
Paradigm
Parallel mixed model design
Population transferability
Population validity (or population external validity)
Qualitizing
Reliability (data reliability or measurement reliability): See data quality.
Rules of integration
Sequential explanatory design
Sequential exploratory design
Sequential mixed method design
Sequential mixed model design
Similarity-contrast principle
Single phase design
Stage (of a study)
Strand
Supervenience theory
Temporal transferability
Temporal validity (or temporal external validity): See inference transferability.
Transferability
Transformative-emancipatory perspective
Transformative mixed methods design
Triangulation

Two Phase Design
Validity (measurement validity or data validity): See data quality.
Validity (design validity or inferential validity): See inference quality.
Within-Design consistency

Complementary inference: This is when the results of two strands of a mixed methods study provide two different but non-conflicting conclusions or interpretations.

Conceptual (or Inferential) Consistency: This refers to the degree to which the inferences are consistent with each other and with the known state of knowledge and theory.

Conceptual Framework: This is a consistent and comprehensive theoretical framework emerging from an inductive integration of previous literature, theories, and other pertinent information. Conceptual framework is usually the basis for reframing the research questions and for formulating hypotheses or making informal tentative predictions about the possible outcome of the study.

Concurrent Mixed Method Design: This is a multistrand design in which both QUAL and QUAN data are collected and analyzed to answer a single type of research question (either QUAL or QUAN). The final inferences are based on both data analysis results. The two types of data are collected independently at the same time or with a time lag.

Concurrent Mixed Model Design: This is a multistrand mixed design in which there are two relatively independent strands/phases: one with QUAL questions and data collection and analysis techniques and the other with QUAN questions and data collection and analysis techniques. The inferences made on the basis of the results of each strand are pulled together to form meta-inferences at the end of the study. See also rules of integration.

Concurrent Nested Design: This is a concurrent mixed model design classified on the basis of (conceptual or paradigmatic) dominance or priority of the study. In this design, a quantitative strand/phase is embedded within a predominantly qualitative study (quan + QUAL) or vice versa (QUAN + qual). QUAL and QUAN approaches are used to “confirm, cross-validate, or corroborate findings within a single study” (Creswell, Plano Clark, Gutmann, & Hanson, 2003).

Concurrent Triangulation Design: This is a concurrent mixed model design classified on the basis of purpose of the study. In this design, QUAL and QUAN approaches are used to “confirm, cross-validate, or corroborate findings within a single study” (Creswell et al., 2003).

Contrast Principle: See similarity-contrast principle.

Convergent Inference: This is when the conclusions or interpretations of two strands of a mixed methods study are consistent with each other (i.e., agree with each other).

Conversion Mixed Model Design: This is a multistrand concurrent design in which mixing of QUAL and QUAN approaches occurs in all components/stages, with data transformed (qualitized or quantitized) and analyzed both qualitatively and quantitatively.

Data Consolidation: This means combining qualitative and quantitative data to create new or consolidated variables or data sets.

Data Conversion/Transformation: Collected quantitative data types are converted into narratives that can be analyzed qualitatively (i.e., qualitized), and/or qualitative data types are converted into numerical codes that can be statistically analyzed (i.e., quantitized).

Data Quality: This is the degree to which the collected data (results of measurement or observation) meet the standards of quality to be considered valid (trustworthy) and reliable (dependable). This term has been used by Punch (1998) to represent “quality control of data (p. 257)” “in terms of procedures in the collection of the data, and … in terms of three technical aspects of quality of the data: reliability, validity, and reactivity, p.257)” (1) Data/Measurement validity: Do the results of data collection truly represent the construct or phenomenon that they are expected to capture (measure or represent)? “How well the data represent the phenomena for which they stand (Punch, 1998, p.258).” See also convergent validity and discriminant validity. (2) Data/Measurement reliability: Do the obtained results of measurement or observation accurately reflect the magnitude, intensity, or quality of the attribute or phenomenon that is being measured or observed?

Deductive inference (in research cycle): This is a process in which hypotheses or predictions are formed on the basis of (1) a conceptual framework that is constructed from the literature, (2) the inferences of a previous strand of a mixed methods study, or (3) an existing theory. See also inference and inference quality.

Deductive logic: (Erzberger & Kelle, 2003) (1) This refers to the application of general rules to specific cases. For example, from the general rule that all men are mortal, it can be deduced that if Socrates is a man, then he will be mortal. (2) This refers to a type of reasoning usually applied if a link is drawn from an already formulated theoretical statement to a statement about observable empirical facts, a link that can be generalized in the following term: “If A (a theoretical statement) is true, then we would expect the fact C to happen.”

Design quality: See inference quality.

Dialectical position: (1) (Greene & Caracelli, 2003) To think dialectically is to invite the juxtaposition of opposed or contradictory ideas, to interact with the tensions invoked by these contesting arguments, or to engage in the play of ideas. The arguments and ideas that are engaged in this dialectic stance emanate from the assumptions that constitute philosophical paradigms—assumptions about the social world, social knowledge, and the purpose of science in society.

Divergent inference: (Erzberger & Kelle, 2003) This is when the inferences made on the basis of the two strands of a mixed methods study are inconsistent or dissonant (Rossman & Wilson, 1985); that is, they do not agree with each other. Inconsistencies between qualitative and quantitative findings might be a consequence of the inadequacy of the applied theoretical concepts. It might, therefore, be necessary to revise and modify the initial theoretical assumptions and to draw on further theoretical concepts that have not yet been related to the domain in question.

Ecological transferability: This refers to generalizability or applicability of inferences obtained in a study to other settings or contexts. Subumes the QUAN term ecological validity and ecological external validity, and the QUAL term transferability See inference transferability.

Ecological validity (or ecological external validity): See inference transferability.

Effect size: This refers to the intensity, magnitude, or practical significance of an obtained result (e.g., relationship, difference) in the QUAL or QUAN strands of a mixed methods study. Onwuegbuzie and Teddlie (2003) explicitly relate this historically QUAN term to QUAL research, naming several new terms, including manifest effect size, frequency (manifest) effect size, and intensity (manifest) effect size.

External validity: This is defined by Cook and Campbell (1979, p. 37) as follows: “the approximate validity with which we can infer that the presumed causal relationship can be generalized to and across alternate measures of the cause and effect and across different types of persons, settings, and times” (p. 37). See inference transferability.

Fully integrated mixed model design: This is a multistrand concurrent design in which mixing of QUAL and QUAN approaches occurs in an interactive (i.e., dynamic, reciprocal, interdependent, iterative) manner at all stages of the study. At each stage (e.g., in formulating questions), one approach (e.g., QUAL) affects the formulation of the other (e.g., QUAN). See also interactive model.

Fundamental principle of mixed methods research: Johnson and Turner (2003) define this principle as follows: “Methods should be mixed in a way that has complementary strengths and nonoverlapping weaknesses. … It involves the recognition that all methods have their limitations as well as their strengths. The fundamental principle is followed for at least three reasons: (a) to obtain convergence or corroboration of findings, (b) to eliminate or minimize key plausible alternative explanations for conclusions drawn from the research data, and (c) to elucidate the divergent aspects of a phenomenon. The fundamental principle can be applied to all stages or components of the research process.”

Generalizability: See external validity and inference transferability.

Gestalt principle: This refers to the whole or the totality. Gestalt psychology is known for the principle (among many others) stating that the whole is bigger than the sum of its parts. The Gestalt principle is applied to mixed methods ... to demonstrate that global inferences made at the end of mixed methods studies are more than the simple sum of the inferences gleaned from QUAL and QUAN strands.

Inductive inference (in research cycle): This refers to a process of creating meaningful and consistent explanations, understandings, conceptual frameworks, and/or theories by integrating (a) the current knowledge gleaned from the literature, (b) concrete observations or facts, (c) results of data analysis in a research project, and (d) findings of a previous strand of a mixed methods study (Tashakkori & Teddlie, 1998).

Inference: (1) This is an umbrella term referring to a final outcome of a study. The outcome may consist of a conclusion about, an understanding of, or an explanation for an event, a behavior, a relationship, or a case. (2) This is “a conclusion reached” where there is either (a) a “deduction from premises that are accepted as true” or (b) an induction by “deriving a conclusion from factual statements taken as evidence for the conclusion” (Angeles, 1981, p. 133). See also deductive inference (in research cycle), deductive logic, inductive inference (in research cycle), inductive logic, meta-inference (or integrated mixed inference), and retroductive inference.

Inference quality: (1) This is proposed as a mixed methods term to incorporate the QUAN term internal validity and the QUAL terms trustworthiness and credibility of interpretations (Tashakkori & Teddlie, 1998, 2003). The definition of the term is as follows: the degree to which the interpretations and conclusions made on the basis of the results meet the professional standards of rigor, trustworthiness, and acceptability as well as the degree to which alternative plausible explanations for the obtained results can be ruled out. Inference quality consists of Design Quality (within-design consistency) and Interpretive Rigor [conceptual (or inferential) consistency, interpretive agreement (or interpretive consistency), and interpretive distinctiveness].

Inference transferability: This refers to generalizability or applicability of inferences obtained in a study to other individuals or entities (see population transferability), other settings or situations(see ecological transferability), other time periods (see temporal transferability), or other methods of observation/measurement (see operational transferability). It subsumes the QUAN terms external validity and generalizability as well as the QUAL term transferability.

Interactive model: (Maxwell & Loomis, 2003) Applied to mixed methods research, this model indicates that “the different components of actual mixed methods studies are … connected in a network or web rather than a linear or cyclic sequence.”

Internal validity: See inference quality.

Interpretive agreement (or interpretive consistency): This refers to consistency of interpretations across people (e.g., consistency among scholars, consistency with participants’ construction of reality).

Interpretive distinctiveness: This is the degree to which the inferences are distinctively different from (and superior to) other possible interpretations of the results and the rival explanations are ruled out (eliminated).

Measurement validity: See data quality.

Meta-inference (or integrated mixed inference): This is an inference developed through an integration of the inferences that are obtained on the basis of QUAL and QUAN strands of a mixed methods study.

Meta-interpretation: Forthofer (2003) describes this term as follows: “Mixed methods designs are inherently more complex, and those that attempt any integration or synthesis of results across methodologies require an additional phase of “meta-interpretation.” See also inference.

Methodological triangulated design: (Morse, 2003) This is a project that is comprised of two or more subprojects, each of which exhibits methodological integrity. While complete in themselves, these projects fit to complement or enable the attainment of the overall programmatic research goals.

Mixed Method Design: (1) This is a design that includes both QUAL and QUAN data collection and analysis in parallel form (concurrent mixed method design, in which two types of data are collected and analyzed), in sequential form (sequential mixed method design, in which one type of data provides a basis for collection of another type of data), or where the data are converted (qualitized or quantitized) and analyzed again (conversion mixed method design). (2) (Bazeley, 2003) This design includes studies that “use mixed data (numerical and text) and alternative tools (statistics and text analysis) but apply the same method, for example, in developing a grounded theory.” See also Mixed Model Design.

Mixed methods sampling: (Kemper, Stringfield, & Teddlie, 2003). This is a simultaneous selection of units of study through both probability (to increase generalizability/transferability) and purposive sampling strategies (to increase inference quality).

Mixed model design: This is a design in which mixing of QUAL and QUAN approaches occurs in all stages of the study (formulation of research questions, data collection procedures and research method, and interpretation of the results to make final inferences) or across stages of the study (e.g., QUAL questions, QUAN data). In multistrand designs, either the strands are parallel (concurrent mixed model design) or sequential (sequential mixed model design, in which inferences of one strand lead to questions of the next strand) or the data are converted and analyzed again to answer different questions (conversion mixed model design).

Monomethod design: See monostrand design.

Monostrand design: These designs use a single research method or data collection technique (QUAN or QUAN) and corresponding data analysis procedures to answer research questions. They are also known as single-phase designs.

Multilevel mixed methods design: This is a design in which QUAL data are collected at one level (e.g., child), and QUAN data are collected at another level (e.g., family) in a concurrent or sequential manner to answer different aspects of the same research question. Both types of data are analyzed accordingly, and the results are used to make inferences. Because the questions and inferences all are in one approach (QUAL or QUAN), this is a predominantly QUAL or QUAN study with some added components. In practice, because research questions and the inferences that are made at the end of the study are usually both QUAL and QUAN (using mixed models), this design is not common. See also multilevel mixed model design.

Multilevel mixed model design: This is a design in which QUAL data are collected at one level (e.g., child) and QUAN data are collected at another level (e.g., family) in a concurrent or sequential manner to answer interrelated research questions with multiple approaches (QUAL and QUAN). Both types of data are analyzed accordingly, and the results are used to make multiple types of inferences (QUAL and QUAN) that are pulled together at the end of the study in the form of “global inferences.” See also multilevel mixed method design.

Multilevel mixed sampling: (Kemper et al., 2003). This is sampling strategy in which probability and purposive sampling techniques are used at different levels of the study (e.g., student, class, school, district).

Multimethods design: This refers to designs in which the research questions are answered by using two data collection procedures or two research methods, both with either the QUAL or QUAN approach. See also multimethods QUAL study and multimethods QUAN study.

Multimethods QUAL study: This refers to designs in which the research questions are answered by using two QUAL data collection procedures or two QUAL research methods.

Multimethods QUAN study: This refers to designs in which the research questions are answered by using two QUAN data collection procedures or two QUAN research methods.

Multiple methods design: (Brewer & Hunter, 2003) This refers to designs in which more than one research method or data collection and analysis technique is used to answer research questions. They include mixed methods designs (QUAL + QUAN) and multimethods designs (QUAN + QUAN or QUAL + QUAL).

Multistrands design: This refers to designs that use more than one research method or data collection procedure. See also multimethods design.

Multitrait-multimethod matrix: This is a matrix of correlations between multiple methods of measuring each of a set of attributes. The diagonal values indicate the reliability of each measure/method. The off-diagonal values indicate the convergent validity and discriminant validity of each procedure/instrument. This method was introduced by Campbell and Fiske (1959) to evaluate the quality of data obtained from measurement instruments.

Operational transferability: This is the degree to which the inferences that are made on the basis of the results of the study are generalizable to other methods of observing/measuring the entities or attributes that the inference is about. Subsumes the QUAN terms external validity of operations and operational external validity (see Ary, Jaccobs, and Razavieh, 2003).

Paradigm: (1) (Mertens, 2003). A conceptual model of a person’s worldview, complete with the assumptions that are associated with that view. (2) (Caracelli and Green, 2003) paradigms are social constructions, historically and culturally embedded discourse practices, and therefore neither inviolate nor unchanging.

Parallel mixed model design: See concurrent mixed model design.

Population transferability: This refers to generalizability or applicability of inferences obtained in a study to other individuals or entities. Subumes the QUAN term population validity and population external validity, and the QUAL term transferability See inference transferability.

Population validity (or population external validity): See inference transferability.

Qualitizing: This is the process by which quantitative data are transformed into data that can be analyzed qualitatively.

Reliability (data reliability or measurement reliability): See data quality.

Rules of integration: (Erzberger & Kelle, 2003) A set of rules can be formulated that may be helpful for drawing inferences from the results of qualitative and quantitative studies in mixed methods designs. These rules should be understood as general guidelines whose significance vary according to the research question, the empirical domain under investigation, and the specific methods employed. Erzberger and Kelle list eight rules of integration. See also inference.

Sequential explanatory design: According to Creswell et al.(2003), this design “is characterized by the collection and analysis of quantitative data followed by the collection and analysis of qualitative data. Priority is typically given to the quantitative data, and the two methods are integrated during the interpretation phase of the study.”

Sequential exploratory design: According to Creswell, et al. (2003), this design “is characterized by an initial phase of qualitative data collection and analysis, followed by a phase of quantitative data collection and analysis. Therefore, the priority is given to the qualitative aspects of the study.

Sequential Mixed Method Design: (Onwuegbuzie and Teddlie, Chapter 13, this volume.) A design in which one type of data (e.g. QUAN) provides a basis for the collection of another type of data (e.g. QUAL). It answers one type of question (QUAL or QUAN) by collecting and analyzing two types of data (QUAL and QUAN). Inferences are based on the analysis of both types of data. This term subsumes “sequential study, “two‑phase design ,” “sequential QUAL-QUAN Analysis” and “sequential QUAN-QUAL analysis”.

Sequential Mixed Model Design: A multi-strand mixed (QUAL-QUAN, or QUAN-QUAL) design in which the conclusions that are made on the basis of the results of the first strand (e.g. a QUAN phase) lead to formulation of questions, data collection, and data analysis for the next strand (e.g. a QUAL phase). The final inferences are based on the results of both strands of the study. The second strand/phase of the study is conducted to either confirm/disconfirm the inferences of the first strand, or to provide further explanation for unexpected findings of the first strand.

Similarity-Contrast principle: A principle underlying data analysis in both qualitative and quantitative research. Governs the process of grouping of units of analysis (e.g., statements, items of a test) into categories (e.g., themes, factors, components, clusters) that are similar to each other, and distinctly different from other groups of such units. Also see Convergent Validity and Discriminant Validity.

Single Phase Design: A study with either a qualitative phase or a quantitative phase. See Monostrand Design.

Stage (of a study): A step or component of a strand/phase of a mixed methods study, i.e. conceptualization, method, inference.

Strand: A phase of a mixed methods study in which a QUAL or a QUAN approach is used in the method of study, in data collection procedures, or in data analysis. Phases/stands might be concurrent (parallel, simultaneous) or sequential, or they might include conversion of one type of data to another for analysis (conversion). See also Monostrand Design and Multistrand Design.

Supervenience theory: This is a theory that holds that certain facts or properties exist and can be explained by a type of “dependency” relationship they have to each other. Thus, social facts “depend” (or supervene) on the existence of individuals but cannot be completely explained by this relation.

Temporal transferability: This refers to generalizability or applicability of inferences obtained in a study to other time periods. See Inference Transferability.

Temporal validity (or temporal external validity): See Inference Transferability.

Transferability: This term was used by Lincoln and Guba (1985, p.300) as a qualitative analogue to external validity. See Inference Transferability.

Transformative-emancipatory perspective: (Mertens, Chapter 5, this volume) Mertens described this perspective in the following way: “Transformative scholars assume that knowledge id not neutral but is influenced by human interests, that all knowledge refelcts the power and social relationships within society, and that an important purpose of knowledge construction is to help improve society: (p.4).

Transformative mixed methods design: (Creswell et al., Chapter 8, this volume) This refers to a research project that Creswell, et al. describe as follows: “In both perspective and outcomes, it is dedicated to promoting change at levels ranging from the personal to the political. Furthermore, it is possible to conduct any quantitative, qualitative, or mixed methods study with a transformative or advocacy purpose.”

Triangulation: The combinations and comparisons of multiple data sources, data collection and analysis procedures , research methods, or inferences that occur at the end of a study., Denzin (1978) used the terms data triangulation, theory triangulation and methodological triangulation. Erzberger and Udo have used the term to refer to agreement between inferences. See Rules of Integration.

Two Phase Design:. (Currall & Towler, Chapter 18, this volume) A study with a qualitative phase followed by a quantitative phase or vice-versa. See Multistrand Design.

Typology of Research Purposes: (Newman, Ridenour, Newman, & DeMarco, Chapter 6, this volume.) A systematic classification of types of purposes for conducting mixed methods research. B

Validity (Measurement Validity, Data Validity): See Data Quality.

Validity (Design Validity, Inferential Validity): See Inference Quality.

Within-Design Consistency: The consistency of the procedures of the study from which the inferences emerged (Tashakkori & Teddlie, 2003, p. 717). See also Inference Quality.