The MMAT is a critical appraisal tool that was developed for use in systematic mixed studies reviews (i.e., reviews combining qualitative, quantitative and/or mixed methods studies).

Limitations & strenghts

• The MMAT is not a guidance for writing and reporting mixed methods studies such as GRAMMS 'Good Reporting of a Mixed Methods Study' (O'Cathain et al., 2008), and does not permit a comprehensive evaluation of mixed methods studies such as the conceptual framework proposed by O'Cathain (2010).
• Crowe and Sheppard (2011) reviewed critical appraisal tools, and found only one tool that addresses the quality of mixed methods studies: the MMAT. They suggest the MMAT is in the top-five tools with respect to the explanation of the development of the items, and the presence of a tutorial. They mentioned the content validation of the MMAT, but were not aware (at the time of the publication) that the pilot version of the MMAT was being tested for reliability.
• The MMAT has been reviewed by the National Collaborating Centre for Methods and Tools (NCCMT): See http://www.nccmt.ca/resources/search/232. E.g., this evaluation states that the MMAT 'is well suited to a public health context, particularly for questions related to complex interventions that are context-dependant and process-oriented.' 

Suggestion for using the MMAT

• Based on discussions with MMAT users, we suggest the following idea.
• Use MMAT results to create 2 groups of similar sizes (similar number of studies in each group) that you can compare (similar to a 'sensitivity analysis'): lower vs. higher MMAT studies.
• State that you included all relevant studies, some with a lower MMAT result (for different reasons) and others with a higher MMAT result, which allows a comparison between lower vs. higher MMAT studies.
• You can repeat such comparison with other study characteristics such as country, year, etc.
• This offers a rationale to compare two subsets of studies.
• If results are similar in the 2 sets, your  synthesis results will be more valid (if QUAN synthesis) or trustworthy (if QUAL synthesis).
• Finally this avoid blaming low-MMAT studies as the MMAT assessment is not an objective measure.

References     

• Crowe, M., & Sheppard, L. (2011). A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed. Journal of Clinical Epidemiology, 64(1), 79-89.

• O'Cathain, A., Murphy, E., & Nicholl, J. (2008). The quality of mixed methods studies in health services research. Journal of Health Services Research and Policy, 13(2), 92-98.

• O'Cathain, A. (2010). Assessing the quality of mixed methods research: towards a comprehensive framework. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 531-555). Thousand Oaks: Sage.

• Pluye, P., Bush, P., Macaulay, A., Khanassov, V., Queiroga, R., Loignon, C., et al. (September 2013). The Mixed Methods Appraisal Tool for assessing studies with diverse designs: Example from a systematic mixed studies review on the key processes and outcomes of Participatory Research with Health Organizations. Annual International Cochrane Colloquium, Quebec City. 

• Pluye, P. & Hong, Q.N. (2014). Combining the power of stories and the power of numbers: Mixed Methods Research and Mixed Studies Reviews. Annual Review of Public Health, 35, 29-45. 

Return to FrontPage

Visitors (since 2018-08-07):