• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!


About the MMAT

Page history last edited by Quan Nha HONG 4 years, 10 months ago

The MMAT is a critical appraisal tool that was developed for use in systematic mixed studies reviews (i.e., reviews combining qualitative, quantitative and/or mixed methods studies).


Limitations & strenghts

  • The MMAT is not a guidance for writing and reporting mixed methods studies such as GRAMMS 'Good Reporting of a Mixed Methods Study' (O'Cathain et al., 2008), and does not permit a comprehensive evaluation of mixed methods studies such as the conceptual framework proposed by O'Cathain (2010).
  • Crowe and Sheppard (2011) reviewed critical appraisal tools, and found only one tool that addresses the quality of mixed methods studies: the MMAT. They suggest the MMAT is in the top-five tools with respect to the explanation of the development of the items, and the presence of a tutorial. They mentioned the content validation of the MMAT, but were not aware (at the time of the publication) that the pilot version of the MMAT was being tested for reliability.
  • The MMAT has been reviewed by the National Collaborating Centre for Methods and Tools (NCCMT): See http://www.nccmt.ca/resources/search/232. E.g., this evaluation states that the MMAT 'is well suited to a public health context, particularly for questions related to complex interventions that are context-dependant and process-oriented.' 


Suggestion for using the MMAT

  • Based on discussions with MMAT users, we suggest the following idea.
  • Use MMAT results to create 2 groups of similar sizes (similar number of studies in each group) that you can compare (similar to a 'sensitivity analysis'): lower vs. higher MMAT studies.
  • State that you included all relevant studies, some with a lower MMAT result (for different reasons) and others with a higher MMAT result, which allows a comparison between lower vs. higher MMAT studies.
  • You can repeat such comparison with other study characteristics such as country, year, etc.
  • This offers a rationale to compare two subsets of studies.
  • If results are similar in the 2 sets, your  synthesis results will be more valid (if QUAN synthesis) or trustworthy (if QUAL synthesis).
  • Finally this avoid blaming low-MMAT studies as the MMAT assessment is not an objective measure.



  • Crowe, M., & Sheppard, L. (2011). A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed. Journal of Clinical Epidemiology, 64(1), 79-89.

  • O'Cathain, A., Murphy, E., & Nicholl, J. (2008). The quality of mixed methods studies in health services research. Journal of Health Services Research and Policy, 13(2), 92-98.

  • O'Cathain, A. (2010). Assessing the quality of mixed methods research: towards a comprehensive framework. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 531-555)Thousand Oaks: Sage.

  • Pluye, P., Bush, P., Macaulay, A., Khanassov, V., Queiroga, R., Loignon, C., et al. (September 2013). The Mixed Methods Appraisal Tool for assessing studies with diverse designs: Example from a systematic mixed studies review on the key processes and outcomes of Participatory Research with Health Organizations. Annual International Cochrane Colloquium, Quebec City. 

  • Pluye, P. & Hong, Q.N. (2014). Combining the power of stories and the power of numbers: Mixed Methods Research and Mixed Studies Reviews. Annual Review of Public Health, 35, 29-45. 



Return to FrontPage


Visitors (since 2018-08-07): 

Comments (0)

You don't have permission to comment on this page.