Advanced search
2 files | 1.34 MB Add to list

A simulation study on the probabilities of rank reversal, tie making, and tie breaking for multiple criteria decision making methods

Author
Organization
Abstract
Multiple criteria decision making (MCDM) methods can be affected by preference reversal, meaning that the order of two alternatives is reversed after adding or deleting another alternative. Here, we focus on methods that produce rankings with ties (i.e., weak orders). In this context, one usually talks about rank reversal. Existing rank reversal probability simulation experiments are subject to improvement on the following points: (1) the small number of MCDM methods included, (2) the unclear relation between the rank reversal probability and the rank of the deleted alternative, and (3) the lack of consideration of ties. In this paper, considering both strict preferences and ties, we distinguish two new phenomena: tie breaking, i.e., the shift from a tie to a strict preference, and tie making, i.e., the shift from a strict preference to a tie. To investigate the probabilities of rank reversal, tie making, and tie breaking, a simulation study involving thirty versions of twelve MCDM methods and six simulation factors is set up. Results demonstrate that for MCDM methods using pairwise comparison data, deleting an alternative ranked first or last leads to smaller probabilities than deleting an alternative in the middle, while the opposite holds for the methods using evaluation data under criteria. Four findings and three suggestions are given to help decision makers select MCDM methods to use.
Keywords
Multiple criteria decision making, Rank reversal, Rank-based probability, Tie breaking, Tie making, DATA ENVELOPMENT ANALYSIS, PROGRAMMING METHOD, WEIGHT DERIVATION, AHP, PRESERVATION, ALTERNATIVES, MODELS

Downloads

  • (...).pdf
    • full text (Published version)
    • |
    • UGent only
    • |
    • PDF
    • |
    • 860.04 KB
  • (...).pdf
    • full text (Accepted manuscript)
    • |
    • UGent only (changes to open access on 2024-07-25)
    • |
    • PDF
    • |
    • 475.47 KB

Citation

Please use this url to cite or link to this publication:

MLA
Jiang, Lisheng, et al. “A Simulation Study on the Probabilities of Rank Reversal, Tie Making, and Tie Breaking for Multiple Criteria Decision Making Methods.” OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE, vol. 125, 2024, doi:10.1016/j.omega.2023.103033.
APA
Jiang, L., Liao, H., & De Baets, B. (2024). A simulation study on the probabilities of rank reversal, tie making, and tie breaking for multiple criteria decision making methods. OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE, 125. https://doi.org/10.1016/j.omega.2023.103033
Chicago author-date
Jiang, Lisheng, Huchang Liao, and Bernard De Baets. 2024. “A Simulation Study on the Probabilities of Rank Reversal, Tie Making, and Tie Breaking for Multiple Criteria Decision Making Methods.” OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE 125. https://doi.org/10.1016/j.omega.2023.103033.
Chicago author-date (all authors)
Jiang, Lisheng, Huchang Liao, and Bernard De Baets. 2024. “A Simulation Study on the Probabilities of Rank Reversal, Tie Making, and Tie Breaking for Multiple Criteria Decision Making Methods.” OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE 125. doi:10.1016/j.omega.2023.103033.
Vancouver
1.
Jiang L, Liao H, De Baets B. A simulation study on the probabilities of rank reversal, tie making, and tie breaking for multiple criteria decision making methods. OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE. 2024;125.
IEEE
[1]
L. Jiang, H. Liao, and B. De Baets, “A simulation study on the probabilities of rank reversal, tie making, and tie breaking for multiple criteria decision making methods,” OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE, vol. 125, 2024.
@article{01HNMEVRH3FVD1QQ1FXBN6PN7A,
  abstract     = {{Multiple criteria decision making (MCDM) methods can be affected by preference reversal, meaning that the order of two alternatives is reversed after adding or deleting another alternative. Here, we focus on methods that produce rankings with ties (i.e., weak orders). In this context, one usually talks about rank reversal. Existing rank reversal probability simulation experiments are subject to improvement on the following points: (1) the small number of MCDM methods included, (2) the unclear relation between the rank reversal probability and the rank of the deleted alternative, and (3) the lack of consideration of ties. In this paper, considering both strict preferences and ties, we distinguish two new phenomena: tie breaking, i.e., the shift from a tie to a strict preference, and tie making, i.e., the shift from a strict preference to a tie. To investigate the probabilities of rank reversal, tie making, and tie breaking, a simulation study involving thirty versions of twelve MCDM methods and six simulation factors is set up. Results demonstrate that for MCDM methods using pairwise comparison data, deleting an alternative ranked first or last leads to smaller probabilities than deleting an alternative in the middle, while the opposite holds for the methods using evaluation data under criteria. Four findings and three suggestions are given to help decision makers select MCDM methods to use.}},
  articleno    = {{103033}},
  author       = {{Jiang, Lisheng and Liao, Huchang and De Baets, Bernard}},
  issn         = {{0305-0483}},
  journal      = {{OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE}},
  keywords     = {{Multiple criteria decision making,Rank reversal,Rank-based probability,Tie breaking,Tie making,DATA ENVELOPMENT ANALYSIS,PROGRAMMING METHOD,WEIGHT DERIVATION,AHP,PRESERVATION,ALTERNATIVES,MODELS}},
  language     = {{eng}},
  pages        = {{13}},
  title        = {{A simulation study on the probabilities of rank reversal, tie making, and tie breaking for multiple criteria decision making methods}},
  url          = {{http://doi.org/10.1016/j.omega.2023.103033}},
  volume       = {{125}},
  year         = {{2024}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: