If we want to evaluate the predictive ability of a logit or probit model,\u00a0Kim and Skinner (2012, JAE, Measuring securities litigation risk) suggest that<\/p>\n
A better way of comparing the predictive ability of different models is to use the Receiver Operating Characteristic, or ROC curve (e.g., Hosmer and Lemeshow, 2000, Chapter 5). This curve \u2018\u2018plots the probability of detecting a true signal (sensitivity) and false signal (1\u2014speci\ufb01city) for the entire range of possible cutpoints\u2019\u2019 (p. 160, our emphasis). The area under the ROC curve (denoted AUC) provides a measure of the model\u2019s ability to discriminate. A value of 0.5 indicates no ability to discriminate (might as well toss a coin) while a value of 1 indicates perfect ability to discriminate, so the effective range of AUC is from 0.5 to 1.0.\u00a0Hosmer-Lemeshow (2000, p. 162) indicate that AUC of 0.5 indicates no discrimination, AUC of between 0.7 and 0.8 indicates acceptable discrimination, AUC of between 0.8 and 0.9 indicates excellent discrimination, and AUC greater than 0.9 is considered outstanding discrimination.<\/p><\/blockquote>\n
The Stata command to report AUC is as follows:<\/p>\n
logit y x1 x2<\/code>\u00a0orprobit y x1 x2<\/code><\/p>\n
lroc, nograph<\/code><\/p>\nThe most recent edition of the book Kim and Skinner refer to is\u00a0Hosmer, D. W., Jr., S. A. Lemeshow, and R. X. Sturdivant. 2013. Applied Logistic Regression. 3rd ed. Hoboken, NJ: Wiley.<\/p>\n
A technical note from Stata:
lroc<\/code> requires that the current estimation results be fromlogistic<\/code>,logit<\/code>,probit<\/code>, orivprobit<\/code>.<\/p>\nA side question: what’s the difference between logistic and logit regression? Nick Cox’s short answer is: “same thing with different emphases in reporting.” (something like\u00a0one gives you the odds ratios, the other gives you the log of the odds ratios.)\u2014thanks to a post<\/a> on Stack Overflow.<\/p>\n","protected":false},"excerpt":{"rendered":"
If we want to evaluate the predictive ability of a logit or probit model,\u00a0Kim and Skinner (2012, JAE, Measuring securities litigation risk) suggest that A better way of comparing the predictive ability of different models is to use the Receiver … Continue reading