Kai Chen

If we want to evaluate the predictive ability of a logit or probit model, Kim and Skinner (2012, JAE, Measuring securities litigation risk) suggest that

A better way of comparing the predictive ability of different models is to use the Receiver Operating Characteristic, or ROC curve (e.g., Hosmer and Lemeshow, 2000, Chapter 5). This curve ‘‘plots the probability of detecting a true signal (sensitivity) and false signal (1—specificity) for the entire range of possible cutpoints’’ (p. 160, our emphasis). The area under the ROC curve (denoted AUC) provides a measure of the model’s ability to discriminate. A value of 0.5 indicates no ability to discriminate (might as well toss a coin) while a value of 1 indicates perfect ability to discriminate, so the effective range of AUC is from 0.5 to 1.0. Hosmer-Lemeshow (2000, p. 162) indicate that AUC of 0.5 indicates no discrimination, AUC of between 0.7 and 0.8 indicates acceptable discrimination, AUC of between 0.8 and 0.9 indicates excellent discrimination, and AUC greater than 0.9 is considered outstanding discrimination.

The Stata command to report AUC is as follows:

logit y x1 x2 or probit y x1 x2

lroc, nograph

The most recent edition of the book Kim and Skinner refer to is Hosmer, D. W., Jr., S. A. Lemeshow, and R. X. Sturdivant. 2013. Applied Logistic Regression. 3rd ed. Hoboken, NJ: Wiley.

A technical note from Stata: lroc requires that the current estimation results be from logistic, logit, probit, or ivprobit.

A side question: what’s the difference between logistic and logit regression? Nick Cox’s short answer is: “same thing with different emphases in reporting.” (something like one gives you the odds ratios, the other gives you the log of the odds ratios.)—thanks to a post on Stack Overflow.

Suppose we are going to calculate the skewness of 12 monthly returns. The 12 returns may be stored in a row (Figure 1) or in a column (Figure 2). This post discusses how to calculate the skewness in these two situations. Please note there are several formulae for skewness out there, which may yield different results. This post uses the formula that yields the same skewness as the Stata command sum var, detail reports.

Figure 1: Returns are stored in a row

Figure 2: Returns are stored in a column

If returns are stored in a row

Stata does not provide a command to calculate the skewness in this situation. The following Stata commands will do the job.

If returns are stored in a column

Stata provides a command to calculate skewness in this situation (egen and skewness). However, the computation is extremely slow if we have millions of observations. I would suggest calculating the skewness manually as follows:

In accounting archival research, we often take it for granted that we must do something to deal with potential outliers before we run a regression. The commonly used methods are: truncate, winsorize, studentized residuals, and Cook’s distance. I discuss in this post which Stata command to use to implement these four methods.

First of all, why and how we deal with potential outliers is perhaps one of the messiest issues that accounting researchers will encounter, because no one ever gives a definitive and satisfactory answer. In my opinion, only outliers resulting from apparent data errors should be deleted from the sample. That said, this post is not going to answer that messy question; instead, the purpose of this post is to summarize the Stata commands for commonly used methods of dealing with outliers (even if we are not sure whether these methods are appropriate—we all know that is true in accounting research!). Let’s start.

Truncate and winsorize

In my opinion, the best Stata commands to do truncate and winsorize are truncateJ and winsorizeJ written by Judson Caskey. I will save time to explain why, but simply highly recommend his work. Please see his website here.

To install these two user-written commands, you can type:

net from https://sites.google.com/site/judsoncaskey/data
net install utilities.pkg

After the installation, you can type help truncateJ or help winsorizeJ to learn how to use these two commands.

Studentized residuals

The first step is to run a regression without specifying any vce parameter in Stata (i.e., not using robust or clustered error terms). Suppose the dependent variable is y, and independent variables are x1 and x2. The first step should look like this:

regress y x1 x2

Then, use the predict command:

predict rstu if e(sample), rstudent

If the absolute value of rstu exceed certain critical values, the data point will be considered as an outlier and be deleted from the final sample. Stata’s manual indicates that “studentized residuals can be interpreted as the t statistic for testing the significance of a dummy variable equal to 1 in the observation in question and 0 elsewhere. Such a dummy variable would effectively absorb the observation and so remove its influence in determining the other coefficients in the model.” To be honest, I do not fully understand this explanation, but since rstu is a t statistics, the critical value for a traditional significance level should be applied, for example, 1.96 (or 2) for 5% significance level. That’s why in literature we often see that data points with absolute values of studentized residuals greater than 2 will be deleted. Some papers use the critical value of 3, which corresponds to 0.27% significance level, and seems to me not very reasonable.

Now use the following command to drop “outliers” based on the critical value of 2:

drop if abs(rstu) > 2

The last step is to re-run the regression, but this time we can add appropriate vce parameters to address additional issues such as heteroskedasticity:

regress y x1 x2, vce(robust), or

regress y x1 x2, vce(cl gvkey)

Cook’s distance

This method is similar to studentized residuals. We predict a specific residual, namely Cook’s distance, and then delete any data points with Cook’s distance greater than 4/N (Cook’s distance is always positive).

regress y x1 x2

predict cooksd if e(sample), cooksd

drop if cooksd > critical value

Next, re-run the regression with appropriate vce parameters:

regress y x1 x2, vce(robust), or

regress y x1 x2, vce(cl gvkey)

Lastly, I thank the authors of the following articles which I benefit from:

https://www3.nd.edu/~rwilliam/stats2/l24.pdf

https://www.stat-d.si/mz/mz16/coend16.pdf

A more formal and complete econometrics book is Belsley, D. A., E. Kuh, and R. E. Welsch. 1980. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: Wiley.

outreg is a time-saving and must-have command in Stata. It will generate a ready-for-use results table like this. I’m sure you will see what a relief this can give us.

outreg is not a built-in command and can be installed by issuing the following command:

ssc install outreg

The typical usage of outreg is:

x1, x2, x3 and x4 are independent variables that you want to report estimates for. Sometimes you may not want to report unimportant independent variables. If you include interaction terms in regressions like this:

regress y c.x1##c.x2

By the way, this is the highly recommended way to include interaction terms in regressions. If you want to report estimates for x1, x2, and x1 × x2, please use the following keep option:

keep(_cons x1 x2 c.x1#c.x2)

I list other frequently used options that control the appearance of the results table:

stats: b, se, t, or p. Usually b and the other statistics will be reported.

• • • b: coefficient estimates
    • se: standard errors of estimate
    • t: t statistics for the test of b=0
    • p: p value of t statistics

nosubstat: I do not include this option in the sample code. This option will report two selected statistics (usually b and the other one) in two columns, rather than putting one (e.g., t statistics) below the other (e.g., b).

varlabels: I do not include this option in the sample code. This option will report variable labels, rather than variable names in the generated table. Sometimes variable labels is easier to understand than variable names. In this case, you can set labels for independent variables and turn on this option.

nosubstat and varlabels can be placed anywhere after the comma.

The complete help file for this command can be found here.

There is a similar command outreg2 that you can check. But I find outreg is good enough and works really well for me.