Learning ordinal regression in R?

Question

I'm working on a project and need resources to get me up to speed.

The dataset is around 35000 observations on 30 or so variables. About half the variables are categorical with some having many different possible values, i.e. if you split the categorical variables into dummy variables you would have a lot more than 30 variables. But still probably on the order of a couple of hundred max. (n>p).

The response we want to predict is ordinal with 5 levels (1,2,3,4,5). Predictors are a mix of continuous and categorical, about half of each. These are my thoughts/plans so far: 1. Treat the response as continuous and run vanilla linear regression. 2. Run nominal and ordinal logistic and probit regression 3. Use MARS and/or another flavor of non-linear regression

I'm familiar with linear regression. MARS is well enough described by Hastie and Tibshirani. But I'm at a loss when it comes to ordinal logit/probit, especially with so many variables and a big data set.

The r package glmnetcr seems to be my best bet so far, but the documentation hardly suffices to get me where I need to be.

Where can I go to learn more?

Answer 1

I suggest this tutorial on ordered logit: http://www.ats.ucla.edu/stat/r/dae/ologit.htm

It showcases the use of polr in the MASS package, and also explains the assumptions and how to interpret the results.

Answer 2

One fairly powerful R package for regression with an ordinal categorical response is VGAM, on the CRAN. The vignette contains some examples of ordinal regression, but admittedly I have never tried it on such a large dataset, so I cannot estimate how long it may take. You may find some additional material about VGAM on the author's page. Alternatively you could take a look at Laura Thompson's companion to Agresti's book "Categorical Data Analysis". Chapter 7 of Thompson's book describes cumulative logit models, which are frequently used with ordinal responses.

Hope this helps!

Answer 3

If you are totally unfamiliar with ordinal regression, I would try to read the Tabachnick / Fidell (http://www.pearsonhighered.com/educator/product/Using-Multivariate-Statistics-6E/0205849571.page) chapter on the topic first - while not written for R, the book is very good at conveying the general logic and the "do's" and "do nots".

As a question: What are your response catgeories exactly? If they are some sort of scale, like "good - bad" it would be ok to use a linear regression (market research does it all the time...), but if the items are more disjunct, an ordinal regression might be better. I dimly remember that some books about structural equatiotion modelling mentioned that linear regression was superior for good scales than probit - bit I cannot recall the book at the moment, sorry!

The most serious problem might be the number of dummy variables - a couple of hundred dummy variables will make the analysis slow, hard to interpret and probably unstable - are there enough cases for each dummy / dummy-combination?

Answer 4

One standard reference written from social science perspective is J Scott Long's Limited Dependent Variables book. It goes much deeper than say Tabachnik suggested in another answer: Tabachnik is a cookbook at best, with little to no explanations of the "why", and it seems like you would benefit from figuring this out in more detail that can be found in Long's book. Ordinal regression should be covered in most introductory econometrics courses (Wooldridge's Cross-Section and Panel Data is a great graduate-level book), as well as quantitative social science courses (sociology, psychology), although I would imagine that the latter will loop back to Long's book.

Given that your number of variables is wa-a-ay lower than the sample size, the R package you should be looking is probably ordinal rather than glmnetcr. Another answer mentioned that you can find this functionality in a more mainstream MASS package.