Ordinal regression can be performed using a generalized linear model (GLM) that fits both a coefficient vector and a set of thresholds to a dataset. Suppose one has a set of observations, represented by length-p vectors x1 through xn, with associated responsesy1 through yn, where each yi is an ordinal variable on a scale 1, ..., K. To this data, one fits a length-p coefficient vector w and a set of thresholds ?1, ..., ?K-1 with the property that ?1 < ?2 < ... < ?K-1. This set of thresholds divides the real number line into K disjoint segments, corresponding to the K response levels.
The model can now be formulated as
or, the cumulative probability of the response y being at most i is given by a function ? (the inverse link function) applied to a linear function of x. Several choices exist for ?; the logistic function
The probit version of the above model can be justified by assuming the existence of a real-valued latent variable (unobserved quantity) y*, determined by
where ? is normally distributed with zero mean and unit variance, conditioned on x. The response variable y results from an "incomplete measurement" of y*, where one only determines the interval into which y* falls:
Defining ?0 = -? and ?K = ?, the above can be summarized as y = kif and only if?k-1 < y* ?k.
From these assumptions, one can derive the conditional distribution of y as
(using the Iverson bracket[yi = k].) The log-likelihood of the ordered logit model is analogous, using the logistic function instead of ?.
In machine learning, alternatives to the latent-variable models of ordinal regression have been proposed. An early result was PRank, a variant of the perceptron algorithm that found multiple parallel hyperplanes separating the various ranks; its output is a weight vector w and a sorted vector of K-1 thresholds ?, as in the ordered logit/probit models. The prediction rule for this model is to output the smallest rank k such that wx < ?k.
Another approach is given by Rennie and Srebro, who, realizing that "even just evaluating the likelihood of a predictor is not straight-forward" in the ordered logit and ordered probit models, propose fitting ordinal regression models by adapting common loss functions from classification (such as the hinge loss and log loss) to the ordinal case.
ORCA (Ordinal Regression and Classification Algorithms) is a Octave/MATLAB framework including a wide set of ordinal regression methods.
Led Digital Marketing Efforts of Top 500 e-Retailers.
Worked with Top Brands at Leading Agencies.
Successfully Managed Over $50 million in Digital Ad Spend.
Developed Strategies and Processes that Enabled Brands to Grow During an Economic Downturn.
Taught Advanced Internet Marketing Strategies at the graduate level.
Manage research, learning and skills at defaultlogic.com. Create an account using LinkedIn to manage and organize your omni-channel knowledge. defaultlogic.com is like a shopping cart for information -- helping you to save, discuss and share.