S-estimator

The goal of S-estimators is to have a simple high-breakdown regression estimator, which share the flexibility and nice asymptotic properties of M-estimators. The name "S-estimators" was chosen as they are based on estimators of scale.

We will consider estimators of scale defined by a function $\rho$, which satisfy

R1 - $\rho$ is symmetric, continuously differentiable and ${\displaystyle \rho (0)=0}$.

R2 - there exists $c>0$ such that $\rho$ is strictly increasing on ${\displaystyle [c,\infty [}$

For any sample ${\displaystyle \{r_{1},...,r_{n}\}}$ of real numbers, we define the scale estimate ${\displaystyle s(r_{1},...,r_{n})}$ as the solution of

${\textstyle {\frac {1}{n}}\sum _{i=1}^{n}\rho (r_{i}/s)=K}$,

where $K$ is the expectation value of ${\displaystyle \rho }$ for a standard normal distribution. (If there are more solutions to the above equation, then we take the one with the smallest solution for s; if there is no solution, then we put ${\displaystyle s(r_{1},...,r_{n})=0}$ .)

Definition:

Let ${\displaystyle (x_{1},y_{1}),...,(x_{n},y_{n})}$ be a sample of regression data with p-dimensional $x_{i}$. For each vector ${\displaystyle \theta }$, we obtain residuals ${\displaystyle s(r_{1}(\theta ),...,r_{n}(\theta ))}$ by solving the equation of scale above, where $\rho$ satisfy R1 and R2. The S-estimator $\theta$ is defined by

${\displaystyle {\text{minimize}}\,s(r_{1}(\theta ),...,r_{n}(\theta ))}$

and the final scale estimator is

${\displaystyle \theta =s(r_{1}(\theta ),...,r_{n}(\theta ))}$ .

^[1]

References

1. P. Rousseeuw and V. Yohai, Robust Regression by Means of S-estimators, from the book: Robust and nonlinear time series analysis, pages 256-272, 1984