We propose a new scheme for adapting support vector classifiers (SVC) in non-stationary environments. This adaptive SVC (ASVC) relies in interpreting the margin and penalty factor of the SVC as a relevance measure over the samples and on an iterative re-weighted least square (IRWLS) approach for optimizing it, which resembles the RLS filtering for adaptive equalization. The ASVC capabilities are shown by means of computer experiments.