This paper proposes sliding mode control of vibration in three types of single-degree-of-freedom (SDOF) fractional oscillators: the Kelvin–Voigt type, the modified Kelvin–Voigt type, and the Duffing type. The dynamical behaviors are all described by second-order differential equations involving fractional derivatives. By introducing state variables of physical significance, the differential equations of motion are transformed into noncommensurate fractional-order state equations. Fractional sliding mode surfaces are constructed and the stability of the sliding mode dynamics is proved by means of the diffusive representation and Lyapunov stability theory. Then, sliding mode control laws are designed for fractional oscillators, respectively, in cases where the bound of the external exciting force is known or unknown. Furthermore, sliding mode control laws for nonzero initialization case are designed. Finally, numerical simulations are carried out to validate the above control designs.