Wind energy is a clean and renewable source for electricity generation. To reduce the costs associated with wind power generation, development of a control methodology that maximizes the wind energy capture and mitigates the turbine fatigue loading is desired. In this paper, a new adaptive gain modified optimal torque controller (AGMOTC) for wind turbine partial load operation is presented. A gain-scheduling technique with an internal proportional integral (PI) control is developed to accelerate the controller's convergence to a reference tip speed ratio (TSR). The reference TSR is then adjusted to its optimal value in real-time through an adaptive algorithm capable of rejecting model uncertainties and estimation errors of the control gain. A fatigue mitigation method is also designed to reduce the impact of exacerbated tower bending moments due to the resonance effect. The proposed AGMOTC is evaluated based on the National Renewable Energy Laboratory (NREL) 5 MW wind turbine model using the NREL fast simulator. Simulation results have shown that the AGMOTC has improved efficiency and robustness in wind energy capture and reduced tower fatigue loading as compared to the traditional control technique.