The recent demonstration of neuromorphic computing with spin-torque nano-oscillators has opened a path to energy efficient data processing. The success of this demonstration hinged on the intrinsic short-term memory of the oscillators. We extend the memory of the spin-torque nano-oscillators through time-delayed feedback. We leverage this extrinsic memory to increase the efficiency of solving pattern recognition tasks that require memory to discriminate different inputs. The large tunability of these nonlinear oscillators allows us to control and optimize the delayed-feedback memory using different operating conditions of applied current and magnetic field.