The energy efficiency of the piston pumps is one of the considerable important factors in design and analysis of hydraulic system, especially in the process of real-time tracking of energy dissipation in a variety of loading conditions. The existing methods for obtaining the energy efficiency curve of piston pumps are either time-consuming or inaccurate. In order to quantify the energy efficiency of the piston pumps quickly and accurately, the leakage and friction energy loss caused by the clearances in the sliding pairs are analyzed, and an overall efficiency model was established, which contains two constants to be determined by two test points. The accuracy of the model was verified based on a test rig for a hydraulic pump, and it can be improved by selecting appropriate test points via the method of deviation analysis. The results show that the proposed efficiency models are in good agreement with the experimental results, and the best test points are in the range of 0–25% and 51–75% of the peak pressure of the investigated piston pump.