The residual stress inside stock materials is a fundamental property related to the quality of manufactured parts in terms of geometric/dimensional stability and fatigue life. For large parts that must meet high-precision requirements, accurately measuring and predicting the residual stress field has been a major challenge. Existing technologies for measuring the residual stress field are either strain-based measurement methods or non-destructive methods with low efficiency and accuracy. This paper reports a new non-destructive method for inferencing the residual stress field based on deformation forces. In the proposed method, the residual stress field of a workpiece is inferred based on the characteristics of the deformation forces that reflect the overall effect of the unbalanced residual stress field after material removal operations. The relationship between deformation forces and the residual stress field is modeled based on the principle of virtual work, and the residual stress field inference problem is solved using an enforced regularization method. Theoretical verification is presented and actual experiment cases are tested, showing reliable accuracy and flexibility for large aviation structural parts. The underlying principle of the method provides an important reference for predicting and compensating workpiece deformation caused by residual stress using dynamic machining monitoring data in the context of digital and intelligent manufacturing.