Our study is concerned with the time-varying formation tracking problem for second-order multi-agent systems that are subject to unknown nonlinear dynamics and external disturbance, and the states of the followers form a predefined time-varying formation while tracking the state of the leader. The total uncertainty lumps the unknown nonlinear dynamics and the external disturbance, and is regarded as an extended state of the agent. To estimate the total uncertainty, we design an extended state observer (ESO). Then we propose a novel ESO based time-varying formation tracking protocol. It is proved that, under the proposed protocol, the ESO estimation error and the time-varying formation tracking error can be made arbitrarily small. An application to the target enclosing problem for multiple unmanned aerial vehicles (UAVs) verifies the effectiveness and superiority of the proposed approach.