The flow regulator pipeline system is an important module in LOX/kerosene engine. Through exploring the stability characteristics of the system， the direction for improving measures to reduce the amplitude of parameter oscillations is presented. Using nonlinear and small deviation linear methods， we reveal the mechanism of self-excited oscillation and obtain the bifurcation characteristics and stable boundaries of the system. Results show that the instability of the equilibrium point is a condition for the formation of self-excited oscillation， and the system gradually develops from a linear dominated divergent oscillation of 78.81 Hz to a nonlinear dominated constant amplitude oscillation of 70.01 Hz. As the pressure difference increases， the system undergoes Hopf supercritical bifurcation， and the stable region decreases accordingly. As the throttling area increases， the system exhibits Hopf subcritical bifurcation， and the stable region increases accordingly. Reducing the pipe length and increasing the pipe diameter both weaken the unstable amplitude conditions， which are beneficial for system stability. The damping hole of the regulator has little effect on the stable boundary， and reducing this aperture can significantly decrease the amplitude of self-excited oscillation. An increase in the height of the rectangular groove of the regulator can increase the stability region， resulting in complex stable boundary branches and bifurcation curve inflection points at heights of 4.5 mm and 2.5 mm， respectively. The stability of the system under the flow boundary depends on the difference of the static load curve. When working in the negative difference region， the system is unstable， and the stability region of the system at the flow boundary is larger than that at the pressure boundary.