When the bullet is fired from the gun, conservation of momentum and inertia are applicable.
The mass of the bullet is much smaller than the mass of the gun.
So, the impact of force is more visible on the bullet as its velocity increases.
The inertia of the bullet is applicable when it is moving away from the gun making the bullet travel at the same speed.