This can be easily explained by the equation of Newton's second law of motion F = m.a .

Earth and any body pull each other towards themselves , the force applied on both of them remaining the same , but the changes in position of an object is determined by its velocity and acceleration . This is where the answer lies . Assume the mass of earth to be M and that of the arbitrary body to be m. Now the force on both these objects is GMm/ r^2. r being the separation between these objects . Now acceleration of these bodies

Earth - Gm / r^2

Body b - GM /r^2

If you try putting in values(as G is very small) , earth's acceleration remains a very minute ,ignorable quantity until the mass of b becomes significantly large , comparable to planetary masses . On the other hand acceleration of b is considerably high and hence we are able to observe if it's position changes w.r.t. time . If b was something as heavy as sun you would have been able to see change in earth's position as we experience earth revolving .