According to newtons third law there is a reaction to every action. In this case the nail pushes back on the hammer just as hard as it is hit. If you account for the fact that the nail loses some of that energy into that wood. In the end the nail pushes back on the hammer with the same force minus whatever was lost into the wood.
