Think about it. Isn’t light+eyes and ears+sound just the same in terms of their “influence at a distance”? We don’t feel that as abnormal or magic - simply because we’ve sensors for them and are used to it. But physically speaking light and magnetism are based on electromagnetic forces.
The example with gravity is interesting indeed. We have only acceleration sensors behind our ears, but our body notices the pressure of the body tissue pressing down towards the gravity. And obviously, we also feel gravity when moving.
However, the difference to magnetism is, that we frankly don't have any contact with magnets during our evolution - except for the earth's magnetic field.
Even if we are able to sense it, it's definitely far from being able to reliably feel it like we do for gravity.