The science is on the other side of this discussion, at least for the bacteria living inside their bio-film which is what attaches them to the solid surfaces.
There are only two things to worry about here chlorine and chloramine. The first if way more deadly as in the presence of any ammonia, chloramine only puts the bacteria to sleep. On the other hand chlorine penetrates the bio-film way, way slower than chloramine.
Next, the amount of residual chlorine or chloramine in one's tap water is very low. So, given the low levels of either of these chemicals in one's tap and the minimal contact time, there is minimal risk in using straight tap water. This is further aided by how little tap water will be in the media after being rinsed and then its being completely diluted in the tank. Finally, chlorine is a gas, its tendency is to dissipate from water into the air. The biggest potential danger here is not to one's bacteria, but to one's fish or inverts. I am not sure if one rinses their media in raw chlorinated tap and returns it to the filter and then waits about 10 minutes to test the tank water that home test kits will be able to detect any level of chlorine or chloramine in the water.
All that said, it is best to be safer rather sorry. I also have well water and never use dechlor, despite that I do use tank water in a bucket to rinse media. It is simply easier and it is waste water anyway. Because, municipal water systems sometimes do a high level system flush with chlorine etc., it is prudent to use dechlor, especially for those who cannot quickly identify a chlorine/chloramine issue and be able to deal with in, jik. When I travel to events to sell fish, I always have a bottle of AmQuel and one of Prime with me.
I picked up this information from a couple of scientific papers. If you are so inclined, you can read this one for yourself:
Monochloramine penetrated fully into nitrifying biofilms within 24 hours when fed at a 4:1 Cl2:N ratio, showing a cessation of aerobic activity via DO penetration following
application of monochloramine. However, monochloramine penetration did not necessarily equate to a loss in viability, and the presence of excess ammonia in the water system prevented microbial inactivation. Biofilm recovery occurred when disinfection stopped........ Monochloramine showed greater penetration compared to chlorine. Monochloramine penetrated into the biofilm surface layer 49 times faster than chlorine within the nitrifying biofilm and 39 times faster in the multi-species biofilm than did chlorine. Phosphate was found to act positively on biofilm development and nitrification
Full PhD Thesis here https://etd.ohiolink.edu/rws_etd/document/get/ucin1258489526/inline