Why does the name of Superman's home planet bear such close resemblance to the most deadly of substances for the Kryptonians?

Because Kryptonite is rock from Krypton.

I'm basing this on my vague memory of the movie but I think the logic is like this:

Superman grew up on Earth.
Not on Krypton.
Thus he would not be used to the Krypton environment.
Thus the radiation from Krypton rock would be harmful to Superman but not to humans (<- bit of a stretch there, I know).
Thus we need to find meteorites that crash-landed in the year that Superman arrived because they must (!) be from the explosion of planet Krypton.
A little bit of handwaving, bake for 30 minutes.
Voila!

Wow- this thing started with Human Man. He's anything BUT a superhero- just the media make him out to be. He is an overly empathetic moi- And as his lawyer, I would like you all not to infringe copywright laws. :P Although I may be this very second, somehow. Anyways, Human Man was not meant- in any way, shape or form- to resemble anything but a suicidal New York business man.

Have a nice day. :D