The term, not the orifice.
My theory, based upon a lifetime spent in the pursuit of useless historical trivia, is that most cultures would find it inane for a person to call another person a body part. I know we use "prick" and "cunt" and "dickhead" and a host of other anatomical pejoratives in addition to "asshole." But you can't sue anyone for calling you those names because it's not a possibility in the real world. For example, "Son of a Bitch" is metaphorical, but "Fool" is a real possibility. Don't call someone a "Fool" or you might find yourself in court.
So, theorizing(there I go again) that being called an "elbow" or a "toe" wouldn't be offensive, I posit that the true reason we throw "asshole" around so freely is that it's a vestigial language element left over from the Romans.
In Latin there were two words that were spelled a-n-u-s. One had a long u, and the other, a short u. I don't remember which was which, but one of those words meant just what we mean by it today in English.
The other anus, however, means "old woman," and we do use the adjective "anile" in English to mean "Like an old woman." All Male dominated cultures bristle at the idea of the manliness of a male being defamed. In Latin, to call someone an ANUS, was, not to imply that they were anatomically incorrect, but that they were old-womanly.
I think that's why the term "Asshole" is so ubiquitous in Western Culture. If you're not convinced, I'd be happy to expand upon what I've said IN LATIN. (with footnotes)