The story I was always told was that the original term was “capital haven”. During the second world war, wealthy Europeans (especially Jewish people, but not just them) who feared losing their wealth due to invasion or otherwise in the conflict, stashed capital away in Switzerland and other neutral countries to protect it. In time the term using a “capital haven” evolved, although for obvious reasons it wasn’t really a mainstream term.
After the war, most of Europe employed sky high taxes to pay for the reconstruction. Most tax systems worked on a territorial basis in those days - income arising within the country was taxed, but income arising abroad was exempt. So the same broad concept applied, people moved their liquid wealth abroad to limit its exposure to tax. Hence the term evolved from “capital haven” to “tax haven”.
Offshore banking refers to the placement of funds by a company or individual in a bank that is located outside of their national residence. Although the term implies that these banks are located on islands, many offshore banks are located in the Cayman Islands, Bermuda, Luxembourg, Channel Islands, Macau, and Panama. In general, you should visit https://www.offshorecorptalk.com/ to learn more about offshore jurisdictions.