You're being over-simplistic
You're being over-simplistic and you're making some assumptions that are actually incorrect. Diesel and gasoline can be made from the same barrel of oil. Generally, an oil refinery tower can be seen as a large boiler with strata of heat. The hotter things are, the higher they are in that tower. Diesel is somewhere in the middle while gasoline is below it (cooler) with much of our gasoline coming from a coker system of refining that adds further steps to the process. More gasoline can be extracted per barrel of fuel vs diesel, but less effort is required to make the diesel fuel. In the past, this was generally a wash and the low cost of oil often made diesel cheaper overall.
Another thing to remember is that "worldwide demand" for diesel and other fuels doesn't mean anything other than how it affects oil prices themselves. We do not import diesel or gasoline, we make it here. Nearly all other countries do the same - import the oil, refine it yourself. It makes no economic sense to do otherwise.
The reality is that the majority of the diesel cost difference at the pump is from two factors: higher state and federal taxes on it (versus gasoline) and higher demand with lower availability due to many areas having bans on building new refineries. California is a prime example of both those paradigms working together to force diesel importation from other states. I know a couple of drivers who make a lot of money hauling diesel fuel from Reno, Nevada to Sacramento due to this. Finally, as you point out, the desulferization process adds cost to the diesel mix. U.S. refineries, however, are not necessarily "optimized for legacy gasoline" as you point out. What's happened is that transportation costs for gasoline have required that at least half a barrel be made into it in order to turn a reasonable profit. This requires 20-30 percent of the oil be cracked or reformed (or both) at the expense of other products in the process. Gasoline is expensive and difficult to transport, hence the cost.
Europe chose diesel over gasoline for one simple reason: in the 1970s/80s, it was cheaper, cleaner, and more cost-effective environmentally and economically to go diesel there. Since most European countries import oil or fuels, they had to choose and diesel is more energy dense and easier to refine than is gasoline. The rest of the oil can be used for other things without requiring that gasoline be extracted unless it's as a by-product.