Difference, I believe there should be. The real question is "how much" are you willing to pay for that difference.
Personally, I am not willing to pay so much more for that difference. But to some, they feel that the amount they pay is ok.
That difference also includes the additional storage.
I bought a Note 3 with 3GB ram which at that time was the highest of any phone. The norm then was 1GB and 2GB was the high end. My Note 3 can still run any app easily without any lag.
Apple is the sneakiest of all. They have the highest specs BUT below average ram. The Apple faithful defended that Apple is efficient so more ram is wasteful. A few years later, their old iPhones are still powerful but the bottleneck is insufficient ram. Go read about it in their forums. LOL.
You really got to "admire" some of those die-hard Apple apologists who still defend Apple low ram strategy....they know it is Apple's strategy to force upgrading in the future and they agree to it because they too want Apple fans to buy the newest and "bestest" later and throw away old iPhones to upkeep the high class image of Apple.
Now that some of the faithfuls also start complaining, Apple lan lan had to increase the ram by the minimum.
Notice that the high end phones are now starting to have 6GB ram. Next year it will probably increase to 8GB.
In IT, there's never enough ram. Software has to trim features to target the most users for the largest market share. As the ram for most users start to creep up, the software also gladly follow.
Notice how Facebook is always updating almost weekly and the size is now more than 50MB? Notice the phone browser is always not as powerful as the desktop version? That's being limited by ram.
Unlike PC, ram on the phone is not user upgradeable. Buy the max you can get now and be ahead of the majority; for them to catch up to you later while you're comfortable even in the future. I hate to waste time grumbling about low ram.
It cost a bit more now but worth it.