Lol.. everything in the audio chain is important, even the power supply. But it all starts with the source. If source is bad.. U cannot make it better, but U can make it sound different.
Start with a good source, preferably lossless music files. Then a good DAP..smartphones are ok (but use lossless music files), but they lose out to good dedicated DAPs.
Then look into the rest of the chain, especially the last part.. your ears. lol.. good ear hygiene, cleaning up the passageways is good practice, especially for IEMs... lol
The type of earphones being used is important to determine if one needs an amp. If U use a low resistance cans, e.g., IEMS, personally, I think that one does not need an amp.. though no one will stop U from having one (overkill), but if U have one of those difficult to drive cans, then, an amp is necessary, if not vital.
Next is cables.. but that is another can of worms..
Personally, I would first decide on the listening situations for my headphone. e.g. portability or sedentary.. if I were to listen to music on the move, I would then choose a good DAP, no amp, and iems (preferably). For sedentary use.. DAP, DAC, AMP, headphones.. lol...