Many years ago, when mini PCs were still a new think (think NUC had not come out yet? I didn't hear of it at that time) I bought one of those Asus mini box things, the first gen of those if I'm not wrong. It had a Celeron processor and I though it would be a good choice as a simple media PC for my parents. But the thing was so crazily slow. Loading Windows (Win 7 I think) took several minutes, and even after you logged in, it would be laggy. Even mainstream laptops of the day performed better. I hated the thing, it was more frustrating to use it than the benefit it offered. Barely used it, few years later wrote it off as a bad purchase.
Now, I know that computing power and CPUs have improved much since then, and what's low end today can possibly still perform decently for mainstream computing tasks like browsing, media, spreadsheets etc. But the universal truth in computing remains that you get what you pay for, and higher performance requires higher energy. It's just nature at work, no going around it. Whether one thinks the price premium is worth it, of course is subjective and it depends on what you're going to use it for, but the principle remains valid.
End of the day, your PC should serve you, not frustrate you. If it is within my means, I'd rather pay a bit more for a more comfortable experience, than scrimp and end up unhappy with the thing. Not sure what TS' intended use is for Pentium/Celeron CPUs, or it's just a commentary, but yeah, as I mentioned in my first post, the likely answer why shops don't carry these anymore is simply no demand, combined probably with low profit margins.