The groupset would drive it for me. If I was buying a new bike, and I knew I wanted to be a rider, I wouldn't mess about with anything less than Shimano 105. At Specialized, the lowest end bike with the 105 groupset on it is $2100. That's the Allez Comp, which has an aluminum frame and wheels.
The next step up the ladder would be their "endurance" frame, which is carbon. It's called the Roubaix, and equipped with 105 it's $2800.
Either of those would be a good first "serious" bike.
If the question is more about diminishing returns, I'd offer my own bike, which is a Giant TCR Advanced. It's a couple years old. I have about $5500 in it, all in, but that includes the middle-grade SRAM electronic shifting group, carbon wheels, and a power meter. The meter is skippable if you're not doing serious training, but I did and do use power data for training. Subtract $800 if you don't want that.
I honestly think spending more is just showing off. If that's your jam, knock yourself out, but it's probably not making a big difference UNLESS you need a custom frame to be comfortable.
In the Uk it’s about £3-4k I think. Beyond that the differences in function are very small and would make no differences at all in an amateur road race. For time trials it’s a bit different and there clothing does make a measurable difference, although still v small relative to training harder!
Working a paid job does nothing to help those less fortunate than you. Maybe by retiring early you can free up time to help others in more creative ways?
reply