Will a retirement job keep me from the things I love? Ideally, your retirement job is one of the things you love. So no, this should not be an issue. But any job, no matter how much you like it and no matter how flexible the arrangement, comes with responsibilities you cannot shirk and unforeseen demands on your time especially if it's your own small business. You'll have less time than you might like for leisure, travel, volunteering and spending time with your family. One possible solution is to partner with a former co-worker who is also retired you'll trust each other enough to comfortably hand off responsibilities. Whatever you choose to work at, make sure you get the flexibility you want and then go ahead and take the plunge. After a few extra years of saving and not tapping your nest egg, you can always quit again this time for good and with your finances well set. That's the whole point.
See more questions about whether you should work or not:
Introduction: Should I Plan to Work?
Should I Launch a New Career?
Where Can I Get Business Funding?
Does Work Reduce My Social Security?