To those of us still working the 9-to-5, stories of people who retire in their 40s or even younger can read like a hero’s journey of mythic proportions.

But getting to that dream goal isn’t without its all-too-human regrets.

Learning what mistakes early retirees made on their path to retirement can help the rest of us avoid those same pitfalls, so we asked people who’ve done it what regrets they have about their path to financial independence.

Maybe because a goal of financial independence demands a focus on one’s finances, most of these early retirees’ regrets center on things less tangible than money. The financial piece is crucial — here’s more on the money aspect of how to retire early — but dealing with the intangibles may be even harder.

One note: You’ll see references to FIRE. That stands for “financial independence, retire early.” Awkward as the name is, the idea of financial independence is an important one. Many early retirees continue to work, but they choose how, where and when because they’re financially independent.

Here are five hard-earned lessons from early retirees:

1. Remember to have fun

hard-lessons-learned-for-those-who-want-to-retire-early-Jensen
Carl Jensen and his family. (Photo courtesy of Carl Jensen)

It’s not easy to retire early. It’s likely you’ll have to drastically reduce your spending and ramp up your savings rate. But early retirees say money strategies shouldn’t be the only thing on your mind.

“My regret is that I was too focused on getting to the FIRE finish line,” says Carl Jensen, founder of the blog 1500Days.com. In 2017, at age 43, he retired early — with kids.

“My 30s are a blur. I expected early retirement to fix all of my problems, and then it didn’t.”
Carl Jensen

“When I was younger, I was dead-set on accumulating wealth,” Jensen says. He often worked 80 hours a week at his job, even as he and his wife flipped houses for profit, doing much of the remodeling work themselves.

“My 30s are a blur,” he says. “I expected early retirement to fix all of my problems, and then it didn’t.”

It took some time to learn that “happiness comes from within,” he says. “While FIRE has improved my life and I’d never go back to work, it didn’t increase my baseline level of happiness.”

No matter where you are in your journey, he adds, “Find lessons and value in the present.”

hard-lessons-learned-for-those-who-want-to-retire-early-TanjaHester
Tanja Hester. (Photo by Mark Bunge)

Jensen isn’t alone in this regret. Tanja Hester, who retired almost two years ago at age 38, has a similar story.

“It’s just as important to enjoy your life now as it is to plan for a great future.”
Tanja Hester

“When we first started getting serious about saving, I got a little bit obsessive about shopping with coupons,” says Hester, author of “Work Optional: Retire the Non-Penny-Pinching Way.” “I saved a ton of money on groceries, but it was so time-consuming and tedious to look for every possible best deal.”

Eventually, she decided it wasn’t worth it. Instead, she and her husband now strategize on cutting expenses. For example, they focus on big expenses, particularly housing and transportation.

“Don’t focus so much on saving that you don’t let yourself spend on things that you enjoy in the short term,” Hester says. “It’s just as important to enjoy your life now as it is to plan for a great future.”

2. Get a jump on saving and investing

Akaisha and Billy Kaderli, who currently live in Chapala, Mexico, retired when they were both 38 years old — that was 28 years ago.

If they had to do it all over again, they say they’d have gotten started even earlier.

“Neither of us came from families who knew about investing. We stumbled into it at the age of 30,” says Akaisha Kaderli, who co-authors RetireEarlyLifestyle.com with her husband.

Had they known about the power of compounding, she says, “we would have begun investing much sooner, and our nest egg would have been bigger.”

In the same boat the Kaderlis were in? Check out our guide on how to invest in stocks.

3. Don’t let the haters get you down

hard-lessons-learned-for-those-who-want-to-retire-early-Kaderli
Billy and Akaisha Kaderli in Mexico. (Photo courtesy of Billy and Akaisha Kaderli)

When the Kaderlis retired in 1991, the idea of leaving the workforce in your 30s was rare. Many of their friends and family didn’t approve. They regret letting that influence their happiness.

“That, truly, was the hardest part. Going out on this journey alone, without their emotional blessing.”
Akaisha Kaderli

If they were doing it now, Akaisha says, “neither of us would let the opinions and judgments of family and friends influence our happiness about our new life path.”

“That, truly, was the hardest part,” she adds. “Going out on this journey alone, without their emotional blessing. We simply went forward anyway, creating our own path, and today, almost 30 years later, these friends are still working and trying to figure out how to retire.”

Even today there are early-retirement haters. Don’t listen to them. But do get your financial ducks in a row. Retiring early is doable, but not easy — here’s what it takes.

4. Take time to figure out what you’ll do now

Going from all work to no work is a serious lifestyle shift. The prospect might reasonably make you nervous — so nervous that you make many plans to keep busy. Think twice about that.

Pete Adeney, writer of the Mr. Money Mustache blog, says his big retirement regret was diving into a stressful and ambitious new real-estate business immediately after retiring at age 30.

“I thought it would be a dream hobby and not feel like real work, because it involved working on houses, which is what I used to do for fun,” he says. Not so much. On top of disagreements with his business partner, he ran into the housing crash of 2007.

“That left me holding an underwater house and renting it out for several years before I could eventually sell everything off,” Adeney says. “Lost a lot of time and money, and a longtime friend, but gained a lot of wisdom and appreciation for what I do and don’t want in my retirement life.”

hard-lessons-learned-for-those-who-want-to-retire-early-JillianJohnsrud
Jillian Johnsrud. (Photo by Shawna Benson Photography)

Jillian Johnsrud, who became financially independent in 2015 at age 32, has a similar regret.

“It takes time to learn how to structure your day,” says Johnsrud, an author and speaker who blogs at MontanaMoneyAdventures.com. “No one feels great after bingeing on Netflix for 12 hours, especially the fifth day in a row,” she says.

She wishes she’d taken steps sooner to be involved in volunteering and other projects. “Instead, I had to start from scratch.” Now she and her husband find their days full. “It just took a few years into FIRE to figure out what that should look like.”

5. Embrace the conversation

Maybe you’re shy about your FIRE plan. Or maybe you’re laser-focused on the day-to-day financial challenge of retiring early. Consider taking a moment to talk with family and friends.

“I wish we would have spent more time talking as a couple about what we wanted this financial independence life to look like,” Johnsrud says. “We were good about planning the current year, but didn’t give much thought to our financial-independence lifestyle. It was more keeping our head down to save and invest.”

She also wishes she’d talked about financial independence with other people in her life.

“I regret not talking to my friends more about this. I would occasionally mention it almost in a whisper,” Johnsrud says. “It would be amazing if all my closest friends were also financially independent now. They could join us on our epic trips!”

More From NerdWallet

Andrea Coombes is a writer at NerdWallet. Email: acoombes@nerdwallet.com. Twitter: @andreacoombes.

This article originally appeared on NerdWallet.