new interview on SiriusXM’s The Howard Stern Show this week, Affleck spoke about what led to his initial split with Lopez.

The media attention around their romance played a huge role: “I would say it was about 50 percent [of what destroyed our relationship],” he said, via Entertainment Tonight. “The idea that people hate you and they hate you together and that being together is poison and ugly and toxic and the thing none of us want to be part of. And who the f*ck would want to have them to dinner? And what the f*ck are they doing together?’” After Lopez ended her engagement to him, “I got f*cking hurt and angry and felt like a fool,” he said.

Lopez and Affleck first began dating in 2002. During their first run, Affleck appeared in her “Jenny from the Block” music video (something he spoke about regretting in 2008: “If I have a regret, it was doing the [‘Jenny from the Block’] music video,” he said. “It not only makes me look like a petulant fool, but it surely qualifies as ungentlemanly”) and proposed to her that same year. The two almost wed in September 2003. They postponed their wedding a day before the ceremony.

At the time, they released a statement explaining their decision, blaming it largely on media attention:

Due to the excessive media attention surrounding our wedding, we have decided to postpone the date. When we found ourselves seriously contemplating hiring three separate “decoy brides” at three different locations, we realized that something was awry. We began to feel that the spirit of what should have been the happiest day of our lives could be compromised. We felt what should have been a joyful and sacred day could be spoiled for us, our families, and our friends.

In January 2004, Lopez’s rep confirmed she ended her relationship with Affleck: “I am confirming the reports that Jennifer Lopez has ended her engagement to Ben Affleck. At this difficult time, we ask that you respect her privacy.”

But 17 years later, things are very different. A source toldEntertainment Tonightin July of Lopez and Affleck’s back-on romance, “They couldn’t be more in agreement. They feel at this point in their lives they were undeniably made for each other. Things between Ben and Jen have been easy and that’s what makes their relationship work so well. They work seamlessly together and it’s like their relationship hardly takes any effort.”

“Nothing has been supremely difficult, and they don’t tend to bump heads,” the source added. “They are always working as a team and they love that about each other and their relationship.”