I think that Fischer was saying that just because you defended your title several times, that alone does not make you the GOAT. Ficher himself was a candidate for the GOAT of chess not only for winning a lot of tournamentss and matches but for playing with a vigorous attacking style that wielded great influence over the development of the game, as opposed to Botvinic's "boring" technique of nursing tiny little positional advantages along for 50 moves. (I personally do not find such technique "boring" -- quite the contrary.)
By the way, the old-time GOATs (pre-1950) always came down to a debate between the Cuban Jose Capablanca and Russian Alexander Alekhine -- and was usually settled on the basis that Capalanca had a gentlemanly, genial and expansive personality in civilian life while Alhekine was a mean and nastly bit of goods. (The true GOAT of chess is Gary Kasparov.

)
Which brings me to...
Rod Laver, over a 24-year career spanning both the "amateur" and open eras, was ranked the #1 player in the world either 5 times as a pro (or 7, depending on what "official" meant back then) and twice as an amateur. He won the Grand Slam twice (1962 and 1969).(Federer 0, Nadal 0, Djokovic 0 -- they are not Steffi Graf after all.

)
Anyway, the one thing I do definitely disagree with is the view that Federer (or Nadal) were better tennis player than Djokovic because they were less taciturn and had a lot of adoring fans. To me, that is irrelevant in GOAT discussions.