You can only compare them that way
when they do the same layout. You can't compare the speed of one guy who jumps 3 quads and 2 triple axels with numerous difficult transitions to a guy who only jumps 1 quad and 1 triple axel with less difficult entries into jumps, and let's not mention the difficulty of the spins. Obviously the guy who does lesser layout has more energy to skate faster. That's it, and that's not my words. I took it from my friend's coach who was there at Skate Canada.
And he said in the SP, their speed weren't much different at all.
If anything, I hope to see Patrick skates to the same layout as Yuzuru so we can have better understanding of how fast he can be with such a demanding program.