I got really lucky when Beijing got the 2008 Summer Games and had them come to me so no traveling logistics was involved. Tickets were cheap and I managed to procure a lot of them, so I ended up going to 32 sessions in 17 different sports, lots of them finals like the Womens IAA gymnastics, Phelps in the pool. It was exhausting but the sports events and athletes were brilliant, even the ones I was less familiar with. What was not great was the very oppressive and strange atmosphere in the city caused by overbearing government security.
I'd love to see a Winter Games at least once in life, but never previously had the chance--either timing, money, or both an issue. Sochi was a non-starter for all kinds of personal/life reasons. Pyeongchang 2018 might be a possibility but I think ticket costs will just keep escalating into the stratosphere and justifying the cost is getting harder for me, at least. I'll probably be too decrepit to travel by 2022. Might have to content myself with a World Championships some year.