Along with watching Olympics, many people go to see the venues and/or trade pins. Trading pins at the Olympics started at the 1896 Athens Olympics between participants, but really didn't become popular until the early 1900s. Pin trading has also been more and more exciting as more and more pins are released. Plus, it's a great way to meet people and have fun at the same time. There's a pin trading center sponsored by Coca Cola in the Olympic Village, too (but the general public can't go in there).
There are many types of pins and they also vary from Olympics to Olympics. The types of pins are quite varied, from pins made by sponsors, to venue pins, to pins with the Olympic Mascots on them. When I went to the men's final beach volleyball match on Friday, I saw these guys trading pins during the break between the bronze medal game and the gold medal game. One guy was from NBC and he was telling the other guy about their special pins, like which ones were sponsor pins, client pins, etc. That's part of the bargaining for pins. One guy even had pins from the Moscow Summer Olympics (1980).
Just like in trading Pokémon cards, the rarer the pin is, the more it is worth! If the pins have moving parts (like a door or a spinning part) then it is worth more, too. I also found out that most people wanted pins with either the Olympic Rings or the logo for that Olympic Game on them. I just had ones with an American flag and a Chinese flag on it. Some collectors will sell you pins, too. I bought my Road Runner one for 20 RMB (US$3.00).
I have just traded pins a couple of times—at the beach volleyball game, and once at a party when I met Gabe Gardner and he gave me a USA volleyball pin when I gave him a China souvenir as thank you for my interview. My mom traded and got me pins from a nice manager at the Coke pavilion, with a silver medalist fencer and the USA diving team organizer. You can see my pin collection on my Olympic shirt.
My parents bought me one for every event that we got to see (those cost about 25 RMB (US$3.75). My brother has a collection, too, but he didn't like collecting pins as much as I did.
So if you ever go to an Olympics, don't forget to bring some pins to trade with. Maybe we'll see each other trading pins at the London 2012 Olympics!