As everyone is saying, definately bring the bike! There are definately riders in southwest CO, you just gotta flag em down. There does not seem to be many in clubs. All the afore mentioned roads are excellent (especially red mountain pass). The one not mentioned though was the Telluride, Gateway, Grand Junction loop. As far as I am concerned that is the best road in Colorado. 55 miles of everything from sharper corners to long gentle ones you can take at crazy speeds and the road was just paved a couple years back so its still in perfect condition! There is almost minimal traffic (even in the summer)
I have a couple friends that live in Durango, Ill see if I can get them to toss out a few names of some riders for ya to link up with. And when ya get a chance time yourself from Silverton to Ouray. My best so far is 21 minutes, taking into account all the summer traffic.