If you're getting a sport license, fly a sport aircraft. But to each his own. On your thoughts of "not getting high"... Clouds here often get quite low, sometimes ground level. You'll learn a lot more about this when you start flying, but especially in something as un-capable as an Ultralight, you will be confined to nice light-wind high-pressure-stable-airmass clear weather days regardless. Don't even *think* about getting anywhere *close* to clouds in one of those things.
Um, I'd be very surprised if they got you anywhere close to the mountains as well. First off, they probably lack the climb performance to get that high up (it's about absolute altitude above sea level, not relative sea level above the ground), and on top of that mountain flying is an artform in itself. Even light winds can create downdrafts that typical small aircraft have a hard time with, so attempting that in a winged scooter just seems suicidal to me.
If you want to go play in the mountains, let me know. We'll go rent a 182 on a nice day and I'll fly you around. Please don't be stupid with an ultralight though.
It does look like fun, and is probably a hell of a lot cheaper than the planes the rest of us fly. Then again I can fly over mountains at faster-than-liter-bike speeds, so to each his own.
If you need references to good instructors let me know...