I wanted a quick selection of 50 tracks, culled randomly from my 5000+ collection, but I didn't want the list to ever get stale. That's why I created the randomizer smartplaylist (I added a ".:." prefix, so this list is always #1 on my iPod).
Create a Smart Playlist:
Match all conditions
Last played - is not in the last - 15 days
Play count - is less than - 5
Limit to 50 songs - selected by - least recently added
Match only check songs