I turned on a SQL trace and "replicated" your testing. The queries are different; I must admit I didn't see any reason for that much time differential.
I don't see that magnitude of a difference on my database.
A layman's suggestion:
Is your DB compacted, indexed, optimized, cleaned up and/or whatever other kind of maintenance is normally done to keep it running smoothly?