As a 180+ IQ kid who grew up in an institution for "crippled and retarded children", as the charming phrase went back then, I can tell you your theory is mostly true. Then again, it depends on your KIND of intelligence. Also, you should read Orson Scott Card's 'Ender's Game', if you haven't already. :)