I think there are many layers of reasons for this, starting with the historical.
Poor performance on IQ tests. In the past many autistics were judged low in IQ as a result of their lack of response to the tests (and possibly many still are). Other factors such as fear of the tester/test surroundings, difficulties with verbal communication, being focussed on other things, or even being in sensory overload, were not taken into account. I feel this one is slowly being overcome, as testers are now using different methods to assess us, and many of those once judged ‘low-functioning’ or ‘low IQ’ are also, with the aid of communicative technology, emerging as perfectly normal in intelligence. To the general public however, the perception is still “autistic = low in intelligence”.
Getting absorbed in our own interests or concerns. Most of us find our special interests and/or our daily organising needs far more interesting, or at least absorbing of our energies, than our social surroundings, which we can often ignore. But this can mean that people who don’t understand how our minds work, or what our needs are, can judge us to be ‘vague’, dim-witted, or ‘useless’.
Poor social skills/eye contact. If we don’t seem to notice many of the ‘little things’ others regard as important, don’t do/say the ‘polite’ things, don’t look at others, blurt out things that seem unrelated to what’s being discussed or happening, or clam up when asked questions, this can also lead others to think us unintelligent.
Auditory processing issues. Many of us, when we do listen to others, find their words can often sound garbled or ‘mashed together’, especially in noisy environments, or when several people are talking, or the speaker has their head turned away and/or speaks softly or high-pitched. It can take anywhere from a couple of seconds to several minutes to decipher what was actually said – and hence we are often slow to respond. (I’ve been called a ‘retard’ for just this reason.)
Social processing issues. When we have deciphered what’s been said, we still have to work out how to respond to it. For most of us, responses are hurriedly pulled out of a mental file of ‘Appropriate Things To Say And Do’, often with a sort of silent prayer that it’s the right one! That too can take a little while, and in the meantime, the person is waiting for a response (or more likely hasn’t waited, but has gone on talking!). If there are several people talking, we tend to get even more behind the play.
Emotional regulation issues. Our emotional responses can often be ‘inappropriate’, or even delayed, as it sometimes takes us a long time to figure out what we feel. Or we feel the emotions, but they aren’t evident in our body language. This can lead others to believe we don’t have the intelligence to properly register what’s happening around us.
Sensory processing issues. And then add on sensory overload to all the above, with background noise, smells, people moving around, visual stimulation of various kinds, and we often can’t keep up with what’s going on, or at least not until we’ve had some time alone to process everything.
Result? We can often appear considerably less than intelligent to others. (Some of them are then very surprised when we do display intelligence, especially in writing.) But even if you feel, or have been made to feel, ‘stupid’, it’s important to remember that we’re actually “not daft, but drowning”!
 I don’t want to get into the whole ‘retard’ debate here [eg see Ellen Seidman’s post at www.parents.com/blogs/to-the-max/2013/03/06/autism/5-things-people-dont-get-about-the-word-retard/ ], suffice to say that I agree with those who would like to see the word banned from everyday conversation.