Yes.
It's probably because the conversation doesn't flow as well as you would like it to. Once you start to feel very different from the group of people, who seem to be easily interacting with the group, things seem to get much worse as time goes on. You might also feel awkward because you spend most of the time trying not to screw up and have everyone think you're crazy.
Just keep at it and try to settle down.
