Letter/Not a Letter They Might Be Giants

What's that?
That is a letter
What is that?
That's not a letter
That's a letter
That's not a letter
That is a letter
Is that a letter?
I don't know what that is
That's so a letter
What is that?
Whatever it is, that's not a letter
That's a letter
Is that a letter, or what?
That's a letter. Only it's upside down