Exhaust, no, you're not hurting it, but you'd get more performance out of a 3" system. Plus, if you're ever cruising with someone, they'd enjoy you having a cat so their eyes aren't burning when they're behind you...
Sound from the motor, can't say. Sounds like the typical supra hickup, but...