IPhone :: Speakerphone Non-functioning But Can Record Voice Memo?

Just got new iPhones a couple of months ago. Up until now everything was great. Now when I put a call on speakerphone, the other party can't hear me at all. If I blow into the mic, they can hear that, but they can't hear me talk. I've cleaned the mesh with a soft, dry toothbrush as suggested, made sure that the volume didn't read "headset", turned the volume up, turned the phone off-on, made sure nothing else was blocking the mesh over the mic, etc.

The really weird thing is that I was able to record a "Voice Memo" which uses the same mic and the recording sounded just fine.

So, is it hardware or is it software? Doesn't seem to be hardware since I can record a "Voice Memo" with no problem and I can also communicate with Siri just fine. It seems that the problem only happens when using the phone speakerphone feature.

Info:
iOS 5.1, App Store Features