Are you using amavis? Try restarting amavis to clear any hung connections. If that doesn't work you may need to increase connection limits on amavis / clamd / spamd