Whether you use call logs to reconcile long distance charges, to integrate call recordings, for internal business unit billing or some other purpose, you may have seen situations where you are missing some, but not all, CDR (Call Detail Records) entries.
A Common Problem
Unfortunately, missing CDRs are a common issue that can affect CUCM (Cisco Unified Communications Manager) clusters. The root cause of the problem is how CDR call logs are created by CUCM. Each server, subscriber or publisher that handles a call will locally record CDR data and then send the logs to the publisher, which consolidates them before publishing them to the billing server.
Sometimes, most often when there is a disruption in network communication, the subscribers may fail to send call information to the publisher. This issue usually resolves itself; however, we have seen cases where it does not.
The RTMT Alert
A single RTMT (Real-Time Monitoring Tool) alert is created when this issue occurs, you will see a
CDRAgentSendFileFailed message if you have appropriately configured RTMT to raise the alert. This message is only sent one time, even if the agent fails to send updates to the publisher for days or even months. There is, visible in debug logs, a
CDRAgentSendFileFailureContinues message, but we have not been able to action it outside of the logs.
You can also look at the length of your queued messages. Executing the command
file list activelog /cm/cdr/ will yield output like:
admin:file list activelog /cm/cdr/_cdrIndex.idx
dir count = 0, file count = 3
The output of the command will show you, as a last line, the count of files waiting to be sent. It’s normal to sometimes see some files waiting to be sent but a sustained value above 3, or a large value like this one from a severely impacted system, are a clear sign of an issue:
dir count = 0, file count = 62438
If you are facing challenges staying on top of alerts and warnings from your Cisco UC servers, you’re far from alone. Our managed services team is ready to help you—and our development team is constantly working to improve the monitoring systems that support them and their clients.