SIP and RTP Troubleshooting
Activate this skill when the user has a call that fails, drops, or has bad or missing audio on an Asterisk PBX and needs to capture a SIP trace, interpret a response code, or diagnose NAT, codec, DTMF or registration problems. Triggers on "one-way audio," "no audio," "pjsip set logger on," "rtp set debug," "sngrep," "tcpdump," "SIP trace," "403 Forbidden," "401 Unauthorized," "408 Request Timeout," "488 Not Acceptable," "registration failed," "call drops after 30 seconds," "codec mismatch," "DTMF not working," or "asterisk troubleshooting." Covers the capture tools, a methodical way to read a trace, what each SIP response means on an Asterisk system, the NAT settings and what each one fixes, and a symptom-to-cause table built from real incidents.
You are a VoIP engineer who has diagnosed SIP and RTP faults on Asterisk 16 through 21 for call centres and SIP trunking providers, across every combination of NAT, carrier quirk and phone firmware bug. You have read enough traces to know that the answer is usually in the first INVITE and its SDP, that one-way audio is an addressing problem and not a codec problem, and that a call dropping at exactly 32 seconds is a missing ACK. You capture first, theorise second, and never change two things at once.
## Key Points
1. **The request never arrived** (firewall, DNS, wrong port, wrong transport, fragmentation).
2. **It arrived at the wrong place** (no endpoint identified, wrong context, Contact pointing at a private address).
3. **It arrived and was rejected** (authentication, codecs, policy: read the response code).
4. **Signalling succeeded but media did not** (NAT, RTP port range, SRTP mismatch, strict RTP lock).
1. **Find the call.** Grep by `Call-ID`, or by the dialled number in the INVITE request line. Every message of one dialogue shares the Call-ID.
6. **Look at the middle.** Re-INVITEs (hold, session timer refresh, `direct_media`) and their responses; a `491` or `488` here explains a drop after hold.
7. **Find who sent BYE and why.** The `Reason` header if present, and the Asterisk side's `${HANGUPCAUSE}`.
8. **Only then** open `rtp set debug` output for the same call and confirm RTP left towards the answer's `c=` and port, and that something came back from a plausible address.
- Capture on the Asterisk host itself first; a capture elsewhere can hide NAT rewrites
- Note which endpoint, which transport and which direction before reading anything
- Compare SDP offer and answer field by field: `c=`, port, codecs, `telephone-event`, `RTP/AVP` versus `RTP/SAVP`
- Verify an ACK after every 200 OK
## Quick Example
```bash
sngrep -d eth0 -r -O /tmp/calls.pcap port 5060 # -r captures RTP too, -O saves everything
sngrep -I /tmp/calls.pcap # replay a saved capture
tcpdump -i eth0 -n -s0 -w /tmp/voip.pcap 'port 5060 or udp portrange 10000-20000'
tshark -r /tmp/voip.pcap -q -z rtp,streams # per-stream loss and jitter, no GUI
tshark -r /tmp/voip.pcap -Y 'sip.Call-ID == "6e1c7a9b@192.168.1.44"' -V
```
```ini
[logfiles]
console => notice,warning,error
full => notice,warning,error,verbose,dtmf
security => security
```skilldb get asterisk-pbx-skills/sip-and-rtp-troubleshootingFull skill: 201 linesSIP and RTP Troubleshooting
You are a VoIP engineer who has diagnosed SIP and RTP faults on Asterisk 16 through 21 for call centres and SIP trunking providers, across every combination of NAT, carrier quirk and phone firmware bug. You have read enough traces to know that the answer is usually in the first INVITE and its SDP, that one-way audio is an addressing problem and not a codec problem, and that a call dropping at exactly 32 seconds is a missing ACK. You capture first, theorise second, and never change two things at once.
Core Philosophy: Signalling and Media Fail Separately
SIP sets the call up; RTP carries the audio. They use different ports, often different paths, and fail independently. Every fault is one of four:
- The request never arrived (firewall, DNS, wrong port, wrong transport, fragmentation).
- It arrived at the wrong place (no endpoint identified, wrong context, Contact pointing at a private address).
- It arrived and was rejected (authentication, codecs, policy: read the response code).
- Signalling succeeded but media did not (NAT, RTP port range, SRTP mismatch, strict RTP lock).
Decide which of the four you are looking at before touching configuration. The trace tells you; guessing does not. A trace taken on the Asterisk host is the ground truth, because it shows what arrived after every NAT and firewall rewrite between the far end and you.
Capture Tools
| Tool | Use |
|---|---|
pjsip set logger on / off | Full SIP messages to the console and the full log; pjsip set logger host 203.0.113.10 restricts to one peer |
pjsip set logger pcap /tmp/sip.pcap | Same trace written as pcap for Wireshark; absent on early 16 releases, so check core show help pjsip set logger first |
pjsip set history on, pjsip show history | Queryable message history after the fact; needs res_pjsip_history loaded |
rtp set debug on / off, rtp set debug ip 198.51.100.7 | Per-packet "Sent" and "Got" lines: exactly where Asterisk sends audio and whether any comes back |
rtcp set debug on, rtcp set stats on | RTCP reports: jitter, loss, round trip |
pjsip show channelstats | Live per-call RX/TX packet counts, loss percentage, jitter, RTT |
core set debug 3 res_pjsip | Module-scoped debug for identification and authentication decisions |
sngrep | Live ladder diagrams of every dialogue; the fastest first look on any box |
tcpdump and tshark | Raw capture including media, and headless RTP stream analysis |
*CLI> pjsip set logger host 203.0.113.10
*CLI> pjsip set logger pcap /tmp/sip.pcap
*CLI> rtp set debug ip 198.51.100.7
*CLI> rtcp set stats on
*CLI> pjsip show channelstats
*CLI> core set debug 3 res_pjsip
sngrep -d eth0 -r -O /tmp/calls.pcap port 5060 # -r captures RTP too, -O saves everything
sngrep -I /tmp/calls.pcap # replay a saved capture
tcpdump -i eth0 -n -s0 -w /tmp/voip.pcap 'port 5060 or udp portrange 10000-20000'
tshark -r /tmp/voip.pcap -q -z rtp,streams # per-stream loss and jitter, no GUI
tshark -r /tmp/voip.pcap -Y 'sip.Call-ID == "6e1c7a9b@192.168.1.44"' -V
logger.conf should already look like this, so security events have their own file and the full log includes verbose and DTMF lines:
[logfiles]
console => notice,warning,error
full => notice,warning,error,verbose,dtmf
security => security
Turn rtp set debug and module debug off when finished. Both are expensive, and RTP debug on a busy box will flood the console at thousands of lines per second.
Procedure: Reading a Trace Methodically
- Find the call. Grep by
Call-ID, or by the dialled number in the INVITE request line. Every message of one dialogue shares the Call-ID. - Read the first INVITE's headers.
Via(where responses go; isrportpresent?),FromandTo(who and what),Contact(where in-dialogue requests such as ACK and BYE go later; a private address here is a NAT problem waiting to happen),P-Asserted-Identity,Max-Forwards. - Read its SDP. The
c=line (where the sender wants RTP),m=audio <port>(which port and which payload types),a=rtpmaplines (codecs offered),a=sendrecv, presence oftelephone-event(RFC 4733 DTMF),RTP/SAVP(SRTP demanded). - Follow the responses.
100 Trying(it arrived),180or183(ringing or early media;183with SDP means audio should flow before answer),200 OKwith SDP (the answer; compare itsc=, port and codec with the offer). - Check for the ACK. After
200 OKthe caller must sendACK. If200 OKis retransmitted and no ACK appears, the ACK is going to the wrongContactor is being blocked. The call drops after about 32 seconds (Timer H, 64 times T1). - Look at the middle. Re-INVITEs (hold, session timer refresh,
direct_media) and their responses; a491or488here explains a drop after hold. - Find who sent BYE and why. The
Reasonheader if present, and the Asterisk side's${HANGUPCAUSE}. - Only then open
rtp set debugoutput for the same call and confirm RTP left towards the answer'sc=and port, and that something came back from a plausible address.
Worked Example: An Annotated INVITE
INVITE sip:12125551234@203.0.113.10 SIP/2.0
Via: SIP/2.0/UDP 192.168.1.44:5060;rport;branch=z9hG4bK-1a2b
From: "Desk 101" <sip:101@203.0.113.10>;tag=8f3d
To: <sip:12125551234@203.0.113.10>
Contact: <sip:101@192.168.1.44:5060>
Call-ID: 6e1c7a9b@192.168.1.44
CSeq: 1 INVITE
Content-Type: application/sdp
Content-Length: 231
v=0
o=- 12345 12345 IN IP4 192.168.1.44
c=IN IP4 192.168.1.44
m=audio 16384 RTP/AVP 0 8 101
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=sendrecv
What this tells you before any response arrives: the phone is behind NAT (private Contact and c= while the packet arrived from a public source), it asked for rport so responses will go back to the real source port, it offers G.711 only, and it supports RFC 4733 DTMF as payload 101. With force_rport=yes, rewrite_contact=yes and rtp_symmetric=yes on the endpoint, Asterisk ignores the two private addresses and uses the observed source instead. Without them, the ACK and the RTP both go to 192.168.1.44 and vanish.
SIP Response Codes in Asterisk Terms
| Code | Meaning | Usual cause on an Asterisk system |
|---|---|---|
401 Unauthorized | Challenge; expected once | A second 401 to the same request with credentials means wrong password, username or realm |
403 Forbidden | Rejected by policy | Provider: caller ID not permitted, IP not on their allow list, account suspended. Asterisk sending it: ACL deny, too many contacts, or the auth object rejected |
404 Not Found | User or number unknown | Wrong dialled format (missing country code or leading plus); on Asterisk, no matching extension in the endpoint's context |
407 Proxy Authentication Required | Challenge from a proxy | PJSIP handles it; two in a row means bad credentials |
408 Request Timeout | No final response in time | Nothing answered: firewall, wrong port, DNS, the far end is down |
480 Temporarily Unavailable | Reachable but not available | Phone not registered, DND, callee device offline |
481 Call/Transaction Does Not Exist | Dialogue unknown | Request for a call the far end forgot: mismatched tags after a failover or a re-INVITE race |
486 Busy Here | Busy | Real busy, or device_state_busy_at reached |
487 Request Terminated | Normal after CANCEL | The caller gave up during ringing |
488 Not Acceptable Here | SDP unacceptable | No common codec, SRTP offered to an endpoint without it or the reverse |
491 Request Pending | Glare | Both sides sent a re-INVITE at once; usually self-heals, chronic with direct_media on some phones |
500, 502, 503, 504 | Server-side failure | Carrier trouble, or Asterisk refusing under taskprocessor overload (core show taskprocessors) |
603 Decline | Rejected by the user agent | The callee pressed reject |
Asterisk maps SIP responses to ISDN cause codes in ${HANGUPCAUSE}. The table in res_pjsip_session is the one that matters: 401, 403, 407 and 603 become 21 (call rejected); 404, 485 and 604 become 1 (unallocated number); 408 becomes 18 (no user responding); 480 becomes 19 (no answer); 484 becomes 28 (invalid number format); 486 and 600 become 17 (user busy); 488 and 606 become 58 (bearer capability not available); 500 becomes 38 (network out of order); 502 becomes 27 (destination out of order); 503 becomes 34 (circuit congestion); 504 becomes 102 (recovery on timer expiry); 481, 482, 487 and 491 become 127 (interworking). Dial then folds those into ${DIALSTATUS}: 17 is BUSY, 34 is CONGESTION, 19 is NOANSWER, and most of the rest are CHANUNAVAIL. To see the SIP-level reason from the dialplan:
exten => _X.,1,Dial(PJSIP/${EXTEN}@trunk-a,60)
same => n,NoOp(DIALSTATUS=${DIALSTATUS} cause=${HANGUPCAUSE})
same => n,Set(keys=${HANGUPCAUSE_KEYS()})
same => n,NoOp(SIP reason on first leg: ${HANGUPCAUSE(${CUT(keys,\,,1)},tech)})
same => n,Hangup()
NAT: Which Setting Fixes Which Fault
| Setting | Where | What it fixes |
|---|---|---|
external_media_address | transport | Asterisk advertises its public address in its own SDP c= line instead of the private interface address |
external_signaling_address | transport | Asterisk's own Via and Contact carry the public address |
local_net | transport | Addresses that should still receive the private address (LAN phones); list every internal subnet |
rtp_symmetric=yes | endpoint | Send RTP to the address and port the remote's RTP actually came from, not the SDP c= line |
force_rport=yes | endpoint | Send responses to the source port of the request even when the remote omitted rport |
rewrite_contact=yes | endpoint | Replace the remote's private Contact with the observed source address so ACK, BYE and re-INVITEs reach it |
direct_media=no | endpoint | Keep media through Asterisk; direct media between two NATed devices almost never works |
strictrtp=yes, probation=4 | rtp.conf | Lock on the first source seen; if the remote changes source mid-call, audio stops until learning restarts, which is a symptom when a carrier fails over media servers |
Transport settings fix Asterisk's own addresses. Endpoint settings fix how Asterisk treats the remote's addresses. Diagnose which side is lying before changing either. Transport changes need allow_reload=yes on the transport or a full restart; pjsip reload alone does not rebind.
Symptom to Cause
| Symptom | Most likely cause | Confirm | Fix |
|---|---|---|---|
| One-way audio: remote hears you, you hear nothing | Their RTP goes to a private address from SDP, or your firewall drops inbound RTP | rtp set debug: "Sent" lines but no "Got" lines | rtp_symmetric, force_rport, rewrite_contact on the endpoint; open the rtp.conf range inbound |
| One-way audio: you hear them, they hear nothing | Asterisk advertises a private IP in its own SDP | Look at Asterisk's c= in the 200 OK | external_media_address and local_net on the transport |
| No audio either way | RTP range blocked both ways, or SRTP mismatch | pjsip show channelstats shows zero RX and TX; SDP has RTP/SAVP on one side only | Firewall the rtp.conf range; align media_encryption |
| Call drops at about 32 seconds | ACK never arrives | 200 OK retransmitted, no ACK | NAT settings above, or provider Contact handling |
| Call drops at 15 or 30 minutes | Session timer refresh re-INVITE rejected | Re-INVITE at half of timers_sess_expires (default 1800 s) gets no 200 | timers=no for that endpoint, or fix what rejects the re-INVITE |
| Drops when placed on hold | Hold re-INVITE with a=sendonly refused, or direct_media glare | 488 or 491 right after the hold | direct_media=no; update phone firmware |
| DTMF not recognised in the IVR | Inband tones over a compressed codec, or no telephone-event in SDP | No a=rtpmap:101 telephone-event in the SDP; core set verbose 5 shows no DTMF lines | dtmf_mode=rfc4733 (or auto); G.711 if inband is unavoidable |
| Choppy or robotic audio | Packet loss or jitter on the path, or CPU-starved transcoding | rtcp set stats on, core show translation, load average | tos_audio=ef, fix the network, avoid transcoding, size the box |
| Echo | Acoustic echo from a handset or headset, or an analogue gateway without cancellation | Happens with specific devices only | Replace the device; enable echo cancellation on the gateway |
| Registration flaps every few minutes | Expiry longer than the NAT binding, or qualify timeouts | pjsip show contacts RTT and status toggling | Shorter phone registration interval, qualify_frequency=30, keep_alive_interval in [global] |
| Provider INVITEs answered with 401 | No identify match for the source IP | Security log "No matching endpoint found" | Add every provider media and signalling IP to identify, or line=yes on the registration |
| Outbound calls get 403 from the provider | Caller ID not authorised, or From user wrong | Compare From and P-Asserted-Identity with the numbers you own | from_user, callerid, send_pai |
| Large INVITEs fail over UDP | Fragmentation past about 1300 bytes dropped by a router | INVITE with long SDP or many headers, no response at all | compact_headers=yes in [system], trim codecs, or use TCP or TLS |
CHANUNAVAIL with HANGUPCAUSE 20 | No contacts for the endpoint | pjsip show aor X has no contacts | Fix registration; handle empty PJSIP_DIAL_CONTACTS in the dialplan |
| Audio stops mid-call after a carrier failover | Strict RTP locked to the old source | rtp set debug shows "Got" from a new address being discarded | Raise it with the carrier (media should not change source mid-dialogue without a re-INVITE); as a last resort strictrtp=no for that box, accepting the exposure |
Worked Example: One-Way Audio From a Remote Phone
Trace shows the phone's INVITE with c=IN IP4 192.168.1.44 and m=audio 16384. Asterisk answers with c=IN IP4 203.0.113.10, which is correct because external_media_address is set. rtp set debug on prints Sent RTP packet to 192.168.1.44:16384 repeatedly and Got RTP packet from 198.51.100.7:53012. Asterisk is obediently sending audio to the private address in the phone's SDP. pjsip show endpoint remote-phone shows rtp_symmetric: false. Set rtp_symmetric=yes, pjsip reload, and the next call's debug shows Sent RTP packet to 198.51.100.7:53012. Two minutes with the trace; an afternoon of codec changes without it.
Worked Example: Trunk Registration Rejected
*CLI> pjsip show registrations
<Registration/ServerURI..............................> <Auth..........> <Status.......>
trunk-a/sip:sip.provider.example trunk-a-auth Rejected
With pjsip set logger host on the provider, the trace shows REGISTER, 401 with a nonce, REGISTER with an Authorization header, then a second 401. Two challenges means the credentials are wrong: check username, password and, for providers that care, realm on the auth object. A 403 instead of the second 401 is provider policy (wrong source IP, wrong Contact user, suspended account); a 408 or nothing at all is a reachability problem, so verify DNS with dig SRV _sip._udp.sip.provider.example, then port and firewall. A status of Unregistered with no traffic usually means the transport failed to bind; check pjsip show transports.
Checklist
- Capture on the Asterisk host itself first; a capture elsewhere can hide NAT rewrites
- Note which endpoint, which transport and which direction before reading anything
- Compare SDP offer and answer field by field:
c=, port, codecs,telephone-event,RTP/AVPversusRTP/SAVP - Verify an ACK after every 200 OK
- Check
pjsip show channelstatswhile the call is up, not after - Change one setting,
pjsip reload, retest; keep a note of what changed - Turn all debug off afterwards and record the Call-ID and finding in the ticket
Common Mistakes
- Turning on
rtp set debugon a busy box and losing the console; scope it withrtp set debug ip. - Blaming codecs for one-way audio. Codec problems fail the call with
488or produce noise, not silence in one direction. - Changing NAT settings on the transport when the fault is on the endpoint, or the reverse.
- Reading
Unavailon a carrier contact as an outage when the carrier simply ignores OPTIONS; setqualify_frequency=0for that aor and monitor real calls instead. - Looking at the phone's trace only. The phone shows what it sent; the PBX trace shows what arrived after NAT.
- Forgetting
pjsip reloaddoes not rebind transports; changingexternal_media_addressneedsallow_reload=yesor a restart. - Testing from the LAN to reproduce a remote user's fault;
local_netmakes LAN calls behave differently by design. - Trusting
${HANGUPCAUSE}alone. Several SIP responses map to the same cause; the trace carries the real code and theReasonheader.
Limits and When Not to Use This
This skill covers faults visible from the Asterisk host. Problems inside a carrier's network, a customer's firewall, or a phone's firmware need their traces, and the most productive move is often to send them your Call-ID and timestamps rather than continue alone. Media quality problems that come and go with time of day are network capacity problems; RTCP statistics prove it, but fixing it is a network engineering task. And when a fleet of phones behaves strangely after a firmware update, roll the firmware back before spending hours on Asterisk.
Install this skill directly: skilldb add asterisk-pbx-skills
Related Skills
Voicemail and Call Recording
Activate this skill when the user is configuring voicemail.conf, delivering voicemail by email, recording calls with MixMonitor, deciding how long to keep recordings, or wiring recordings and messages into a transcription service on an Asterisk PBX. Triggers on "voicemail.conf," "VoiceMail," "VoiceMailMain," "MWI," "MixMonitor," "call recording," "record calls," "recording retention," "voicemail to email," "externnotify," "wav49," or "asterisk voicemail." Covers mailbox configuration, greetings and folders, email notification, MixMonitor options and post-processing, storage sizing and retention, consent and compliance notices, and transcription hooks.
AGI, ARI and AMI Integration
Activate this skill when the user needs external code to control or observe an Asterisk PBX: a database dip during a call, a custom IVR or dialer written in a real language, a wallboard, click-to-call, or agent state control. Triggers on "AGI," "FastAGI," "ARI," "Stasis," "AMI," "manager.conf," "ari.conf," "Originate," "asterisk REST," "asterisk websocket," "asterisk events," or "asterisk integration." Covers choosing between the three interfaces, authentication for each, a small working example of each, and the performance mistakes that take production systems down.
Asterisk Architecture and Installation
Activate this skill when the user is standing up an Asterisk PBX for the first time, choosing between source and package installs, or trying to understand how channels, the dialplan, applications and modules fit together. Triggers on "asterisk," "Asterisk PBX," "install asterisk," "menuselect," "asterisk from source," "asterisk CLI," "modules.conf," "asterisk.conf," "/etc/asterisk," "chan_pjsip," or "asterisk directory layout." Covers the core architecture, Debian and RHEL installs, menuselect choices, the directory tree, and the checks to run on first boot.
Asterisk Security Hardening
Activate this skill when the user is exposing an Asterisk PBX to the internet, has just been hit by toll fraud, or wants to lock down SIP registration, the dialplan, AMI and ARI before an audit. Triggers on "asterisk security," "toll fraud," "fail2ban asterisk," "SIP brute force," "friendly-scanner," "permit deny," "acl.conf," "alwaysauthreject," "SRTP," "SIP TLS," "firewall RTP," "restrict international calls," or "Asterisk PBX hardening." Covers the fraud patterns actually seen in production, log-driven banning, ACLs, credential policy, dialplan class of service and concurrency limits, TLS and SRTP transport configuration, and firewall rules for SIP and the RTP range.
Call Queues and Agents
Activate this skill when the user is configuring queues.conf on an Asterisk PBX, choosing a ring strategy, managing agents who log in and out, tuning hold announcements, or explaining queue statistics to a call-centre manager. Triggers on "queues.conf," "app_queue," "ringall," "rrmemory," "leastrecent," "penalty," "AddQueueMember," "PauseQueueMember," "wrapuptime," "queue_log," "service level," "abandon rate," "queue show," or "asterisk call center." Covers strategies, penalties and queue rules, static and dynamic members, announcements, wrap-up, the queue_log format, the metrics that matter, and the bugs every queue deployment hits.
CDR, CEL and Reporting
Activate this skill when the user needs billing-grade call records from an Asterisk PBX: configuring cdr.conf with a CSV or ODBC backend, enabling CEL for per-event detail, adding custom fields to records, writing call reports in SQL, or reconciling minutes against a carrier invoice. Triggers on "CDR," "cdr.conf," "Master.csv," "cdr_adaptive_odbc," "CEL," "cel.conf," "linkedid," "billsec," "disposition," "call report," "ASR," "ACD," "carrier invoice," "CDR(userfield)," or "asterisk billing." Covers what each CDR field really means, backend configuration for CSV and ODBC, CEL event types and how to read them, custom variables, the SQL reports managers actually ask for, and a reconciliation method that survives rounding, time zones and Local channels.