Posted: Wed May 27, 2009 10:37 am Post subject: [asterisk-users] stucked calls in asterisk 1.4
hello,
i have a problem with stucked or hanging calls in asterisk 1.4.25
we had this problem before and so we upgradet from 1.2.32 to 1.4.25 but
it still exists and as i could see, happens even more.
on this server there are > 1500 clients registered all with qualify on
and we had 2 routing server with 4 E1 pstn connects on each. The connect
between this server and the 2 routing server runs over sip. This problem
only appears on this server and not on the routing server. Even if i
soft hangup a stucked channel i get the sip response 481 call/leg
transaction doesnt exists back from the routing server, so one call leg
had allready send a bye but this server hasnt closed the call.
i think there could be a network problem but also the server itself and
the switch where the 3 servers are connected is younger than 3 months
and we had the same problem with this system on an older server.
sometimes i see that most of the sip peers get unreachable or too lagged
so i think that there could be a problem for asterisk to handle that
amount of pakets.
i´ve just splittet up the traffic on 2 interfaces, so that normal
traffic from the clients comes to eth0 and the traffic from and to the
routing servers runs over eth1 but the problem still occurs.
do you have any ideas what i could do to solve this problem?
best regards
steve smith
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Wed May 27, 2009 11:48 am Post subject: [asterisk-users] stucked calls in asterisk 1.4
On Wed, May 27, 2009 at 7:32 AM, Stefan Schmidt <sst@sil.at> wrote:
Quote:
i have a problem with stucked or hanging calls in asterisk 1.4.25
only appears on this server and not on the routing server. Even if i
I'm confused. So the server where the calls get stuck has both SIP and
DAHDI/Zaptel channel calls?
And the SIP side of those calls isn't getting hungup?
It is true that if your server doesn't receive a BYE packet it will
think that the call is still there. Does your dialplan that attaches
to the routing server do a hangup at the end of the call?
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Wed May 27, 2009 12:31 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
hello
David Backeberg schrieb:
Quote:
On Wed, May 27, 2009 at 7:32 AM, Stefan Schmidt <sst@sil.at> wrote:
> i have a problem with stucked or hanging calls in asterisk 1.4.25
> only appears on this server and not on the routing server. Even if i
I'm confused. So the server where the calls get stuck has both SIP and
DAHDI/Zaptel channel calls?
There are 3 servers.
Server A call it PBX there are the sip clients connected
Server B call it gateway1 has an Sangoma Card in it
Server C call it gateway2 also has an sangoma Card in it
A call comes from server B or C to server A and then to a client, gets
stucked on Server A when PSTN side hangs up. On server B or C the call
is closed.
This also happens in other direction, if client dials out over Server a
to server b or c to the pstn net.
Quote:
And the SIP side of those calls isn't getting hungup?
thats correct.
Quote:
It is true that if your server doesn't receive a BYE packet it will
think that the call is still there. Does your dialplan that attaches
to the routing server do a hangup at the end of the call?
on the routing server the call is closed every time. There we didnt have
this problem. On this server (B+C) also terminate calls from a ser proxy
and another asterisk server but the call stucks only on server A.
best regards
steve
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Wed May 27, 2009 12:57 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
On Wed, May 27, 2009 at 9:26 AM, Stefan Schmidt <sst@sil.at> wrote:
Quote:
Server A call it PBX there are the sip clients connected
A call comes from server B or C to server A and then to a client, gets
stucked on Server A when PSTN side hangs up. On server B or C the call
You may not have properly configured your card for the way to detect
hangups. If this is going to a proprietary PBX, you may need to change
around the line signaling. Some kinds of line signaling reverse the
polarity on the line voltage to signal a hangup. Other lines don't.
The cheap trick is to try your settings both ways and when one way
works to use that.
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Wed May 27, 2009 1:37 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
David Backeberg schrieb:
Quote:
On Wed, May 27, 2009 at 9:26 AM, Stefan Schmidt <sst@sil.at> wrote:
> Server A call it PBX there are the sip clients connected
> A call comes from server B or C to server A and then to a client, gets
> stucked on Server A when PSTN side hangs up. On server B or C the call
You may not have properly configured your card for the way to detect
hangups. If this is going to a proprietary PBX, you may need to change
around the line signaling. Some kinds of line signaling reverse the
polarity on the line voltage to signal a hangup. Other lines don't.
The cheap trick is to try your settings both ways and when one way
works to use that.
its not a proprietary pbx its just a self developed asterisk and the
server where the card is recognize the hangup, but the bye from the
server (b or c) to the pbx dont work.
as i said the routing server also handles calls from an ser proxy and
another asterisk server where iax accounts terminates and this problem
is only on the pbx server.
Maybe it is a network problem but the quality of the rtp streams is ok
but i think that there are too much sip pakets for the system.
there are around 1600 sip users registerd, 300 - 400 sip channels
(register, options, notifys and invite) and 600 - 700 subscriptions so
there is much sip traffic.
this server also does rtp handling and have 50 to 100 calls (active
ones) and in peek time there is around 10mbit of traffic with 5 to 6 kpps.
maybe a problem with udp buffer size??
thanks for your help so far david!
best regards
steve
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Wed May 27, 2009 2:02 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
On Wed, May 27, 2009 at 10:30 AM, Stefan Schmidt <sst@sil.at> wrote:
Quote:
as i said the routing server also handles calls from an ser proxy and
another asterisk server where iax accounts terminates and this problem
is only on the pbx server.
Maybe it is a network problem but the quality of the rtp streams is ok
but i think that there are too much sip pakets for the system.
there are around 1600 sip users registerd, 300 - 400 sip channels
(register, options, notifys and invite) and 600 - 700 subscriptions so
there is much sip traffic.
this server also does rtp handling and have 50 to 100 calls (active
ones) and in peek time there is around 10mbit of traffic with 5 to 6 kpps.
maybe a problem with udp buffer size??
Now that I better understand your problem, I'm out of ideas.
You are correct that if a BYE sip packet gets lost,
a) it won't get retransmitted if it's UDP
b) the side that's waiting for the hangup will think the call is still active
I've seen this in my system where the network switch went down while
calls were active. The system where the call was happening caught the
hangup, but the trunk system never got the bye as the network was
down.
My only questions are:
Are you using a quality network switch, or maybe you can use a
cross-over cable to eliminate collisions with other systems?
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Wed May 27, 2009 4:54 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
David Backeberg schrieb:
Quote:
Now that I better understand your problem, I'm out of ideas.
thats the point where i stand ;)
Quote:
You are correct that if a BYE sip packet gets lost,
a) it won't get retransmitted if it's UDP
b) the side that's waiting for the hangup will think the call is still active
I've seen this in my system where the network switch went down while
calls were active. The system where the call was happening caught the
hangup, but the trunk system never got the bye as the network was
down.
My only questions are:
Are you using a quality network switch, or maybe you can use a
cross-over cable to eliminate collisions with other systems?
all server are in one rack in our datacenter and are connected to an HP
Procurve 2650 switch, which has been setup around 3 months ago, cause of
the old switch died silent in the night.
all server had two interfaces and i have allready tried to route the
traffic between the pbx and the routing server over the second
interface, where database requests normally run. But this didnt solved
the problem too.
i will try to increase the UDP buffer size in the linux kernel, maybe
this will take some affect.
best regards
steve
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Wed May 27, 2009 5:34 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
On Wed, May 27, 2009 at 1:49 PM, Stefan Schmidt <sst@sil.at> wrote:
Quote:
all server are in one rack in our datacenter and are connected to an HP
Procurve 2650 switch, which has been setup around 3 months ago, cause of
the old switch died silent in the night.
all server had two interfaces and i have allready tried to route the
traffic between the pbx and the routing server over the second
interface, where database requests normally run. But this didnt solved
the problem too.
i will try to increase the UDP buffer size in the linux kernel, maybe
this will take some affect.
I will say that asterisk-1.6 is supposed to have a better SIP stack
than 1.4. Perhaps the difference in performance will help you.
Specifically, check out:
Posted: Thu May 28, 2009 7:57 am Post subject: [asterisk-users] stucked calls in asterisk 1.4
David Backeberg schrieb:
Quote:
On Wed, May 27, 2009 at 1:49 PM, Stefan Schmidt <sst@sil.at> wrote:
> all server are in one rack in our datacenter and are connected to an HP
> Procurve 2650 switch, which has been setup around 3 months ago, cause of
> the old switch died silent in the night.
>
> all server had two interfaces and i have allready tried to route the
> traffic between the pbx and the routing server over the second
> interface, where database requests normally run. But this didnt solved
> the problem too.
>
> i will try to increase the UDP buffer size in the linux kernel, maybe
> this will take some affect.
I will say that asterisk-1.6 is supposed to have a better SIP stack
than 1.4. Perhaps the difference in performance will help you.
Specifically, check out:
Posted: Thu May 28, 2009 10:24 am Post subject: [asterisk-users] stucked calls in asterisk 1.4
On Thu, May 28, 2009 at 10:49:38AM +0200, Stefan Schmidt wrote:
Quote:
David Backeberg schrieb:
> On Wed, May 27, 2009 at 1:49 PM, Stefan Schmidt <sst@sil.at> wrote:
>> all server are in one rack in our datacenter and are connected to an HP
>> Procurve 2650 switch, which has been setup around 3 months ago, cause of
>> the old switch died silent in the night.
>>
>> all server had two interfaces and i have allready tried to route the
>> traffic between the pbx and the routing server over the second
>> interface, where database requests normally run. But this didnt solved
>> the problem too.
>>
>> i will try to increase the UDP buffer size in the linux kernel, maybe
>> this will take some affect.
>
> I will say that asterisk-1.6 is supposed to have a better SIP stack
> than 1.4. Perhaps the difference in performance will help you.
> Specifically, check out:
>
> http://svn.digium.com/svn/asterisk/branches/1.6.0/CHANGES
> http://svn.digium.com/svn/asterisk/branches/1.6.1/CHANGES
>
> I'd recommend you go to at least 1.6.1.* series
i will have a look at 1.6.1 version but the problem is a network/kernel
problem.
i´ve tried to set the udp rcv buffer to 1mb but that didnt change the
problem.
Hi
I am new to asterisk so my suggestions might be a bit silly.
Why not setup a iax2 connection bettween the asterisk servers, because
its a lower overhear and more efficient.
Quote:
what i can see is that the asterisk itself sometimes lag so no call can
be started, but running calls wont be broken.
in netstat -su i see a big amount of udp packet receive errors when it
lags. Like 600 to 800 pakets per second.
if you are seeing packet loss, have you check the stats on the cards and
the ports on the switch any sort of loss might indicate a faulty
card/port
you could also try some network testing with iperf by default it tests
udp.
the other thought is keep going the way youwere add another nic card to
server a such that you have
eth0 for your sip clients
eth1 for router server b
eth2 for router server c
it would just take a bit of routing magic. The thing to watch then is
the bus these card are on if they share the same pci bus. But you are
not really moving around much data.
Try linux-kernel mailing list or do a google for udp buffer tunning
from memory there is the kernel option which is the abs max, but the
program must also select a buffer amount which is usually the default
value which is not the max value.
another thing to look for (you haven't mentioned you nic type or kernel
version), but I had a realtek 8168b which would sort of work with the in
kernel driver (8111c) on any kernel < 2.6.28. After 2.6.28 the in kernel
driver worked really well. system would be under load packets would be
come corrupt
Posted: Thu May 28, 2009 11:20 am Post subject: [asterisk-users] stucked calls in asterisk 1.4
Alex Samad schrieb:
Quote:
Hi
Hi Alex,
Quote:
I am new to asterisk so my suggestions might be a bit silly.
Why not setup a iax2 connection bettween the asterisk servers, because
its a lower overhear and more efficient.
We had changed from iax connections to sip connections cause we had
timing problems with iax. Also in meetme and when a client send a fax
over the connection. This problems has gone after changing to sip connects.
Quote:
> what i can see is that the asterisk itself sometimes lag so no call can
> be started, but running calls wont be broken.
>
> in netstat -su i see a big amount of udp packet receive errors when it
> lags. Like 600 to 800 pakets per second.
if you are seeing packet loss, have you check the stats on the cards and
the ports on the switch any sort of loss might indicate a faulty
card/port
its not a network packet loss its in the kernel or asterisk itself.
Quote:
you could also try some network testing with iperf by default it tests
udp.
the other thought is keep going the way youwere add another nic card to
server a such that you have
eth0 for your sip clients
eth1 for router server b
eth2 for router server c
i´ve allready tried this, and the problem still occurs.
Quote:
it would just take a bit of routing magic. The thing to watch then is
the bus these card are on if they share the same pci bus. But you are
not really moving around much data.
Try linux-kernel mailing list or do a google for udp buffer tunning
from memory there is the kernel option which is the abs max, but the
program must also select a buffer amount which is usually the default
value which is not the max value.
increase the max udp buffer size was just something i´ve found to try
but i´ve set the default value to 512kB by now, maybe this will help.
Quote:
another thing to look for (you haven't mentioned you nic type or kernel
version), but I had a realtek 8168b which would sort of work with the in
kernel driver (8111c) on any kernel < 2.6.28. After 2.6.28 the in kernel
driver worked really well. system would be under load packets would be
come corrupt
Posted: Thu May 28, 2009 1:17 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
Stefan,
I'm not sure if you've considered the underlying hardware, firmware, and device drivers yet, but I was brought in to evaluate a site under heavy load and was able to stabilize things by applying vendor supplied drivers and firmware to the system.
What is the underlying hardware (E.g. Dell PowerEdge 2950 Series 3, HP DL380G5, etc.) and OS distribution?
I've seen plenty of timing issues (and resulting kernel panics) due to outdated firmware or unsupported/inappropriate drivers. To be clear, in most cases of brand named hardware, these will come from the manufacturer's repository (or website), not the OS distribution's.
For example, if you're on HP hardware with RHEL 5.X, you might want to check HP's website for the latest firmware and kernel modules (e.g. drivers) for your system.
Just a thought.
Regards,
Elliot
-----Original Message-----
From: asterisk-users-bounces@lists.digium.com [mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Stefan Schmidt
Sent: Thursday, May 28, 2009 4:50 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [asterisk-users] stucked calls in asterisk 1.4
David Backeberg schrieb:
Quote:
On Wed, May 27, 2009 at 1:49 PM, Stefan Schmidt <sst@sil.at> wrote:
> all server are in one rack in our datacenter and are connected to an HP
> Procurve 2650 switch, which has been setup around 3 months ago, cause of
> the old switch died silent in the night.
>
> all server had two interfaces and i have allready tried to route the
> traffic between the pbx and the routing server over the second
> interface, where database requests normally run. But this didnt solved
> the problem too.
>
> i will try to increase the UDP buffer size in the linux kernel, maybe
> this will take some affect.
I will say that asterisk-1.6 is supposed to have a better SIP stack
than 1.4. Perhaps the difference in performance will help you.
Specifically, check out:
This message is intended only for the use of the individual (s) or entity to which it is addressed and may contain information that is privileged, confidential, and/or proprietary to Calling Circles LLC and its affiliates. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, forwarding or copying of this communication is prohibited without the express permission of the sender. If you have received this communication in error, please notify the sender immediately and delete the original message.
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
Posted: Thu May 28, 2009 9:31 pm Post subject: [asterisk-users] stucked calls in asterisk 1.4
On Thu, May 28, 2009 at 02:15:08PM +0200, Stefan Schmidt wrote:
Quote:
Alex Samad schrieb:
> Hi
Hi Alex,
>
> I am new to asterisk so my suggestions might be a bit silly.
>
> Why not setup a iax2 connection bettween the asterisk servers, because
> its a lower overhear and more efficient.
We had changed from iax connections to sip connections cause we had
timing problems with iax. Also in meetme and when a client send a fax
over the connection. This problems has gone after changing to sip connects.
okay
Quote:
[snip]
Quote:
its not a network packet loss its in the kernel or asterisk itself.
okay
[snip]
Quote:
>
> the other thought is keep going the way youwere add another nic card to
> server a such that you have
>
> eth0 for your sip clients
> eth1 for router server b
> eth2 for router server c
i´ve allready tried this, and the problem still occurs.
i thought you had only add 1 extra nic. The thought was that 2 servers
could push 1 nic over the limit, where as a 1 nic per server possibly
not. But if you are sure its not a networking packet loss issue then
its mute
Quote:
[snip]
Quote:
>
> Try linux-kernel mailing list or do a google for udp buffer tunning
> from memory there is the kernel option which is the abs max, but the
> program must also select a buffer amount which is usually the default
> value which is not the max value.
increase the max udp buffer size was just something i´ve found to try
but i´ve set the default value to 512kB by now, maybe this will help.
At work there was a thread similiar to this, but different application
problem with udp. Some who new said (if I can remember it rightly).
there is a max/default/min for the system and an application can ask for
a bigger buff, but by default it gets default, so changing the max isn't
going to help by itself.
Quote:
> another thing to look for (you haven't mentioned you nic type or kernel
> version), but I had a realtek 8168b which would sort of work with the in
> kernel driver (8111c) on any kernel < 2.6.28. After 2.6.28 the in kernel
> driver worked really well. system would be under load packets would be
> come corrupt
I´ve recompiled with dont optimize and debug threads and when the
asterisk hangs or lag i see in core show lock the following:
[snip]
Quote:
maybe this is a bug?
I am not a asterisk person so can't comment on that. But another thought
came to mind, maybe run 2 instances of asterisk - it seems like you have
plenty of horse power here it looks like it should be able to cope (I am
presume no translating ?)
The other thing is HP have a PSP pack for there servers - they have just
started to release debian based ones as well - current alien'ed rpm
packages but its a start :)
alex
Quote:
best regards
steve
--
There is an old custom among my people. When a woman saves a man's
life, he is grateful.
-- Nona, the Kanuto witch woman, "A Private Little War",
stardate 4211.8.
_______________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
