Posted: Tue Jun 19, 2007 2:39 pm Post subject: [Asterisk-video] Videomixing in MeetMe
Hi Klaus,
I've understood the problem, it's here:
Quote:
[h263 @ 0xb7eafca0]bitrate tolerance too small for bitrate
CVM_CLI*> Error opening codec
CVM_CLI*> Destination 4294967295 added to Session 4
CVM_CLI*> Error 'Message: Invalid Port for destination'
It's a problem I fixed recently, when I updated ffmpeg I had this issue
too. You basically need to increase the bit_rate_tolerance value in
cvm_session.c which was 0, if I remember well. So change it as follows:
and recompile/reinstall the Confiance VideoMixer. This fixes the problem
(at least it worked for me).
About QCIF/CIF, I can see from your log that H.263 is negotiated
(payload type 34), which in Eyebeam's SDP is
a=fmtp:34 QCIF=1 MaxBR=10485
So QCIF is used, I suppose.
Let me know if it works this way!
Cheers,
Lorenzo
--
Lorenzo Miniero, Junior Researcher
Dipartimento di Informatica e Sistemistica
Università degli Studi di Napoli "Federico II"
Via Claudio 21 -- 80125 Napoli (Italy)
Phone: +390817683821 - Fax: +390817683816
Email: lorenzo.miniero@unina.it
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
Posted: Wed Jun 20, 2007 11:26 am Post subject: [Asterisk-video] Videomixing in MeetMe
HI Lorenzo - this fix works. Thus, I got mixed video. Below some comments:
- I have lots of troubles with eyebeam when video is activated. System
and eyebeam need lots of CPU resource and I can't do anything else on my PC.
- ffmpeg reports some compilation problems when started - do you know
how to solve this?
CVM_CLI*> Session 1 created
CVM_CLI*> Process thread started for Session n.1 (Conference 869)
CVM_CLI*> Process thread for Session n.1 (Conference 869) put to sleep...
ortp-message-Using permissive algorithm
CVM_CLI*> Source 1 added to Session 1
CVM_CLI*> Source thread: session 1, peer 1, RTP /27770
CVM_CLI*> RTP: OK
CVM_CLI*> Context: OK
CVM_CLI*> Decoder: OK
CVM_CLI*> Frames: OK
CVM_CLI*> Waking up the process thread...
CVM_CLI*> Process thread for Session n.1 (Conference 869) woken up
ortp-message-Using permissive algorithm
CVM_CLI*> Destination 2 added to Session 1
Compiler did not align stack variables. Libavcodec has been miscompiled
and may be very slow or crash. This is not a bug in libavcodec,
but in the compiler. Do not report crashes to FFmpeg developers.
- There are lots of ffmpeg error messages during mixing:
-- this one happens always - even if there is only one, deactiated peer
and the logo is sent out:
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
....
-- this one happens very often during mixing
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
[h263 @ 0xb7de8ca0]Error at MB: 164
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
[h263 @ 0xb7de8ca0]Error at MB: 328
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
[h263 @ 0xb7de8ca0]Error at MB: 492
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
[h263 @ 0xb7de8ca0]Error at MB: 656
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
[h263 @ 0xb7de8ca0]Error at MB: 820
[h263 @ 0xb7de8ca0]I cbpy damaged at 0 24
[h263 @ 0xb7de8ca0]Error at MB: 984
[h263 @ 0xb7de8ca0]concealing 609 DC, 609 AC, 609 MV errors
[h263 @ 0xb7de8ca0]Bad picture start code
[h263 @ 0xb7de8ca0]header damaged
...
[h263 @ 0xb7de8ca0]concealing 707 DC, 707 AC, 707 MV errors
[h263 @ 0xb7de8ca0]Bad picture start code
[h263 @ 0xb7de8ca0]header damaged
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
[h263 @ 0xb7de8ca0]Error at MB: 164
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
[h263 @ 0xb7de8ca0]Error at MB: 328
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
[h263 @ 0xb7de8ca0]Error at MB: 492
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
[h263 @ 0xb7de8ca0]Error at MB: 656
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
[h263 @ 0xb7de8ca0]Error at MB: 820
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 24
[h263 @ 0xb7de8ca0]Error at MB: 984
...
- I tested with 2 peers, thus the video was split into two parts
(left/right). One video source was shrunk (as it should be), but the
other source was enlarged (zoomed in): I found out that the enlarged
video had different settings in xlite. After setting bandwidth in xlite
from "Fast Cable ..." to "DSl, Cable..." it worked fine.
- If the videomixer is restarted, Asterisk fails to reconnect to the
videomixer.
- the videomixer does not close all UDP sockets when a conference is
terminated.
Finally I have some questions about the architecture of the videomixer.
If I understand it correctly, the Meetme application receives the video
frames from Asterisk core (which received it from chan_sip) and forwards
it to the videomixer via UDP. Videomixer receives the video, mixes it,
and sends the streams back to Meetme. Meetme sends video to the SIP clients.
Why is ortp needed? Is the communication between videomixer and MeetMe
based on RTP and Meetme acts as bridge for the RTP video packets?
regards
klaus
Quote:
Hi Klaus,
I've understood the problem, it's here:
> [h263 @ 0xb7eafca0]bitrate tolerance too small for bitrate
> CVM_CLI*> Error opening codec
> CVM_CLI*> Destination 4294967295 added to Session 4
> CVM_CLI*> Error 'Message: Invalid Port for destination'
It's a problem I fixed recently, when I updated ffmpeg I had this issue
too. You basically need to increase the bit_rate_tolerance value in
cvm_session.c which was 0, if I remember well. So change it as follows:
and recompile/reinstall the Confiance VideoMixer. This fixes the problem
(at least it worked for me).
About QCIF/CIF, I can see from your log that H.263 is negotiated
(payload type 34), which in Eyebeam's SDP is
a=fmtp:34 QCIF=1 MaxBR=10485
So QCIF is used, I suppose.
Let me know if it works this way!
Cheers,
Lorenzo
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
Posted: Wed Jun 20, 2007 1:56 pm Post subject: [Asterisk-video] Videomixing in MeetMe
Hi Klaus,
I'm glad you got the mixer working!
My answers to your comments are inline.
Quote:
HI Lorenzo - this fix works. Thus, I got mixed video. Below some comments:
- I have lots of troubles with eyebeam when video is activated. System
and eyebeam need lots of CPU resource and I can't do anything else on my PC.
Is it always so with eyebeam+video (with any other video call you have with
anyone), or only when eyebeam receives the frames from the mixer?
Quote:
- ffmpeg reports some compilation problems when started - do you know
how to solve this?
Compiler did not align stack variables. Libavcodec has been miscompiled
and may be very slow or crash. This is not a bug in libavcodec,
but in the compiler. Do not report crashes to FFmpeg developers.
I frankly don't know what may cause this. You could try a make clean and
reconfigure/recompile/reinstall ffmpeg to see it gets fixed this way.
Quote:
- There are lots of ffmpeg error messages during mixing:
-- this one happens always - even if there is only one, deactiated peer
and the logo is sent out:
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]vbv buffer overflow
....
Yes, I noticed this too. I think it's related to rate control and to the fact
that the logo is a fixed frame (i.e. no motion), but I still couldn't
understand how to handle this.
Quote:
-- this one happens very often during mixing
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
[h263 @ 0xb7de8ca0]Error at MB: 164
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
[h263 @ 0xb7de8ca0]Error at MB: 328
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
[h263 @ 0xb7de8ca0]Error at MB: 492
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
[h263 @ 0xb7de8ca0]Error at MB: 656
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
[h263 @ 0xb7de8ca0]Error at MB: 820
[h263 @ 0xb7de8ca0]I cbpy damaged at 0 24
[h263 @ 0xb7de8ca0]Error at MB: 984
[h263 @ 0xb7de8ca0]concealing 609 DC, 609 AC, 609 MV errors
[h263 @ 0xb7de8ca0]Bad picture start code
[h263 @ 0xb7de8ca0]header damaged
...
[h263 @ 0xb7de8ca0]concealing 707 DC, 707 AC, 707 MV errors
[h263 @ 0xb7de8ca0]Bad picture start code
[h263 @ 0xb7de8ca0]header damaged
[h263 @ 0xb7de8ca0]vbv buffer overflow
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
[h263 @ 0xb7de8ca0]Error at MB: 164
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
[h263 @ 0xb7de8ca0]Error at MB: 328
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
[h263 @ 0xb7de8ca0]Error at MB: 492
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
[h263 @ 0xb7de8ca0]Error at MB: 656
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
[h263 @ 0xb7de8ca0]Error at MB: 820
[h263 @ 0xb7de8ca0]I cbpc damaged at 0 24
[h263 @ 0xb7de8ca0]Error at MB: 984
...
This could be related to two aspects:
* the H.263 codec used by eyebeam could be not-100%-compliant with the
ffmpeg H.263 codec (and this is something we'd have to live with);
* there might be problems with the H.263 payload header and how I handle
it.
I'm still investigating the H.263 payload header issue, also considering that
there are three different modes for it. In fact, different settings give
different results according to the client (for example, a setting that works
fine with our client gives strange behaviour with Wengophone, while settings
that work fine with Wengophone don't work at all with our client, and the same
applies with some other clients we tested). I'll try eyebeam myself as soon as
I can to see if I can sort it out. Of course, if you have advices for this, let
me know!
Quote:
- I tested with 2 peers, thus the video was split into two parts
(left/right). One video source was shrunk (as it should be), but the
other source was enlarged (zoomed in): I found out that the enlarged
video had different settings in xlite. After setting bandwidth in xlite
from "Fast Cable ..." to "DSl, Cable..." it worked fine.
Considering how the layout for two sources is codec in the mixer, both the
videos should be shrunk (in all other layouts, sources are almost always simply
resized). The fact that changing bandwidth gave the expected result might imply
a fps-related issue: in fact, the mixer currently uses a fixed outgoing fps, so
very different in-fps and out-fps might explain this...
Quote:
- If the videomixer is restarted, Asterisk fails to reconnect to the
videomixer.
Yes, the integration is very rough at the moment, since all is work-in-progress.
I focused on the mixing aspects, and still didn't take care of reconnections
and so on. If Asterisk fails at connecting to the videomixer at boot, or the
connection gets lost, there's no way to recontact the videomixer, and a restart
is needed. I'll take care of this in the next release.
Quote:
- the videomixer does not close all UDP sockets when a conference is
terminated.
Same as above, not all resources, including connections and RTP channels, are
appropriately freed when the mixer shutdowns. I'll add this in the next
release.
Quote:
Finally I have some questions about the architecture of the videomixer.
If I understand it correctly, the Meetme application receives the video
frames from Asterisk core (which received it from chan_sip) and forwards
it to the videomixer via UDP. Videomixer receives the video, mixes it,
and sends the streams back to Meetme. Meetme sends video to the SIP clients.
Why is ortp needed? Is the communication between videomixer and MeetMe
based on RTP and Meetme acts as bridge for the RTP video packets?
Exactly, Asterisk acts as an RTP bridge for media streams. Even if the ideal
case would be a direct media flow between users and the mixer, having Asterisk
in the video path is unavoidable at the moment. In fact, such a feature would
require appropriate renegotiation, which:
1. can't be done in a transparent way for whatever channel driver (you can
only bridge channels);
2. can't be done hacking a single channel driver as chan_sip either,
considering how chan_sip is conceived at the moment; in fact, redirecting the
video flow would require a media-specific c-line for video, a granularity that
is not possible in the current chan_sip because of the fact that the SDP
processing still is not media-specific at all; a CANREINVITE would not be the
solution since it would involve all the media in the SDP (so audio too).
Quote:
regards
klaus
Thanks a lot for your precious feedback, it's much appreciated!
If you have any other question and/or advices/suggestions/criticisms about this
work, I'll be glad to hear them.
Regards,
Lorenzo
--
Lorenzo Miniero, Junior Researcher
Dipartimento di Informatica e Sistemistica
Università degli Studi di Napoli "Federico II"
Via Claudio 21 -- 80125 Napoli (Italy)
Phone: +390817683821 - Fax: +390817683816
Email: lorenzo.miniero@unina.it
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
Posted: Wed Jun 20, 2007 4:24 pm Post subject: [Asterisk-video] Videomixing in MeetMe
Lorenzo Miniero wrote:
Quote:
Hi Klaus,
I'm glad you got the mixer working!
My answers to your comments are inline.
> HI Lorenzo - this fix works. Thus, I got mixed video. Below some comments:
>
> - I have lots of troubles with eyebeam when video is activated. System
> and eyebeam need lots of CPU resource and I can't do anything else on my PC.
>
Quote:
Is it always so with eyebeam+video (with any other video call you have with
anyone), or only when eyebeam receives the frames from the mixer?
Always - IMO a problem inside eyebeam or Windows (i have a "System"
process which takes up to 70% CPU)
Quote:
> -- this one happens very often during mixing
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
> [h263 @ 0xb7de8ca0]Error at MB: 164
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
> [h263 @ 0xb7de8ca0]Error at MB: 328
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
> [h263 @ 0xb7de8ca0]Error at MB: 492
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
> [h263 @ 0xb7de8ca0]Error at MB: 656
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
> [h263 @ 0xb7de8ca0]Error at MB: 820
> [h263 @ 0xb7de8ca0]I cbpy damaged at 0 24
> [h263 @ 0xb7de8ca0]Error at MB: 984
> [h263 @ 0xb7de8ca0]concealing 609 DC, 609 AC, 609 MV errors
> [h263 @ 0xb7de8ca0]Bad picture start code
> [h263 @ 0xb7de8ca0]header damaged
> ...
> [h263 @ 0xb7de8ca0]concealing 707 DC, 707 AC, 707 MV errors
> [h263 @ 0xb7de8ca0]Bad picture start code
> [h263 @ 0xb7de8ca0]header damaged
> [h263 @ 0xb7de8ca0]vbv buffer overflow
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 4
> [h263 @ 0xb7de8ca0]Error at MB: 164
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 8
> [h263 @ 0xb7de8ca0]Error at MB: 328
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 12
> [h263 @ 0xb7de8ca0]Error at MB: 492
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 16
> [h263 @ 0xb7de8ca0]Error at MB: 656
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 20
> [h263 @ 0xb7de8ca0]Error at MB: 820
> [h263 @ 0xb7de8ca0]I cbpc damaged at 0 24
> [h263 @ 0xb7de8ca0]Error at MB: 984
> ...
This could be related to two aspects:
* the H.263 codec used by eyebeam could be not-100%-compliant with the
ffmpeg H.263 codec (and this is something we'd have to live with);
* there might be problems with the H.263 payload header and how I handle
it.
I'm still investigating the H.263 payload header issue, also considering that
there are three different modes for it. In fact, different settings give
different results according to the client (for example, a setting that works
fine with our client gives strange behaviour with Wengophone, while settings
that work fine with Wengophone don't work at all with our client, and the same
applies with some other clients we tested). I'll try eyebeam myself as soon as
I can to see if I can sort it out. Of course, if you have advices for this, let
me know!
Sorry - no advices. But interesting to know that wengophone works too.
Which version of wengophone do you use? I never had success with using
wengophone at all (lots of crashes).
Quote:
Thanks a lot for your precious feedback, it's much appreciated!
If you have any other question and/or advices/suggestions/criticisms about this
work, I'll be glad to hear them.
Where should the discussion happen - here or on another mailing list? (I
prefer here - asterisk-dev).
btw: some feature requests for the next version ;-)
- a MeetMe option which automatically activates this caller as video
source (an implicit "videoswitch x y")
- A overlay in each video showing the name of the caller. e.g. by
setting a certain variable before calling MeetMe.
- a option to have only all other peers mixed, but not yourself. E.g.
currently if all sources are activiated, using eyebeam you see yourself
two times: 1: in the eyebam video preview, 2: in the received mixed
video. Of course this requires that each participant gets a dedicated
mixed stream which probably requires much more load for the mixing.
-more debug outpout, e,g, in "meetme list 123" with details about the
video (which source is activated, codec, ...)
btw: When activating video I always get logs like:
Incoming call: Got SIP response 415 "Unsupported Media Type" back from
83.136.33.3
Is this only a warning that my SIP client does not support some
conference extension (I saw INFO requests with XML payload)
regards
klaus
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
Posted: Wed Jun 20, 2007 6:50 pm Post subject: [Asterisk-video] Videomixing in MeetMe
On Wednesday 20 June 2007 18:14:00 Klaus Darilion wrote:
Quote:
Always - IMO a problem inside eyebeam or Windows (i have a "System"
process which takes up to 70% CPU)
Ah ok, I was afraid that it could be fault of the mixed frames, or a wrong
header.
Quote:
> This could be related to two aspects:
>
> * the H.263 codec used by eyebeam could be not-100%-compliant
> with the ffmpeg H.263 codec (and this is something we'd have to live
> with); * there might be problems with the H.263 payload header and how I
> handle it.
>
> I'm still investigating the H.263 payload header issue, also considering
> that there are three different modes for it. In fact, different settings
> give different results according to the client (for example, a setting
> that works fine with our client gives strange behaviour with Wengophone,
> while settings that work fine with Wengophone don't work at all with our
> client, and the same applies with some other clients we tested). I'll try
> eyebeam myself as soon as I can to see if I can sort it out. Of course,
> if you have advices for this, let me know!
Sorry - no advices. But interesting to know that wengophone works too.
Which version of wengophone do you use? I never had success with using
wengophone at all (lots of crashes).
I've tried with Wengophone classic. It allows you to specify another SIP
account that is not their one, even if it forgets it each time you close the
client... however, the video calls to MeetMe worked fine. I'll give you more
details about the client tomorrow (I'm not at work now), since I can't
remember if I installed it with a rpm or by compiling it.
Quote:
> Thanks a lot for your precious feedback, it's much appreciated!
> If you have any other question and/or advices/suggestions/criticisms
> about this work, I'll be glad to hear them.
Where should the discussion happen - here or on another mailing list? (I
prefer here - asterisk-dev).
It's the same for me, even if I think that the asterisk-dev wouldn't be very
interested at the moment, since they probably have, and rightly so, other
priorities over this one. So maybe keeping the discussion on asterisk-video
would be better.
Quote:
btw: some feature requests for the next version ;-)
- a MeetMe option which automatically activates this caller as video
source (an implicit "videoswitch x y")
I'll work on some basic DTMF mechanism to automize this: something like the
mute/unmute that already exists for audio.
Quote:
- A overlay in each video showing the name of the caller. e.g. by
setting a certain variable before calling MeetMe.
That would be a really cool feature! Which will require some more work, of
course... I'll study a bit how it could be done.
Quote:
- a option to have only all other peers mixed, but not yourself. E.g.
currently if all sources are activiated, using eyebeam you see yourself
two times: 1: in the eyebam video preview, 2: in the received mixed
video. Of course this requires that each participant gets a dedicated
mixed stream which probably requires much more load for the mixing.
Yes, the problem is that currently the layout is fixed for everyone. Besides,
there's not an encoding context for each user, but only one for each involved
codec. However, such a step was already in our head (each user should get a
customized profile, and not only for layout, but also for bitrate, fps and
all that is related to content adaption) so sooner or later we'll try to get
this done too.
Quote:
-more debug outpout, e,g, in "meetme list 123" with details about the
video (which source is activated, codec, ...)
You're right, this can be added without much efforts, and I'll surely do it.
Quote:
btw: When activating video I always get logs like:
Incoming call: Got SIP response 415 "Unsupported Media Type" back from
83.136.33.3
Is this only a warning that my SIP client does not support some
conference extension (I saw INFO requests with XML payload)
As soon as a source gets enabled, I request a VIDUPDATE to the source as
suggested on this mailing list when I uploaded the videoswitching some time
ago. It is used to ask the user to send a full frame, which is useful when
you add a new source to mixing. In SIP this is done with an INFO message,
even if not all clients support it, so I guess Eyebam is one of them.
Quote:
regards
klaus
Thanks again for your great help, cheers,
Lorenzo
--
Lorenzo Miniero, Junior Researcher
Dipartimento di Informatica e Sistemistica
Università degli Studi di Napoli "Federico II"
Via Claudio 21 -- 80125 Napoli (Italy)
Phone: +390817683821 - Fax: +390817683816
Email: lorenzo.miniero@unina.it
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
You can post new topics in this forum You can reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum