Live Streaming ( Socket ) - How to sync audio and video?

sockets streaming video-streaming live-streaming
aryaxt picture aryaxt · Oct 29, 2010 · Viewed 7.6k times · Source

1- Which one is better to use for streaming video ? TCP or UDP socket and why?

2- While streaming live, audio and video are coming from the server separately, so how can i make sure that the video i display and the audio I play on the device are in sync?

Answer

David Titarenco picture David Titarenco · Oct 29, 2010

I wrote a voice chat application a while ago and TCP was out of the question, UDP multicasting is really the only way to go if you're looking for near-realtime data flow. There's two main issues with streaming stuff over UDP though:

  1. The dropped packets. In the case of audio, it's a pretty easy fix. Usually the dropped packets won't make an audible difference (the packets are decompressed individually). However, when dealing with video, especially if the video is compressed (it usually is), figuring out a proper transfer protocol that ensures network robustness is a daunting task to say the least, especially if you're doing this from scratch. Video frames are split up in various packets. Figuring out what to do when these packets are missing is tough.
  2. Synchronization between audio and video. This is a very tough problem and I suggest reading up on protocols such as RTSP (Real-Time Streaming Protocol). This is not an easy task, but here's some introductory info: http://www.cs.columbia.edu/~hgs/rtsp/ - sometimes it's done by sending separate sync packets (some protocols send these over TCP) that tell the player how the sound should match up with the video.

Related questions

an attempt was made to access a socket in a way forbbiden by its access permissions. why?

private void StartReceivingData(string ipAddress, int iPort) { try { if (!_bContinueReciving) { //initializeMainSocket(ipAddress, iPort); _mSocket = new Socket(AddressFamily.InterNetwork, SocketType.Raw, ProtocolType.IP);//<------HERE IS RAISED THE EXCEPTION _mSocket.Bind(new IPEndPoint(IPAddress.Parse(ipAddress), iPort)); // _mSocket.Bind(new IPEndPoint(…

Read More
Chrome hangs after certain amount of data transfered - waiting for available socket

I've got a browser game and I have recently started adding audio to the game. Chrome does not load the whole page and gets stuck at "91 requests | 8.1 MB transferred" and does not load any more content; and it even breaks …

Read More
How can I code a server/client video and audio streaming application?

I have to create a client/server system to stream video and audio. It would be very simple. Like youtube style. The server should attend clients providing a list of medias first and waiting the choice of each client to …

Read More