dynamically (un)link elements in a running (gstreamer) pipeline?
there are plenty of examples in the gstreamer documentation on constructing and running static pipelines. however, there isn't much about changing/relinking elements in a live pipeline - while the media is actually flowing. it's definitely possible, so the question is:
- what gstreamer concepts/mechanics should i understand before attempting this?
- are there any pitfalls to watch out for?
- what is the basic procedure, or a good example?
accepted answer will be spoon fed, comprehensive, and with source code
Answer
My favorite "concept" for understanding linking (and dynamic linking), is thinking about the pipeline as a real pipe with water streaming through it. Once you do this, some things will become very obvious. Like, "do you set the source to PLAYING before linking the element?", becomes "do you turn on the water before connecting the hose?", and it sort of answers itself. Even more so with dynamic linking, how would you ensure that no water "leaks" (that is bad, "leaks" in GStreamer is the equivalent of getting a GST_FLOW_NOT_LINKED, and will stop your source and the fun) or get clogged up (can cause dropping or congestion of packets).
Yes. Many. With a little disclaimer that I still currently work with 0.10 and some of this might have been fixed with 1.0, there is unfortunately very, very hard to do dynamic linking and unlinking with GStreamer 0.10. Let me explain: Let's say you are using a Tee, and you want to unlink one branch. You would start by releasing the Tees srcpad (never mind unlinking it, that happens as part of the release of the pad), and now you should safely be able to tear down the elements downstream from that pad. (The water equivalent is that you close a valve after the tee, and should now be able to dismantle the pipes after the valve, you would not start dismantling the pipes without closing the valve first unless you wanted to get wet...) This will work most of the time, but there is a race here. Because after you have released the pad, there might still be a push or a pad-alloc on their way on that pad, and if you now in your code start tearing down the downstream elements, this might now crash because of the race that exists in some elements if they get a push or pad-alloc while tearing down, or you get a GST_FLOW_WRONG_STATE or GST_FLOW_NOT_LINKED and they will go back to the source stopping the stream for everyone...
I did a lot of experiments with this, and found that if you need stability, and crashing/freezing occasionally is not an option you need an element that will serve as your dynamic safety-net. An element that will guarantee that absolutely no activity will happen on a pad after you release/unlink it. The only way to do this is to break another GStreamer paradigm of not pushing while holding a lock: you need to hold a lock while pushing / sending events / pad-allocing. I made such a thing a while back here. (test-case being the most important thing of course, as it allows you to test your own / other elements for their safeness) You could also imagine a lock-free element that would swallow all bad FlowReturns, and just paint a pretty picture for its upstream, but then you would need to be absolutely sure that all your downstream-elements would be "push or pad-alloc received while shutting down"-safe, since your element would not be able to guarantee that once "stop the flow" (release/unlink) has been executed, a little drop would not squeeze past.
Of course you have to put some of this in perspective. The window for these terrible race-conditions I am talking about is in fact very, very small, and might only happen every 1000th or 10.000th time you run your program. But for a professional application this is of course not acceptable. I did a talk where I covered some of this stuff here