How to set a VideoProcessor in Pexip Android?

Mobile SDKs Android
1

Hello there, I was wondering if there is a way in Pexip to set a VideoProcessor in order to catch the frames captured by the camera, make some modification to the frames.
Using WebRTC, in the VideoSource class, we can set a VideoSource for that like so:

videoSource.setVideoProcessor(object : VideoProcessor {
                override fun onCapturerStarted(p0: Boolean) { }
                override fun onCapturerStopped() {}
                override fun onFrameCaptured(inputVideoFrame: VideoFrame?) {
                  //Do processing with inputVideoFrame here
                }
                override fun setSink(sink: VideoSink?) {
                  //set sink here to send updated videoFrame back to WebRTC 
                }
            })

I was wondering if there is a way to do so in Pexip library for Android.

2

Hi, @faycal and thanks for your question!

The current stable version of the Pexip Android SDK does not allow one to access the VideoSource and set a VideoProcessor.

The following pull request will allow one to pass in an instance of VideoProcessor, albeit only if they supply a VideoCapturer as well. Once merged, you will be able to test it using a SNAPSHOT version or wait until a stable version is available.

3

Hi @drymarau thank you very much for replying to my topic and for adding the functionality to add the videoProcessor. That will help a lot :love_you_gesture:
Do you think you can also add a way to remove this videoProcessor after creating the the WebRtcLocalVideoTrack? In the pull request, I see we can pass the videoProcessor only at the creation of the WebRtcLocalVideoTrack.
Do you think you can offer a public method in the WebRtcLocalVideoTrack to be able to set this videoProcessor? That way, one can just call that method and pass it null to remove the videoProcessor.

4

Thanks for the suggestion, @faycal.

SDK doesn’t expose WebRtcLocalVideoTrack to clients and we currently can’t recommend using it directly (although you can always cast LocalVideoTrack, but this may be unsafe).

If you want to enable/disable your VideoProcessor at runtime based on some condition you can either have a flag in said VideoProcessor that controls whether it’s active or not or use delegation:

class DelegatingVideoProcessor(var delegate: VideoProcessor? = null) : VideoProcessor {

    override fun onCapturerStarted(started: Boolean) {
        delegate?.onCapturerStarted(started)
    }

    override fun onCapturerStopped() {
        delegate?.onCapturerStopped()
    }

    override fun onFrameCaptured(frame: VideoFrame) {
        delegate?.onFrameCaptured(frame)
    }

    override fun setSink(sink: VideoSink?) {
        delegate?.setSink(sink)
    }
}
5

Hi @drymarau thank you very much for pointing that out. This will be helpful.
I will try the delegate, seems to me to be the most safe approach.
Thank you very much for your time and hard work.
Approximately when can I expect a new stable release to have that changes in the pull request available ?

6

We would like to release a new version sometime this month, but the date is not set in stone.

7

Okay I really appreciate your help. thanks a lot.