[BUG] Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread causing UI unresponsiveness

[ShawnHuangS]

Describe the bug

When initiating a video call and enabling the local video, the following error occurs: Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread. Calling it on the main thread can lead to UI unresponsiveness. This issue causes my UI to freeze and become unresponsive.

Exception or Stack Trace

Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread. Calling it on the main thread can lead to UI unresponsiveness
PID: 4823, TID: 3378073
Backtrace
=================================================================
3 AzureCommunicationCalling 0x0000000106dee770 -[AVFoundationCapturer startImpl:frameRate:fourCC:] + 1116
4 libdispatch.dylib 0x00000001028b0b98 _dispatch_call_block_and_release + 32
5 libdispatch.dylib 0x00000001028b27bc _dispatch_client_callout + 20
6 libdispatch.dylib 0x00000001028c2d58 _dispatch_main_queue_drain + 1084
7 libdispatch.dylib 0x00000001028c290c _dispatch_main_queue_callback_4CF + 44
8 CoreFoundation 0x000000019b86151c 3A5F992A-D1CD-312E-BD2E-F7C66343A417 + 226588
9 CoreFoundation 0x000000019b85e218 3A5F992A-D1CD-312E-BD2E-F7C66343A417 + 213528
10 CoreFoundation 0x000000019b85d968 CFRunLoopRunSpecific + 608
11 GraphicsServices 0x00000001dfb534e0 GSEventRunModal + 164
12 UIKitCore 0x000000019dcd0edc 7BF01CFC-23F1-326A-AFD8-AD967FFECE28 + 2272988
13 UIKitCore 0x000000019dcd0518 UIApplicationMain + 340
14 UIKitCore 0x000000019df09734 7BF01CFC-23F1-326A-AFD8-AD967FFECE28 + 4601652
15 xxxxxxxxxxxxx 0x000000010084943c $sSo21UIApplicationDelegateP5UIKitE4mainyyFZ + 120
16 xxxxxxxxxxxxx 0x00000001008493b4 $s1411AppDelegateC5$mainyyFZ + 44
17 xxxxxxxxxxxxx 0x000000010084c0b8 main + 28
18 dyld 0x00000001bed7ed84 7BE2B757-3B3D-3E91-8CB7-74F3887660C7 + 23940
error in connection_block_invoke_2: Connection invalid

To Reproduce

1. Initiate a video call.
2. Enable local video.
3. Observe the UI becoming unresponsive.

Expected behavior

I expected the video call to start without causing the UI to freeze or become unresponsive.

Setup (please complete the following information):

• OS: iOS 17.4.1
• IDE : XCode
• AzureCommunicationCalling 2.10.0

Additional context
Even when attempting to call startRunning from a background thread, the issue persists.

[tjprescott]

@raosanat can you take a look?

[lucianopa-msft]

Hi @ShawnHuangS,

Can you share a small sample project with that reproducer? That would help a lot the investigation.

[ShawnHuangS]

Hi @lucianopa-msft,

Here is my code snippet:

        let groupCallLocator = GroupCallLocator(groupId: groupId)
        let joinCallOptions = JoinCallOptions()
        if hasVideo {
            getVideoOptionFromLocalVideoStreams { [self] outgoingVideoOptions in
                rendererLoacalStream()
                joinCallOptions.outgoingVideoOptions = outgoingVideoOptions
            }
        }

    func rendererLoacalStream() {
        getLocalVideoStream { [self] localStream in
            guard let localStream = localStream else { return }
            do {
                let viewOption = CreateViewOptions(scalingMode: .crop)
                localPreviewRenderer = try VideoStreamRenderer(localVideoStream: localStream) 
                DispatchQueue.main.async { [self] in
                    do {
                        let localPreivewView = try localPreviewRenderer!.createView(withOptions: viewOption)
//                        localRenderView = localPreivewView
                    } catch{
                        print("### renderer local preview error: \(error)")
                    }
                }
                print("### renderer local preview success")
            } catch {
                print("### renderer local preview error :\(error)")
            }
        }
    }

When I comment out the line let localPreivewView = try localPreviewRenderer!.createView(withOptions: viewOption), the warning Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread does not appear.

Thank you for your response.

[lucianopa-msft]

If you pause the debugger when UI frozen down, can you assess in which stack trace the Thread 1 (main) is blocked?

[ShawnHuangS]

When I set a breakpoint right after the line let localPreivewView = try localPreviewRenderer!.createView(withOptions: viewOption), the warning -[AVCaptureSession startRunning] should be called from background thread does not immediately appear. It only shows up after I press continue, and no other code is executed afterwards.

When I remove DispatchQueue.main.async, I get the warning

Main Thread Checker: UI API called on a background thread: -[UIView init]

, so I have to keep it inside DispatchQueue.main.async. However, -[AVCaptureSession startRunning] needs to be called on a background thread. Do you have a better solution for this issue?

Additionally, the freeze I am referring to is temporary. The screen resumes only after the warning -[AVCaptureSession startRunning] should be called from background thread appears in Xcode.

[lucianopa-msft]

Ok, thank you for the input, we will look into it and get back when we have more info

[ShawnHuangS]

Is it necessary to add DispatchQueue.global(qos: .background).async {} around captureSession.startRunning() in the SDK's AVCaptureSession?

In my other apps that use the camera, I get the same warning message if I don't run startRunning on a background thread.

[ShawnHuangS]

Hi @lucianopa-msft ,

I believe I've identified the cause of the freezing issue, which occurs during callClient.getDeviceManager. Moving this operation to the background thread has significantly improved the flow. However, I still encounter the warning - [AVCaptureSession startRunning] should be called from background thread, but it doesn't affect the process. Thank you

[lucianopa-msft]

Hey @ShawnHuangS,
Sorry I have missed you question before

Is it necessary to add DispatchQueue.global(qos: .background).async {} around captureSession.startRunning() in the SDK's AVCaptureSession?

In my other apps that use the camera, I get the same warning message if I don't run startRunning on a background thread.

Yes, I think that should be possible, although for us we had this under a lower level layer so we have to look into that before making any change.

I believe I've identified the cause of the freezing issue, which occurs during callClient.getDeviceManager. Moving this operation to the background thread has significantly improved the flow. However, I still encounter the warning - [AVCaptureSession startRunning] should be called from background thread, but it doesn't affect the process. Thank you

Ok, let's leave this open still so we can still carry an investigation on this issue.