AVPlayer stalling on large video files using resource loader delegate

Question

I am using this approach to save the buffer data of the AVPlayer for video files. Found as the answer in this question Saving buffer data of AVPlayer.

iPhone and iPad - iOS 8.1.3

I made the necessary changes to play video and it is working very nicely except when I try to play a very long video (11-12 minutes long and about 85mb in size) the video will stall roughly 4 minutes after the connection finishes loading. I get an event for playbackBufferEmpty and a player item stalled notification.

This is the gist of the code

viewController.m
@property (nonatomic, strong) NSMutableData *videoData;
@property (nonatomic, strong) NSURLConnection *connection;
@property (nonatomic, strong) AVURLAsset *vidAsset;
@property (nonatomic, strong) AVPlayerItem *playerItem;
@property (nonatomic, strong) AVPlayerLayer *avlayer;
@property (nonatomic, strong) NSHTTPURLResponse *response;
@property (nonatomic, strong) NSMutableArray *pendingRequests;


/**
    Startup a Video
 */
- (void)startVideo
{
    self.vidAsset = [AVURLAsset URLAssetWithURL:[self videoURLWithCustomScheme:@"streaming"] options:nil];
    [self.vidAsset.resourceLoader setDelegate:self queue:dispatch_get_main_queue()];
    self.pendingRequests = [NSMutableArray array];

    // Init Player Item
    self.playerItem = [AVPlayerItem playerItemWithAsset:self.vidAsset];
    [self.playerItem addObserver:self forKeyPath:@"status" options:NSKeyValueObservingOptionNew context:NULL];

    self.player = [[AVPlayer alloc] initWithPlayerItem:self.playerItem];

    // Init a video Layer
    self.avlayer = [AVPlayerLayer playerLayerWithPlayer:self.player];
    [self.avlayer setFrame:self.view.frame];
    [self.view.layer addSublayer:self.avlayer];
}

- (NSURL *)getRemoteVideoURL
{
    NSString *urlString = [@"http://path/to/your/long.mp4"];
    return [NSURL URLWithString:urlString];
}

- (NSURL *)videoURLWithCustomScheme:(NSString *)scheme
{
    NSURLComponents *components = [[NSURLComponents alloc] initWithURL:[self getRemoteVideoURL] resolvingAgainstBaseURL:NO];
    components.scheme = scheme;
    return [components URL];
}



/**
    NSURLConnection Delegate Methods
 */
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
    NSLog(@"didReceiveResponse");
    self.videoData = [NSMutableData data];
    self.response = (NSHTTPURLResponse *)response;
    [self processPendingRequests];
}

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
    NSLog(@"Received Data - appending to video & processing request");
    [self.videoData appendData:data];
    [self processPendingRequests];
}

- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
    NSLog(@"connectionDidFinishLoading::WriteToFile");

    [self processPendingRequests];
    [self.videoData writeToFile:[self getVideoCachePath:self.vidSelected] atomically:YES];
}


/**
    AVURLAsset resource loader methods
 */

- (void)processPendingRequests
{
    NSMutableArray *requestsCompleted = [NSMutableArray array];

    for (AVAssetResourceLoadingRequest *loadingRequest in self.pendingRequests)
    {
        [self fillInContentInformation:loadingRequest.contentInformationRequest];

        BOOL didRespondCompletely = [self respondWithDataForRequest:loadingRequest.dataRequest];

        if (didRespondCompletely)
        {
            [requestsCompleted addObject:loadingRequest];

            [loadingRequest finishLoading];
        }
    }

    [self.pendingRequests removeObjectsInArray:requestsCompleted];
}


- (void)fillInContentInformation:(AVAssetResourceLoadingContentInformationRequest *)contentInformationRequest
{
    if (contentInformationRequest == nil || self.response == nil)
    {
        return;
    }

    NSString *mimeType = [self.response MIMEType];
    CFStringRef contentType = UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, (__bridge CFStringRef)(mimeType), NULL);

    contentInformationRequest.byteRangeAccessSupported = YES;
    contentInformationRequest.contentType = CFBridgingRelease(contentType);
    contentInformationRequest.contentLength = [self.response expectedContentLength];
}


- (BOOL)respondWithDataForRequest:(AVAssetResourceLoadingDataRequest *)dataRequest
{
    long long startOffset = dataRequest.requestedOffset;
    if (dataRequest.currentOffset != 0)
    {
        startOffset = dataRequest.currentOffset;
    }

    // Don't have any data at all for this request
    if (self.videoData.length < startOffset)
    {
        NSLog(@"NO DATA FOR REQUEST");
        return NO;
    }

    // This is the total data we have from startOffset to whatever has been downloaded so far
    NSUInteger unreadBytes = self.videoData.length - (NSUInteger)startOffset;

    // Respond with whatever is available if we can't satisfy the request fully yet
    NSUInteger numberOfBytesToRespondWith = MIN((NSUInteger)dataRequest.requestedLength, unreadBytes);

    [dataRequest respondWithData:[self.videoData subdataWithRange:NSMakeRange((NSUInteger)startOffset, numberOfBytesToRespondWith)]];

    long long endOffset = startOffset + dataRequest.requestedLength;
    BOOL didRespondFully = self.videoData.length >= endOffset;

    return didRespondFully;
}

- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest
{
    if (self.connection == nil)
    {
        NSURL *interceptedURL = [loadingRequest.request URL];
        NSURLComponents *actualURLComponents = [[NSURLComponents alloc] initWithURL:interceptedURL resolvingAgainstBaseURL:NO];
        actualURLComponents.scheme = @"http";

        NSURLRequest *request = [NSURLRequest requestWithURL:[actualURLComponents URL]];
        self.connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:NO];
        [self.connection setDelegateQueue:[NSOperationQueue mainQueue]];

        [self.connection start];
    }

    [self.pendingRequests addObject:loadingRequest];

    return YES;
}

- (void)resourceLoader:(AVAssetResourceLoader *)resourceLoader didCancelLoadingRequest:(AVAssetResourceLoadingRequest *)loadingRequest
{
    NSLog(@"didCancelLoadingRequest");
    [self.pendingRequests removeObject:loadingRequest];
}


/**
    KVO
 */

- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
{
    if (context == StatusObservationContext)
{
    AVPlayerStatus status = [[change objectForKey:NSKeyValueChangeNewKey] integerValue];

    if (status == AVPlayerStatusReadyToPlay) {
        [self initHud];
        [self play:NO];
    } else if (status == AVPlayerStatusFailed)
    {
        NSLog(@"ERROR::AVPlayerStatusFailed");

    } else if (status == AVPlayerItemStatusUnknown)
    {
        NSLog(@"ERROR::AVPlayerItemStatusUnknown");
    }

} else if (context == CurrentItemObservationContext) {


} else if (context == RateObservationContext) {


} else if (context == BufferObservationContext){


} else if (context == playbackLikelyToKeepUp) {

    if (self.player.currentItem.playbackLikelyToKeepUp)


    }

} else if (context == playbackBufferEmpty) {

    if (self.player.currentItem.playbackBufferEmpty)
    {
        NSLog(@"Video Asset is playable: %d", self.videoAsset.isPlayable);

        NSLog(@"Player Item Status: %ld", self.player.currentItem.status);

        NSLog(@"Connection Request: %@", self.connection.currentRequest);

        NSLog(@"Video Data: %lu", (unsigned long)self.videoData.length);


    }

} else if(context == playbackBufferFull) {


} else {

    [super observeValueForKeyPath:keyPath ofObject:object change:change context:context];
}

}

The problem seems to be that some time after the connection finishes loading, the player item buffer goes empty. My thought at the moment is that something is being deallocated when the connection finishes loading and messing up the playerItem buffer.

However at the time the buffer goes empty the playerItem status is good, the video asset is playable, the video data is good

If I throttle the wifi through charles and slow down the connection, the video will play as long as the connection does not finish loading within a few minutes of the end of the video.

If I set the connection nil on the finished loading event, the resource loader will fire up a new connection when shouldWaitForLoadingOfRequestedResource fires again. In this case the loading starts all over again and the video will continue playing.

I should mention that this long video plays fine if I play it as a normal http url asset, and also plays fine after being saved to the device and loaded from there.

Answer 1

when the resource loader delegate fires up the NSURLConnection, the connection takes over saving the NSData to the pending requests and processing them. when the connection finished loading, the resource loader regains responsibility for handling the loading requests. the code was adding the loading request to the pending requests array but the issue was that they were not being processed. changed the method to the following and it works.

//AVAssetResourceLoader
- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest
{
    if(isLoadingComplete == YES)
    {
        //NSLog(@"LOADING WAS COMPLETE");
        [self.pendingRequests addObject:loadingRequest];
        [self processPendingRequests];
        return YES;
    }

    if (self.connection == nil)
    {
        NSURL *interceptedURL = [loadingRequest.request URL];
        NSURLComponents *actualURLComponents = [[NSURLComponents alloc] initWithURL:interceptedURL resolvingAgainstBaseURL:NO];
        actualURLComponents.scheme = @"http";
        self.request = [NSURLRequest requestWithURL:[actualURLComponents URL]];
        self.connection = [[NSURLConnection alloc] initWithRequest:self.request delegate:self startImmediately:NO];
        [self.connection setDelegateQueue:[NSOperationQueue mainQueue]];

        isLoadingComplete = NO;
        [self.connection start];
    }

    [self.pendingRequests addObject:loadingRequest];
    return YES;
}