我有一个以NSData*
格式下载和检索的视频(以后将是NSInputStream*
)。
文件格式暂时不相关。我想通过MTLTexture(或与视频等效的视频)将视频渲染到3D对象上。
我不知道从哪里开始,因为AVFoundation框架和CoreGraphics框架中的类似乎无穷无尽。您能帮我看看我需要使用的相关课程吗?
由于@warrenm,我找到了正确的解决方案。这是我的VideoPlayer
类的一部分
-(instancetype)initWithURL:(NSURL*)url {
self = [super init];
if (self) {
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, [[Renderer sharedInstance] renderDevice], nil, &self->_textureCache);
NSError* error1;
NSString* mimeType = @"video/mp4";
NSArray* keys = @[@"tracks"];
AVAsset* videoAsset = [[AVURLAsset alloc] initWithURL:url options:@{@"AVURLAssetOutOfBandMIMETypeKey" : mimeType}];
[videoAsset loadValuesAsynchronouslyForKeys:keys completionHandler:^{
[self onVideoLoaded:videoAsset];
}
];
}
return self;
}
-(void)onVideoLoaded:(AVAsset*)videoAsset {
NSError* error;
AVKeyValueStatus status = [videoAsset statusOfValueForKey:@"tracks" error:&error];
switch (status) {
//error handlings
}
NSDictionary* settings = @{(id) kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA)};
[self setVideoPlayerOutput:[[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:settings]];
[self setVideoPlayerItem:[AVPlayerItem playerItemWithAsset:videoAsset]];
[[self videoPlayerItem] addOutput:[self videoPlayerOutput]];
[self setVideoPlayer:[[AVPlayer alloc] initWithPlayerItem:[self videoPlayerItem]]];
[[self videoPlayer] setVolume:1.0f];
[self setIsLoaded:YES];
}
要获得纹理,我具有此功能:
-(id <MTLTexture>)getCurrentFrameTexture {
CVPixelBufferRef buffer = [[self videoPlayerOutput] copyPixelBufferForItemTime:[[self videoPlayerItem] currentTime] itemTimeForDisplay:nil];
NSUInteger width = CVPixelBufferGetWidth(buffer);
NSUInteger height = CVPixelBufferGetHeight(buffer);
NSUInteger planeCount = CVPixelBufferGetPlaneCount(buffer);
CVMetalTextureRef textureRef;
CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, [self textureCache], buffer, nil, MTLPixelFormatBGRA8Unorm_sRGB, width, height, planeCount, &textureRef);
id <MTLTexture> tex = CVMetalTextureGetTexture(textureRef);
CVBufferRelease(textureRef);
CVBufferRelease(buffer);
return tex;
}