BBMetalImage 1.2.0

BBMetalImage 1.2.0

×

由Silence维护。

BBMetalImage 1.2.0

  • 由
  • Kaibo Lu

BBMetalImage

一个基于 Metal 的，用于 GPU 加速图像/视频处理的高性能 Swift 库。

此库极大地受到了 GPUImage 的启发。

性能

测试库是 BBMetalImage (0.1.1) 和 GPUImage (0.1.7)。测试设备是搭载了 iOS 12.1 的 iPhone 7。代码可以在 CompareImageLib 项目中找到，测试结果数据可以在 CompareImageLib.numbers 中找到。

  • BBMetalImage 在处理图像时具有较低的内存使用率。

  • BBMetalImage 在相机捕捉、处理和渲染时，拥有较低的 CPU 使用率和高速。

特性

  • 80 多个内置过滤器
  • 支持过滤器链
  • 自定义过滤器
  • 捕获视频和音频
  • 支持多摄像头
  • 处理视频文件源视频
  • 图像源提供图像纹理
  • UI 源记录视图动画
  • 显示 Metal 纹理的 Metal 视图
  • 视频写入器写入视频
  • 高性能

要求

  • iOS 10.0+
  • Swift 5

安装

使用 CocoaPods 安装

  1. 在 Podfile 中添加 use_frameworks! 和 pod 'BBMetalImage'。
  2. 运行 pod install 或 pod update。
  3. 在 Swift 源文件中添加 import BBMetalImage。

如何使用

演示

在演示项目中查看示例代码。

单滤器

调用滤器的 filteredImage(with:) 函数是获取过滤图的最简单方法。

let filteredImage = BBMetalContrastFilter(contrast: 3).filteredImage(with: image)

过滤链

捕获、预览和录制

以下代码

  1. 使用摄像头捕获图像和音频
  2. 摄像头捕获的图像经过 3 个滤器处理
  3. 处理后的图像渲染到金属视图中
  4. 处理后的图像和音频写入视频文件
  5. 写完视频文件后执行某些操作
// Hold camera and video writer
var camera: BBMetalCamera!
var videoWriter: BBMetalVideoWriter!

func setup() {
    // Set up camera to capture image
    camera = BBMetalCamera(sessionPreset: .hd1920x1080)

    // Set up 3 filters to process image
    let contrastFilter = BBMetalContrastFilter(contrast: 3)
    let lookupFilter = BBMetalLookupFilter(lookupTable: UIImage(named: "test_lookup")!.bb_metalTexture!)
    let sharpenFilter = BBMetalSharpenFilter(sharpeness: 1)

    // Set up metal view to display image
    let metalView = BBMetalView(frame: frame)
    view.addSubview(metalView)

    // Set up video writer
    let filePath = NSTemporaryDirectory() + "test.mp4"
    let url = URL(fileURLWithPath: filePath)
    videoWriter = BBMetalVideoWriter(url: url, frameSize: camera.textureSize)

    // Set camera audio consumer to record audio
    camera.audioConsumer = videoWriter

    // Set up filter chain
    camera.add(consumer: contrastFilter)
        .add(consumer: lookupFilter)
        .add(consumer: sharpenFilter)
        .add(consumer: metalView)

    sharpenFilter.add(consumer: videoWriter)

    // Start capturing
    camera.start()

    // Start writing video file
    videoWriter.start()
}

func finishRecording() {
    videoWriter.finish {
        // Do something after recording the video file
    }
}

捕获图像

使用相机捕获图像有两种方式。使用capturePhoto(completion:)函数或takePhoto()函数。

capturePhoto(completion:)函数运行速度更快，并提供了原始帧纹理。如果滤镜链中包含滤镜，我们可以使用addCompletedHandler(_:)函数获取过滤后的帧纹理。

// Hold camera
var camera: BBMetalCamera!

func setup() {
    // Set up camera to capture image
    camera = BBMetalCamera(sessionPreset: .hd1920x1080)

    // Set up metal view to display image
    let metalView = BBMetalView(frame: frame)
    view.addSubview(metalView)

    // Set up filter to process image
    let filter = BBMetalContrastFilter(contrast: 3)
    
    // Add completed handler to get filtered image
    filter.addCompletedHandler { [weak self] info in
        // Check whether is camera photo
        guard info.isCameraPhoto else { return }
        switch info.result {
        case let .success(texture):
            // Convert filtered texture to image
            let image = texture.bb_image
            DispatchQueue.main.async {
                guard let self = self else { return }
                // Display filtered image
            }
        case let .failure(error):
            // Handle error
        }
    }
    
    // Set up filter chain
    camera.add(consumer: filter)
        .add(consumer: metalView)

    // Start capturing
    camera.start()
}

func takePhoto() {
    camera.capturePhoto { [weak self] info in
        // No need to check whether is camera photo
        switch info.result {
        case let .success(texture):
            // Convert filtered texture to image
            let image = texture.bb_image
            DispatchQueue.main.async {
                guard let self = self else { return }
                // Display filtered image
            }
        case let .failure(error):
            // Handle error
        }
    }
}

takePhoto()函数运行速度较慢，并提供了原始帧纹理。

// Hold camera
var camera: BBMetalCamera!

func setup() {
    // Set up camera to capture image
    // Set `canTakePhoto` to true and set `photoDelegate` to nonnull
    camera = BBMetalCamera(sessionPreset: .hd1920x1080)
    camera.canTakePhoto = true
    camera.photoDelegate = self

    // Set up metal view to display image
    let metalView = BBMetalView(frame: frame)
    view.addSubview(metalView)

    // Set up filter chain
    camera.add(consumer: metalView)

    // Start capturing
    camera.start()
}

func takePhoto() {
    camera.takePhoto()
}

// BBMetalCameraPhotoDelegate
func camera(_ camera: BBMetalCamera, didOutput texture: MTLTexture) {
    // Do something to the photo texture
    // Note: the `texture` is the original photo which is not filtered even though there are filters in the filter chain
}

处理视频文件

// Hold video source and writer
var videoSource: BBMetalVideoSource!
var videoWriter: BBMetalVideoWriter!

func setup() {
    // Set up video writer
    let filePath = NSTemporaryDirectory() + "test.mp4"
    let outputUrl = URL(fileURLWithPath: filePath)
    videoWriter = BBMetalVideoWriter(url: outputUrl, frameSize: BBMetalIntSize(width: 1080, height: 1920))

    // Set up video source
    let sourceURL = Bundle.main.url(forResource: "test_video_2", withExtension: "mov")!
    videoSource = BBMetalVideoSource(url: sourceURL)

    // Set video source audio consumer to write audio data
    videoSource.audioConsumer = videoWriter

    // Set up 3 filters to process image
    let contrastFilter = BBMetalContrastFilter(contrast: 3)
    let lookupFilter = BBMetalLookupFilter(lookupTable: UIImage(named: "test_lookup")!.bb_metalTexture!)
    let sharpenFilter = BBMetalSharpenFilter(sharpeness: 1)

    // Set up filter chain
    videoSource.add(consumer: contrastFilter)
        .add(consumer: lookupFilter)
        .add(consumer: sharpenFilter)
        .add(consumer: videoWriter)

    // Start receiving Metal texture and writing video file
    videoWriter.start()

    // Start reading and processing video frame and auido data
    videoSource.start { [weak self] (_) in
        // All video data is processed
        guard let self = self else { return }

        // Finish writing video file
        self.videoWriter.finish {
            // Do something after writing the video file
        }
    }
}

同步处理图像

// Set up image source
let imageSource = BBMetalStaticImageSource(image: image)

// Set up 3 filters to process image
let contrastFilter = BBMetalContrastFilter(contrast: 3)
let lookupFilter = BBMetalLookupFilter(lookupTable: UIImage(named: "test_lookup")!.bb_metalTexture!)
let sharpenFilter = BBMetalSharpenFilter(sharpeness: 1)

// Set up filter chain
// Make last filter run synchronously
imageSource.add(consumer: contrastFilter)
    .add(consumer: lookupFilter)
    .add(consumer: sharpenFilter)
    .runSynchronously = true

// Start processing
imageSource.transmitTexture()

// Get filtered image
let filteredImage = sharpenFilter.outputTexture?.bb_image

异步处理图像

// Hold image source
var imageSource: BBMetalStaticImageSource!

func process() {
    // Set up image source
    imageSource = BBMetalStaticImageSource(image: image)
    
    // Set up 3 filters to process image
    let contrastFilter = BBMetalContrastFilter(contrast: 3)
    let lookupFilter = BBMetalLookupFilter(lookupTable: UIImage(named: "test_lookup")!.bb_metalTexture!)
    let sharpenFilter = BBMetalSharpenFilter(sharpeness: 1)
    
    // Set up filter chain
    // Add complete handler to last filter
    weak var wLastFilter = sharpenFilter
    imageSource.add(consumer: contrastFilter)
        .add(consumer: lookupFilter)
        .add(consumer: sharpenFilter)
        .addCompletedHandler { [weak self] _ in
            if let filteredImage = wLastFilter?.outputTexture?.bb_image {
                DispatchQueue.main.async {
                    guard let self = self else { return }
                    // Display filtered image
                }
            }
    }
    
    // Start processing
    imageSource.transmitTexture()
}

录制视图动画

使用BBMetalUISource捕获UIView快照并传递纹理。

// Hold UI source and video writer
var uiSource: BBMetalUISource!
var videoWriter: BBMetalVideoWriter!

func setup() {
    // Set up UI source with a view
    uiSource = BBMetalUISource(view: animationView)
    
    // Set up filter
    let filter = BBMetalContrastFilter(contrast: 3)
    
    // Set up video writer
    let filePath = NSTemporaryDirectory() + "test.mp4"
    let outputUrl = URL(fileURLWithPath: filePath)
    let frameSize = uiSource.renderPixelSize!
    videoWriter = BBMetalVideoWriter(url: outputUrl, frameSize: BBMetalIntSize(width: Int(frameSize.width), height: Int(frameSize.height)))
    
    // Set up filter chain
    uiSource.add(consumer: filter)
        .add(consumer: videoWriter)
    
    // Start recording
    videoWriter.start()
}

@objc func refreshDisplayLink(_ link: CADisplayLink) {
    // Update UI
    // ...
    
    // Transmit texture and video frame sample time
    // Repeat this step for each video frame
    // If `CADisplayLink` is used for the view animation, repeat this step in the target selector
    uiSource.transmitTexture(with: sampleTime)
}

func finishRecording() {
    videoWriter.finish {
        // Do something after recording the video file
    }
}

内置滤镜

  • 亮度（Brightness）
  • 曝光（Exposure）
  • 对比度（Contrast）
  • 饱和度（Saturation）
  • 伽玛（Gamma）
  • 级别（Levels）
  • 颜色矩阵（Color Matrix）
  • RGBA
  • 色调（Hue）
  • 活力（Vibrance）
  • 白平衡（White Balance）
  • 高光阴影（Highlight Shadow）
  • 高光阴影色调（Highlight Shadow Tint）
  • 查找
  • 颜色反转
  • 灰度
  • 伪彩色
  • 雾霾
  • 亮度
  • 亮度阈值
  • 腐蚀
  • RGBA腐蚀
  • 膨胀
  • RGBA膨胀
  • 色键
  • 裁剪
  • 调整大小
  • 旋转
  • 翻转
  • 变换
  • 锐化
  • 非锐化遮罩
  • 高斯模糊
  • 盒状模糊
  • 缩放模糊
  • 运动模糊
  • 倾斜模糊
  • 混合模式
    • 正常
    • 色键
    • 溶解
    • 添加
    • 减去
    • 乘以
    • 除以
    • 叠加
    • 变暗
    • 变亮
    • 颜色
    • 颜色交融
    • 颜色减淡
    • 屏幕
    • 排除
    • 差异
    • 硬光
    • 柔光
    • Alpha
    • 源覆盖
    • 色调（Hue）
    • 饱和度（Saturation）
    • 亮度
    • 线性减淡
    • 遮罩
  • 像素化
  • 极地像素化
  • 圆点
  • 网眼
  • 交叉线
  • 素描
  • 阈值素描
  • 卡通
  • 符号化
  • 晕影
  • 久保田
  • 漩涡
  • 膨胀
  • 压痕
  • 3x3卷积
  • 浮雕
  • 索贝尔边缘检测
  • 双边模糊
  • 美丽

许可协议

BBMetalImage 在MIT许可证下发布。请参阅LICENSE以获取详细信息。