Skip to content

Commit

Permalink
feat: Add Android support to videoBitRate (move it to props)
Browse files Browse the repository at this point in the history
  • Loading branch information
mrousavy committed Oct 30, 2024
1 parent 48b4300 commit cddd016
Show file tree
Hide file tree
Showing 14 changed files with 93 additions and 74 deletions.
29 changes: 10 additions & 19 deletions docs/docs/guides/RECORDING_VIDEOS.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ camera.current.startRecording({
})
```

You can customize capture options such as [video codec](/docs/api/interfaces/RecordVideoOptions#videocodec), [video bit-rate](/docs/api/interfaces/RecordVideoOptions#videobitrate), [file type](/docs/api/interfaces/RecordVideoOptions#filetype), [enable flash](/docs/api/interfaces/RecordVideoOptions#flash) and more using the [`RecordVideoOptions`](/docs/api/interfaces/RecordVideoOptions) parameter.
You can customize capture options such as [video codec](/docs/api/interfaces/RecordVideoOptions#videocodec), [file type](/docs/api/interfaces/RecordVideoOptions#filetype), [enable flash](/docs/api/interfaces/RecordVideoOptions#flash) and more using the [`RecordVideoOptions`](/docs/api/interfaces/RecordVideoOptions) parameter.

For any error that occured _while recording the video_, the `onRecordingError` callback will be invoked with a [`CaptureError`](/docs/api/classes/CameraCaptureError) and the recording is therefore cancelled.

Expand Down Expand Up @@ -119,22 +119,16 @@ If the device does not support `h265`, VisionCamera will automatically fall-back

Videos are recorded with a target bit-rate, which the encoder aims to match as closely as possible. A lower bit-rate means less quality (and less file size), a higher bit-rate means higher quality (and larger file size) since it can assign more bits to moving pixels.

To simply record videos with higher quality, use a [`videoBitRate`](/docs/api/interfaces/RecordVideoOptions#videobitrate) of `'high'`, which effectively increases the bit-rate by 20%:
To simply record videos with higher quality, use a [`videoBitRate`](/docs/api/interfaces/CameraProps#videobitrate) of `'high'`, which effectively increases the bit-rate by 20%:

```ts
camera.current.startRecording({
...props,
videoBitRate: 'high'
})
```jsx
<Camera {...props} videoBitRate="high" />
```

To use a lower bit-rate for lower quality and lower file-size, use a [`videoBitRate`](/docs/api/interfaces/RecordVideoOptions#videobitrate) of `'low'`, which effectively decreases the bit-rate by 20%:
To use a lower bit-rate for lower quality and lower file-size, use a [`videoBitRate`](/docs/api/interfaces/CameraProps#videobitrate) of `'low'`, which effectively decreases the bit-rate by 20%:

```ts
camera.current.startRecording({
...props,
videoBitRate: 'low'
})
```jsx
<Camera {...props} videoBitRate="low" />
```

#### Custom Bit Rate
Expand Down Expand Up @@ -162,13 +156,10 @@ if (codec === 'h265') bitRate *= 0.8 // H.265
bitRate *= yourCustomFactor // e.g. 0.5x for half the bit-rate
```

And then pass it to the [`startRecording(...)`](/docs/api/classes/Camera#startrecording) function (in Mbps):
And then pass it to the `<Camera>` component (in Mbps):

```ts
camera.current.startRecording({
...props,
videoBitRate: bitRate // Mbps
})
```jsx
<Camera {...props} videoBitRate={bitRate} />
```

### Video Frame Rate (FPS)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ data class CameraConfiguration(
// Output<T> types, those need to be comparable
data class CodeScanner(val codeTypes: List<CodeType>)
data class Photo(val isMirrored: Boolean, val enableHdr: Boolean, val photoQualityBalance: QualityBalance)
data class Video(val isMirrored: Boolean, val enableHdr: Boolean)
data class Video(val isMirrored: Boolean, val enableHdr: Boolean, val bitRateOverride: Double?, val bitRateModifier: Double?)
data class FrameProcessor(val isMirrored: Boolean, val pixelFormat: PixelFormat)
data class Audio(val nothing: Unit)
data class Preview(val surfaceProvider: SurfaceProvider)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -123,8 +123,10 @@ internal fun CameraSession.configureOutputs(configuration: CameraConfiguration)
configuration.format?.let { format ->
recorder.setQualitySelector(format.videoQualitySelector)
}
// TODO: Make videoBitRate a Camera Prop
// video.setTargetVideoEncodingBitRate()
videoConfig.config.bitRateOverride?.let { bitRateOverride ->
video.setTargetVideoEncodingBitRate(bitRateOverride)
}
// TODO: BitRate Modifier????
}.build()
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,17 @@ import com.mrousavy.camera.core.utils.OutputFile

class RecordVideoOptions(
val file: OutputFile,
val videoCodec: VideoCodec,
val videoBitRateOverride: Double?,
val videoBitRateMultiplier: Double?
val videoCodec: VideoCodec
) {

companion object {
fun fromJSValue(context: Context, map: ReadableMap): RecordVideoOptions {
val directory = if (map.hasKey("path")) FileUtils.getDirectory(map.getString("path")) else context.cacheDir
val fileType = if (map.hasKey("fileType")) VideoFileType.fromUnionValue(map.getString("fileType")) else VideoFileType.MOV
val videoCodec = if (map.hasKey("videoCodec")) VideoCodec.fromUnionValue(map.getString("videoCodec")) else VideoCodec.H264
val videoBitRateOverride = if (map.hasKey("videoBitRateOverride")) map.getDouble("videoBitRateOverride") else null
val videoBitRateMultiplier = if (map.hasKey("videoBitRateMultiplier")) map.getDouble("videoBitRateMultiplier") else null
val outputFile = OutputFile(context, directory, fileType.toExtension())
return RecordVideoOptions(outputFile, videoCodec, videoBitRateOverride, videoBitRateMultiplier)
return RecordVideoOptions(outputFile, videoCodec)
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ import kotlinx.coroutines.launch
// TODO: takePhoto() depth data
// TODO: takePhoto() raw capture
// TODO: takePhoto() return with jsi::Value Image reference for faster capture
// TODO: Support videoCodec and videoBitRate on Android
// TODO: Support videoCodec on Android

@SuppressLint("ClickableViewAccessibility", "ViewConstructor", "MissingPermission")
class CameraView(context: Context) :
Expand Down Expand Up @@ -75,6 +75,8 @@ class CameraView(context: Context) :
var videoStabilizationMode: VideoStabilizationMode? = null
var videoHdr = false
var photoHdr = false
var videoBitRateOverride: Double? = null
var videoBitRateModifier: Double? = null

// TODO: Use .BALANCED once CameraX fixes it https://issuetracker.google.com/issues/337214687
var photoQualityBalance = QualityBalance.SPEED
Expand Down Expand Up @@ -180,7 +182,7 @@ class CameraView(context: Context) :

// Video
if (video || enableFrameProcessor) {
config.video = CameraConfiguration.Output.Enabled.create(CameraConfiguration.Video(isMirrored, videoHdr))
config.video = CameraConfiguration.Output.Enabled.create(CameraConfiguration.Video(isMirrored, videoHdr, videoBitRateOverride, videoBitRateModifier))
} else {
config.video = CameraConfiguration.Output.Disabled.create()
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,24 @@ class CameraViewManager : ViewGroupManager<CameraView>() {
view.videoHdr = videoHdr
}

@ReactProp(name = "videoBitRateOverride", defaultDouble = -1)
fun setVideoBitRateOverride(view: CameraView, videoBitRateOverride: Double) {
if (videoBitRateOverride != -1) {
view.videoBitRateOverride = videoBitRateOverride
} else {
view.videoBitRateOverride = null
}
}

@ReactProp(name = "videoBitRateModifier", defaultDouble = -1)
fun setVideoBitRateOverride(view: CameraView, videoBitRateModifier: Double) {
if (videoBitRateModifier != -1) {
view.videoBitRateModifier = videoBitRateModifier
} else {
view.videoBitRateModifier = null
}
}

@ReactProp(name = "lowLightBoost")
fun setLowLightBoost(view: CameraView, lowLightBoost: Boolean) {
view.lowLightBoost = lowLightBoost
Expand Down
10 changes: 3 additions & 7 deletions package/ios/Core/Types/RecordVideoOptions.swift
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ struct RecordVideoOptions {
*/
var bitRateMultiplier: Double?

init(fromJSValue dictionary: NSDictionary) throws {
init(fromJSValue dictionary: NSDictionary, bitRateOverride: Double? = nil, bitRateModifier: Double? = nil) throws {
// File Type (.mov or .mp4)
if let fileTypeOption = dictionary["fileType"] as? String {
fileType = try AVFileType(withString: fileTypeOption)
Expand All @@ -38,13 +38,9 @@ struct RecordVideoOptions {
codec = try AVVideoCodecType(withString: codecOption)
}
// BitRate Override
if let parsed = dictionary["videoBitRateOverride"] as? Double {
bitRateOverride = parsed
}
self.bitRateOverride = bitRateOverride
// BitRate Multiplier
if let parsed = dictionary["videoBitRateMultiplier"] as? Double {
bitRateMultiplier = parsed
}
self.bitRateModifier = bitRateModifier
// Custom Path
let fileExtension = fileType.descriptor ?? "mov"
if let customPath = dictionary["path"] as? String {
Expand Down
4 changes: 3 additions & 1 deletion package/ios/React/CameraView+RecordVideo.swift
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ extension CameraView: AVCaptureVideoDataOutputSampleBufferDelegate, AVCaptureAud
let callback = Callback(jsCallback)

do {
let options = try RecordVideoOptions(fromJSValue: options)
let options = try RecordVideoOptions(fromJSValue: options,
bitRateOverride: videoBitRateOverride,
videoBitRateModifier: videoBitRateModifier)

// Start Recording with success and error callbacks
cameraSession.startRecording(
Expand Down
6 changes: 5 additions & 1 deletion package/ios/React/CameraView.swift
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ public final class CameraView: UIView, CameraSessionDelegate, PreviewViewDelegat
@objc var photoQualityBalance: NSString?
@objc var lowLightBoost = false
@objc var outputOrientation: NSString?
@objc var videoBitRateOverride: NSNumber?
@objc var videoBitRateModifier: NSNumber?

// other props
@objc var isActive = false
Expand Down Expand Up @@ -209,7 +211,9 @@ public final class CameraView: UIView, CameraSessionDelegate, PreviewViewDelegat
config.video = .enabled(config: CameraConfiguration.Video(pixelFormat: getPixelFormat(),
enableBufferCompression: enableBufferCompression,
enableHdr: videoHdr,
enableFrameProcessor: enableFrameProcessor))
enableFrameProcessor: enableFrameProcessor,
bitRateOverride: videoBitRateOverride,
bitRateModifier: videoBitRateModifier))
} else {
config.video = .disabled
}
Expand Down
2 changes: 2 additions & 0 deletions package/ios/React/CameraViewManager.m
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ @interface RCT_EXTERN_REMAP_MODULE (CameraView, CameraViewManager, RCTViewManage
RCT_EXPORT_VIEW_PROPERTY(lowLightBoost, BOOL);
RCT_EXPORT_VIEW_PROPERTY(videoStabilizationMode, NSString);
RCT_EXPORT_VIEW_PROPERTY(pixelFormat, NSString);
RCT_EXPORT_VIEW_PROPERTY(videoBitRateOverride, NSNumber);
RCT_EXPORT_VIEW_PROPERTY(videoBitRateModifier, NSNumber);
// other props
RCT_EXPORT_VIEW_PROPERTY(torch, NSString);
RCT_EXPORT_VIEW_PROPERTY(zoom, NSNumber);
Expand Down
39 changes: 21 additions & 18 deletions package/src/Camera.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,7 @@ import { RotationHelper } from './RotationHelper'
export type CameraPermissionStatus = 'granted' | 'not-determined' | 'denied' | 'restricted'
export type CameraPermissionRequestResult = 'granted' | 'denied'

type NativeRecordVideoOptions = Omit<RecordVideoOptions, 'onRecordingError' | 'onRecordingFinished' | 'videoBitRate'> & {
videoBitRateOverride?: number
videoBitRateMultiplier?: number
}
type NativeRecordVideoOptions = Omit<RecordVideoOptions, 'onRecordingError' | 'onRecordingFinished'>
type RefType = React.Component<NativeCameraViewProps> & Readonly<NativeMethods>
interface CameraState {
isRecordingWithFlash: boolean
Expand Down Expand Up @@ -170,7 +167,7 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
}
}

private getBitRateMultiplier(bitRate: RecordVideoOptions['videoBitRate']): number {
private getBitRateMultiplier(bitRate: CameraProps['videoBitRate']): number {
if (typeof bitRate === 'number' || bitRate == null) return 1
switch (bitRate) {
case 'extra-low':
Expand Down Expand Up @@ -204,7 +201,7 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
* ```
*/
public startRecording(options: RecordVideoOptions): void {
const { onRecordingError, onRecordingFinished, videoBitRate, ...passThruOptions } = options
const { onRecordingError, onRecordingFinished, ...passThruOptions } = options
if (typeof onRecordingError !== 'function' || typeof onRecordingFinished !== 'function')
throw new CameraRuntimeError('parameter/invalid-parameter', 'The onRecordingError or onRecordingFinished functions were not set!')

Expand All @@ -215,15 +212,6 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
})
}

const nativeOptions: NativeRecordVideoOptions = passThruOptions
if (typeof videoBitRate === 'number') {
// If the user passed an absolute number as a bit-rate, we just use this as a full override.
nativeOptions.videoBitRateOverride = videoBitRate
} else if (typeof videoBitRate === 'string' && videoBitRate !== 'normal') {
// If the user passed 'low'/'normal'/'high', we need to apply this as a multiplier to the native bitrate instead of absolutely setting it
nativeOptions.videoBitRateMultiplier = this.getBitRateMultiplier(videoBitRate)
}

const onRecordCallback = (video?: VideoFile, error?: CameraCaptureError): void => {
if (this.state.isRecordingWithFlash) {
// disable torch again if it was enabled
Expand All @@ -235,9 +223,11 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
if (error != null) return onRecordingError(error)
if (video != null) return onRecordingFinished(video)
}

const nativeRecordVideoOptions: NativeRecordVideoOptions = passThruOptions
try {
// TODO: Use TurboModules to make this awaitable.
CameraModule.startRecording(this.handle, nativeOptions, onRecordCallback)
CameraModule.startRecording(this.handle, nativeRecordVideoOptions, onRecordCallback)
} catch (e) {
throw tryParseNativeCameraError(e)
}
Expand Down Expand Up @@ -626,7 +616,7 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
/** @internal */
public render(): React.ReactNode {
// We remove the big `device` object from the props because we only need to pass `cameraId` to native.
const { device, frameProcessor, codeScanner, enableFpsGraph, fps, ...props } = this.props
const { device, frameProcessor, codeScanner, enableFpsGraph, fps, videoBitRate, ...props } = this.props

// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition
if (device == null) {
Expand All @@ -645,6 +635,17 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
const minFps = fps == null ? undefined : typeof fps === 'number' ? fps : fps[0]
const maxFps = fps == null ? undefined : typeof fps === 'number' ? fps : fps[1]

// bitrate is number (override) or string (multiplier)
let bitRateMultiplier: number | undefined
let bitRateOverride: number | undefined
if (typeof videoBitRate === 'number') {
// If the user passed an absolute number as a bit-rate, we just use this as a full override.
bitRateOverride = videoBitRate
} else if (typeof videoBitRate === 'string' && videoBitRate !== 'normal') {
// If the user passed 'low'/'normal'/'high', we need to apply this as a multiplier to the native bitrate instead of absolutely setting it
bitRateMultiplier = this.getBitRateMultiplier(videoBitRate)
}

return (
<NativeCameraView
{...props}
Expand All @@ -663,13 +664,15 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
onPreviewStarted={this.onPreviewStarted}
onPreviewStopped={this.onPreviewStopped}
onShutter={this.onShutter}
videoBitRateMultiplier={bitRateMultiplier}
videoBitRateOverride={bitRateOverride}
onOutputOrientationChanged={this.onOutputOrientationChanged}
onPreviewOrientationChanged={this.onPreviewOrientationChanged}
onError={this.onError}
codeScannerOptions={codeScanner}
enableFrameProcessor={frameProcessor != null}
enableBufferCompression={props.enableBufferCompression ?? shouldEnableBufferCompression}
preview={isRenderingWithSkia ? false : (props.preview ?? true)}>
preview={isRenderingWithSkia ? false : props.preview ?? true}>

Check warning on line 675 in package/src/Camera.tsx

View workflow job for this annotation

GitHub Actions / Lint JS (eslint, prettier)

Replace `props.preview·??·true` with `(props.preview·??·true)`
{isRenderingWithSkia && (
<SkiaCameraCanvas
style={styles.customPreviewView}
Expand Down
3 changes: 3 additions & 0 deletions package/src/NativeCameraView.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,16 @@ export type NativeCameraViewProps = Omit<
| 'frameProcessor'
| 'codeScanner'
| 'fps'
| 'videoBitRate'
> & {
// private intermediate props
cameraId: string
enableFrameProcessor: boolean
codeScannerOptions?: Omit<CodeScanner, 'onCodeScanned'>
minFps?: number
maxFps?: number
videoBitRateOverride?: number
videoBitRateMultiplier?: number
// private events
onViewReady: (event: NativeSyntheticEvent<void>) => void
onAverageFpsChanged?: (event: NativeSyntheticEvent<AverageFpsChangedEvent>) => void
Expand Down
17 changes: 17 additions & 0 deletions package/src/types/CameraProps.ts
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,23 @@ export interface CameraProps extends ViewProps {
* Make sure the given {@linkcode format} supports HDR (see {@linkcode CameraDeviceFormat.supportsVideoHdr format.supportsVideoHdr}).
*/
videoHdr?: boolean
/**
* The bit-rate for encoding the video into a file, in Mbps (Megabits per second).
*
* Bit-rate is dependant on various factors such as resolution, FPS, pixel format (whether it's 10 bit HDR or not), and video codec.
*
* By default, it will be calculated by the hardware encoder, which takes all those factors into account.
*
* * `extra-low`: 40% lower than whatever the hardware encoder recommends.
* * `low`: 20% lower than whatever the hardware encoder recommends.
* * `normal`: The recommended value by the hardware encoder.
* * `high`: 20% higher than whatever the hardware encoder recommends.
* * `extra-high`: 40% higher than whatever the hardware encoder recommends.
* * `number`: Any custom number for the bit-rate, in Mbps.
*
* @default 'normal'
*/
videoBitRate?: 'extra-low' | 'low' | 'normal' | 'high' | 'extra-high' | number
/**
* Enables or disables HDR Photo Capture via a double capture routine that combines low- and high exposure photos.
*
Expand Down
17 changes: 0 additions & 17 deletions package/src/types/VideoFile.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,6 @@ export interface RecordVideoOptions {
* - `h265`: The HEVC (High-Efficient-Video-Codec) for higher efficient video recordings. Results in up to 50% smaller file-sizes.
*/
videoCodec?: 'h264' | 'h265'
/**
* The bit-rate for encoding the video into a file, in Mbps (Megabits per second).
*
* Bit-rate is dependant on various factors such as resolution, FPS, pixel format (whether it's 10 bit HDR or not), and video codec.
*
* By default, it will be calculated by the hardware encoder, which takes all those factors into account.
*
* * `extra-low`: 40% lower than whatever the hardware encoder recommends.
* * `low`: 20% lower than whatever the hardware encoder recommends.
* * `normal`: The recommended value by the hardware encoder.
* * `high`: 20% higher than whatever the hardware encoder recommends.
* * `extra-high`: 40% higher than whatever the hardware encoder recommends.
* * `number`: Any custom number for the bit-rate, in Mbps.
*
* @default 'normal'
*/
videoBitRate?: 'extra-low' | 'low' | 'normal' | 'high' | 'extra-high' | number
}

/**
Expand Down

0 comments on commit cddd016

Please sign in to comment.