metafmt-d4xx.rst

  1.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
  2.. c:namespace:: V4L
  3
  4.. _v4l2-meta-fmt-d4xx:
  5
  6*******************************
  7V4L2_META_FMT_D4XX ('D4XX')
  8*******************************
  9
 10Intel D4xx UVC Cameras Metadata
 11
 12
 13Description
 14===========
 15
 16Intel D4xx (D435, D455 and others) cameras include per-frame metadata in their UVC
 17payload headers, following the Microsoft(R) UVC extension proposal [1_]. That
 18means, that the private D4XX metadata, following the standard UVC header, is
 19organised in blocks. D4XX cameras implement several standard block types,
 20proposed by Microsoft, and several proprietary ones. Supported standard metadata
 21types are MetadataId_CaptureStats (ID 3), MetadataId_CameraExtrinsics (ID 4),
 22and MetadataId_CameraIntrinsics (ID 5). For their description see [1_]. This
 23document describes proprietary metadata types, used by D4xx cameras.
 24
 25V4L2_META_FMT_D4XX buffers follow the metadata buffer layout of
 26V4L2_META_FMT_UVC with the only difference, that it also includes proprietary
 27payload header data. D4xx cameras use bulk transfers and only send one payload
 28per frame, therefore their headers cannot be larger than 255 bytes.
 29
 30This document implements Intel Configuration version 3 [9_].
 31
 32Below are proprietary Microsoft style metadata types, used by D4xx cameras,
 33where all fields are in little endian order:
 34
 35.. tabularcolumns:: |p{5.0cm}|p{12.5cm}|
 36
 37
 38.. flat-table:: D4xx metadata
 39    :widths: 1 2
 40    :header-rows:  1
 41    :stub-columns: 0
 42
 43    * - **Field**
 44      - **Description**
 45    * - :cspan:`1` *Depth Control*
 46    * - __u32 ID
 47      - 0x80000000
 48    * - __u32 Size
 49      - Size in bytes, include ID (all protocol versions: 60)
 50    * - __u32 Version
 51      - Version of this structure. The documentation herein covers versions 1,
 52        2 and 3. The version number will be incremented when new fields are
 53        added.
 54    * - __u32 Flags
 55      - A bitmask of flags: see [2_] below
 56    * - __u32 Gain
 57      - Gain value in internal units, same as the V4L2_CID_GAIN control, used to
 58	capture the frame
 59    * - __u32 Exposure
 60      - Exposure time (in microseconds) used to capture the frame
 61    * - __u32 Laser power
 62      - Power of the laser LED 0-360, used for depth measurement
 63    * - __u32 AE mode
 64      - 0: manual; 1: automatic exposure
 65    * - __u32 Exposure priority
 66      - Exposure priority value: 0 - constant frame rate
 67    * - __u32 AE ROI left
 68      - Left border of the AE Region of Interest (all ROI values are in pixels
 69	and lie between 0 and maximum width or height respectively)
 70    * - __u32 AE ROI right
 71      - Right border of the AE Region of Interest
 72    * - __u32 AE ROI top
 73      - Top border of the AE Region of Interest
 74    * - __u32 AE ROI bottom
 75      - Bottom border of the AE Region of Interest
 76    * - __u32 Preset
 77      - Preset selector value, default: 0, unless changed by the user
 78    * - __u8 Emitter mode (v3 only) (__u32 Laser mode for v1) [8_]
 79      - 0: off, 1: on, same as __u32 Laser mode for v1
 80    * - __u8 RFU byte (v3 only)
 81      - Spare byte for future use
 82    * - __u16 LED Power (v3 only)
 83      - Led power value 0-360 (F416 SKU)
 84    * - :cspan:`1` *Capture Timing*
 85    * - __u32 ID
 86      - 0x80000001
 87    * - __u32 Size
 88      - Size in bytes, include ID (all protocol versions: 40)
 89    * - __u32 Version
 90      - Version of this structure. The documentation herein corresponds to
 91        version xxx. The version number will be incremented when new fields are
 92        added.
 93    * - __u32 Flags
 94      - A bitmask of flags: see [3_] below
 95    * - __u32 Frame counter
 96      - Monotonically increasing counter
 97    * - __u32 Optical time
 98      - Time in microseconds from the beginning of a frame till its middle
 99    * - __u32 Readout time
100      - Time, used to read out a frame in microseconds
101    * - __u32 Exposure time
102      - Frame exposure time in microseconds
103    * - __u32 Frame interval
104      - In microseconds = 1000000 / framerate
105    * - __u32 Pipe latency
106      - Time in microseconds from start of frame to data in USB buffer
107    * - :cspan:`1` *Configuration*
108    * - __u32 ID
109      - 0x80000002
110    * - __u32 Size
111      - Size in bytes, include ID (v1:36, v3:40)
112    * - __u32 Version
113      - Version of this structure. The documentation herein corresponds to
114        version xxx. The version number will be incremented when new fields are
115        added.
116    * - __u32 Flags
117      - A bitmask of flags: see [4_] below
118    * - __u8 Hardware type
119      - Camera hardware version [5_]
120    * - __u8 SKU ID
121      - Camera hardware configuration [6_]
122    * - __u32 Cookie
123      - Internal synchronisation
124    * - __u16 Format
125      - Image format code [7_]
126    * - __u16 Width
127      - Width in pixels
128    * - __u16 Height
129      - Height in pixels
130    * - __u16 Framerate
131      - Requested frame rate per second
132    * - __u16 Trigger
133      - Byte 0: bit 0: depth and RGB are synchronised, bit 1: external trigger
134    * - __u16 Calibration count (v3 only)
135      - Calibration counter, see [4_] below
136    * - __u8 GPIO input data (v3 only)
137      - GPIO readout, see [4_] below (Supported from FW 5.12.7.0)
138    * - __u32 Sub-preset info (v3 only)
139      - Sub-preset choice information, see [4_] below
140    * - __u8 reserved (v3 only)
141      - RFU byte.
142
143.. _1:
144
145[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/uvc-extensions-1-5
146
147.. _2:
148
149[2] Depth Control flags specify which fields are valid: ::
150
151  0x00000001 Gain
152  0x00000002 Exposure
153  0x00000004 Laser power
154  0x00000008 AE mode
155  0x00000010 Exposure priority
156  0x00000020 AE ROI
157  0x00000040 Preset
158  0x00000080 Emitter mode
159  0x00000100 LED Power
160
161.. _3:
162
163[3] Capture Timing flags specify which fields are valid: ::
164
165  0x00000001 Frame counter
166  0x00000002 Optical time
167  0x00000004 Readout time
168  0x00000008 Exposure time
169  0x00000010 Frame interval
170  0x00000020 Pipe latency
171
172.. _4:
173
174[4] Configuration flags specify which fields are valid: ::
175
176  0x00000001 Hardware type
177  0x00000002 SKU ID
178  0x00000004 Cookie
179  0x00000008 Format
180  0x00000010 Width
181  0x00000020 Height
182  0x00000040 Framerate
183  0x00000080 Trigger
184  0x00000100 Cal count
185  0x00000200 GPIO Input Data
186  0x00000400 Sub-preset Info
187
188.. _5:
189
190[5] Camera model: ::
191
192  0 DS5
193  1 IVCAM2
194
195.. _6:
196
197[6] 8-bit camera hardware configuration bitfield: ::
198
199  [1:0] depthCamera
200	00: no depth
201	01: standard depth
202	10: wide depth
203	11: reserved
204  [2]   depthIsActive - has a laser projector
205  [3]   RGB presence
206  [4]   Inertial Measurement Unit (IMU) presence
207  [5]   projectorType
208	0: HPTG
209	1: Princeton
210  [6]   0: a projector, 1: an LED
211  [7]   reserved
212
213.. _7:
214
215[7] Image format codes per video streaming interface:
216
217Depth: ::
218
219  1 Z16
220  2 Z
221
222Left sensor: ::
223
224  1 Y8
225  2 UYVY
226  3 R8L8
227  4 Calibration
228  5 W10
229
230Fish Eye sensor: ::
231
232  1 RAW8
233
234.. _8:
235
236[8] The "Laser mode" has been replaced in version 3 by three different fields.
237"Laser" has been renamed to "Emitter" as there are multiple technologies for
238camera projectors. As we have another field for "Laser Power" we introduced
239"LED Power" for extra emitter.
240
241The "Laser mode" __u32 fields has been split into: ::
242   1 __u8 Emitter mode
243   2 __u8 RFU byte
244   3 __u16 LED Power
245
246This is a change between versions 1 and 3. All versions 1, 2 and 3 are backward
247compatible with the same data format and they are supported. See [2_] for which
248attributes are valid.
249
250.. _9:
251
252[9] LibRealSense SDK metadata source:
253https://github.com/IntelRealSense/librealsense/blob/master/src/metadata.h