sh_mobile_ceu_camera.rst

  1.. SPDX-License-Identifier: GPL-2.0
  2
  3Cropping and Scaling algorithm, used in the sh_mobile_ceu_camera driver
  4=======================================================================
  5
  6Author: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
  7
  8Terminology
  9-----------
 10
 11sensor scales: horizontal and vertical scales, configured by the sensor driver
 12host scales: -"- host driver
 13combined scales: sensor_scale * host_scale
 14
 15
 16Generic scaling / cropping scheme
 17---------------------------------
 18
 19.. code-block:: none
 20
 21	-1--
 22	|
 23	-2-- -\
 24	|      --\
 25	|         --\
 26	+-5-- .      -- -3-- -\
 27	|      `...            -\
 28	|          `... -4-- .   - -7..
 29	|                     `.
 30	|                       `. .6--
 31	|
 32	|                        . .6'-
 33	|                      .´
 34	|           ... -4'- .´
 35	|       ...´             - -7'.
 36	+-5'- .´               -/
 37	|            -- -3'- -/
 38	|         --/
 39	|      --/
 40	-2'- -/
 41	|
 42	|
 43	-1'-
 44
 45In the above chart minuses and slashes represent "real" data amounts, points and
 46accents represent "useful" data, basically, CEU scaled and cropped output,
 47mapped back onto the client's source plane.
 48
 49Such a configuration can be produced by user requests:
 50
 51S_CROP(left / top = (5) - (1), width / height = (5') - (5))
 52S_FMT(width / height = (6') - (6))
 53
 54Here:
 55
 56(1) to (1') - whole max width or height
 57(1) to (2)  - sensor cropped left or top
 58(2) to (2') - sensor cropped width or height
 59(3) to (3') - sensor scale
 60(3) to (4)  - CEU cropped left or top
 61(4) to (4') - CEU cropped width or height
 62(5) to (5') - reverse sensor scale applied to CEU cropped width or height
 63(2) to (5)  - reverse sensor scale applied to CEU cropped left or top
 64(6) to (6') - CEU scale - user window
 65
 66
 67S_FMT
 68-----
 69
 70Do not touch input rectangle - it is already optimal.
 71
 721. Calculate current sensor scales:
 73
 74	scale_s = ((2') - (2)) / ((3') - (3))
 75
 762. Calculate "effective" input crop (sensor subwindow) - CEU crop scaled back at
 77current sensor scales onto input window - this is user S_CROP:
 78
 79	width_u = (5') - (5) = ((4') - (4)) * scale_s
 80
 813. Calculate new combined scales from "effective" input window to requested user
 82window:
 83
 84	scale_comb = width_u / ((6') - (6))
 85
 864. Calculate sensor output window by applying combined scales to real input
 87window:
 88
 89	width_s_out = ((7') - (7)) = ((2') - (2)) / scale_comb
 90
 915. Apply iterative sensor S_FMT for sensor output window.
 92
 93	subdev->video_ops->s_fmt(.width = width_s_out)
 94
 956. Retrieve sensor output window (g_fmt)
 96
 977. Calculate new sensor scales:
 98
 99	scale_s_new = ((3')_new - (3)_new) / ((2') - (2))
100
1018. Calculate new CEU crop - apply sensor scales to previously calculated
102"effective" crop:
103
104	width_ceu = (4')_new - (4)_new = width_u / scale_s_new
105	left_ceu = (4)_new - (3)_new = ((5) - (2)) / scale_s_new
106
1079. Use CEU cropping to crop to the new window:
108
109	ceu_crop(.width = width_ceu, .left = left_ceu)
110
11110. Use CEU scaling to scale to the requested user window:
112
113	scale_ceu = width_ceu / width
114
115
116S_CROP
117------
118
119The :ref:`V4L2 crop API <crop-scale>` says:
120
121"...specification does not define an origin or units. However by convention
122drivers should horizontally count unscaled samples relative to 0H."
123
124We choose to follow the advise and interpret cropping units as client input
125pixels.
126
127Cropping is performed in the following 6 steps:
128
1291. Request exactly user rectangle from the sensor.
130
1312. If smaller - iterate until a larger one is obtained. Result: sensor cropped
132   to 2 : 2', target crop 5 : 5', current output format 6' - 6.
133
1343. In the previous step the sensor has tried to preserve its output frame as
135   good as possible, but it could have changed. Retrieve it again.
136
1374. Sensor scaled to 3 : 3'. Sensor's scale is (2' - 2) / (3' - 3). Calculate
138   intermediate window: 4' - 4 = (5' - 5) * (3' - 3) / (2' - 2)
139
1405. Calculate and apply host scale = (6' - 6) / (4' - 4)
141
1426. Calculate and apply host crop: 6 - 7 = (5 - 2) * (6' - 6) / (5' - 5)