diff --git a/doc/py_tutorials/py_photo/images/hdr_icon.jpg b/doc/py_tutorials/py_photo/images/hdr_icon.jpg new file mode 100644 index 0000000000..b59200bf84 Binary files /dev/null and b/doc/py_tutorials/py_photo/images/hdr_icon.jpg differ diff --git a/doc/py_tutorials/py_photo/py_hdr/images/crf.jpg b/doc/py_tutorials/py_photo/py_hdr/images/crf.jpg new file mode 100644 index 0000000000..d793655a24 Binary files /dev/null and b/doc/py_tutorials/py_photo/py_hdr/images/crf.jpg differ diff --git a/doc/py_tutorials/py_photo/py_hdr/images/exposures.jpg b/doc/py_tutorials/py_photo/py_hdr/images/exposures.jpg new file mode 100644 index 0000000000..118c53f969 Binary files /dev/null and b/doc/py_tutorials/py_photo/py_hdr/images/exposures.jpg differ diff --git a/doc/py_tutorials/py_photo/py_hdr/images/fusion_mertens.jpg b/doc/py_tutorials/py_photo/py_hdr/images/fusion_mertens.jpg new file mode 100644 index 0000000000..f852ec2e64 Binary files /dev/null and b/doc/py_tutorials/py_photo/py_hdr/images/fusion_mertens.jpg differ diff --git a/doc/py_tutorials/py_photo/py_hdr/images/ldr_debvec.jpg b/doc/py_tutorials/py_photo/py_hdr/images/ldr_debvec.jpg new file mode 100644 index 0000000000..03173c3c5f Binary files /dev/null and b/doc/py_tutorials/py_photo/py_hdr/images/ldr_debvec.jpg differ diff --git a/doc/py_tutorials/py_photo/py_hdr/images/ldr_robertson.jpg b/doc/py_tutorials/py_photo/py_hdr/images/ldr_robertson.jpg new file mode 100644 index 0000000000..25bb9dea55 Binary files /dev/null and b/doc/py_tutorials/py_photo/py_hdr/images/ldr_robertson.jpg differ diff --git a/doc/py_tutorials/py_photo/py_hdr/py_hdr.markdown b/doc/py_tutorials/py_photo/py_hdr/py_hdr.markdown new file mode 100644 index 0000000000..eeae90094d --- /dev/null +++ b/doc/py_tutorials/py_photo/py_hdr/py_hdr.markdown @@ -0,0 +1,184 @@ +High Dynamic Range (HDR) {#tutorial_py_hdr} +======================== + +Goal +---- + +In this chapter, we will + +- Learn how to generate and display HDR image from an exposure sequence. +- Use exposure fusion to merge an exposure sequence. + +Theory +------ + +High-dynamic-range imaging (HDRI or HDR) is a technique used in imaging and photography to reproduce +a greater dynamic range of luminosity than is possible with standard digital imaging or photographic +techniques. While the human eye can adjust to a wide range of light conditions, most imaging devices use 8-bits +per channel, so we are limited to only 256 levels. When we take photographs of a real +world scene, bright regions may be overexposed, while the dark ones may be underexposed, so we +can’t capture all details using a single exposure. HDR imaging works with images that use more +than 8 bits per channel (usually 32-bit float values), allowing much wider dynamic range. + +There are different ways to obtain HDR images, but the most common one is to use photographs of +the scene taken with different exposure values. To combine these exposures it is useful to know your +camera’s response function and there are algorithms to estimate it. After the HDR image has been +merged, it has to be converted back to 8-bit to view it on usual displays. This process is called +tonemapping. Additional complexities arise when objects of the scene or camera move between shots, +since images with different exposures should be registered and aligned. + +In this tutorial we show 2 algorithms (Debvec, Robertson) to generate and display HDR image from an +exposure sequence, and demonstrate an alternative approach called exposure fusion (Mertens), that +produces low dynamic range image and does not need the exposure times data. +Furthermore, we estimate the camera response function (CRF) which is of great value for many computer +vision algorithms. +Each step of HDR pipeline can be implemented using different algorithms and parameters, so take a +look at the reference manual to see them all. + + +Exposure sequence HDR +--------------------- + +In this tutorial we will look on the following scene, where we have 4 exposure +images, with exposure times of: 15, 2.5, 1/4 and 1/30 seconds. (You can download +the images from [Wikipedia](https://en.wikipedia.org/wiki/High-dynamic-range_imaging)) + +![image](images/exposures.jpg) + +### 1. Loading exposure images into a list + +The first stage is simply loading all images into a list. +In addition, we will need the exposure times for the regular HDR algorithms. +Pay attention for the data types, as the images should be 1-channel or 3-channels +8-bit (np.uint8) and the exposure times need to be float32 and in seconds. + +@code{.py} +import cv2 +import numpy as np + +# Loading exposure images into a list +img_fn = ["img0.jpg", "img1.jpg", "img2.jpg", "img3.jpg"] +img_list = [cv2.imread(fn) for fn in img_fn] +exposure_times = np.array([15.0, 2.5, 0.25, 0.0333], dtype=np.float32) +@endcode + +### 2. Merge exposures into HDR image + +In this stage we merge the exposure sequence into one HDR image, showing 2 possibilities +which we have in OpenCV. The first method is Debvec and the second one is Robertson. +Notice that the HDR image is of type float32, and not uint8, as it contains the +full dynamic range of all exposure images. + +@code{.py} +# Merge exposures to HDR image +merge_debvec = cv2.createMergeDebevec() +hdr_debvec = merge_debvec.process(img_list, times=exposure_times.copy()) +merge_robertson = cv2.createMergeRobertson() +hdr_robertson = merge_robertson.process(img_list, times=exposure_times.copy()) +@endcode + +### 3. Tonemap HDR image + +We map the 32-bit float HDR data into the range [0..1]. +Actually, in some cases the values can be larger than 1 or lower the 0, so notice +we will later have to clip the data in order to avoid overflow. + +@code{.py} +# Tonemap HDR image +tonemap1 = cv2.createTonemapDurand(gamma=2.2) +res_debvec = tonemap1.process(hdr_debvec.copy()) +tonemap2 = cv2.createTonemapDurand(gamma=1.3) +res_robertson = tonemap2.process(hdr_robertson.copy()) +@endcode + +### 4. Merge exposures using Mertens fusion + +Here we show an alternative algorithm to merge the exposure images, where +we do not need the exposure times. We also do not need to use any tonemap +algorithm because the Mertens algorithm already gives us the result in the +range of [0..1]. + +@code{.py} +# Exposure fusion using Mertens +merge_mertens = cv2.createMergeMertens() +res_mertens = merge_mertens.process(img_list) +@endcode + +### 5. Convert to 8-bit and save + +In order to save or display the results, we need to convert the data into 8-bit +integers in the range of [0..255]. + +@code{.py} +# Convert datatype to 8-bit and save +res_debvec_8bit = np.clip(res_debvec*255, 0, 255).astype('uint8') +res_robertson_8bit = np.clip(res_robertson*255, 0, 255).astype('uint8') +res_mertens_8bit = np.clip(res_mertens*255, 0, 255).astype('uint8') + +cv2.imwrite("ldr_debvec.jpg", res_debvec_8bit) +cv2.imwrite("ldr_robertson.jpg", res_robertson_8bit) +cv2.imwrite("fusion_mertens.jpg", res_mertens_8bit) +@endcode + +Results +------- + +You can see the different results but consider that each algorithm have additional +extra parameters that you should fit to get your desired outcome. Best practice is +to try the different methods and see which one performs best for your scene. + +### Debvec: + +![image](images/ldr_debvec.jpg) + +### Robertson: + +![image](images/ldr_robertson.jpg) + +### Mertenes Fusion: + +![image](images/fusion_mertens.jpg) + + +Estimating Camera Response Function +----------------------------------- + +The camera response function (CRF) gives us the connection between the scene radiance +to the measured intensity values. The CRF if of great importance in some computer vision +algorithms, including HDR algorithms. Here we estimate the inverse camera response +function and use it for the HDR merge. + +@code{.py} +# Estimate camera response function (CRF) +cal_debvec = cv2.createCalibrateDebevec() +crf_debvec = cal_debvec.process(img_list, times=exposure_times) +hdr_debvec = merge_debvec.process(img_list, times=exposure_times.copy(), response=crf_debvec.copy()) +cal_robertson = cv2.createCalibrateRobertson() +crf_robertson = cal_robertson.process(img_list, times=exposure_times) +hdr_robertson = merge_robertson.process(img_list, times=exposure_times.copy(), response=crf_robertson.copy()) +@endcode + +The camera response function is represented by a 256-length vector for each color channel. +For this sequence we got the following estimation: + +![image](images/crf.jpg) + +Additional Resources +-------------------- + +1. Paul E Debevec and Jitendra Malik. Recovering high dynamic range radiance maps from photographs. + In ACM SIGGRAPH 2008 classes, page 31. ACM, 2008. +2. Mark A Robertson, Sean Borman, and Robert L Stevenson. Dynamic range improvement through multiple exposures. + In Image Processing, 1999. ICIP 99. Proceedings. 1999 International Conference on, volume 3, pages 159–163. + IEEE, 1999. +3. Tom Mertens, Jan Kautz, and Frank Van Reeth. Exposure fusion. In Computer Graphics and Applications, 2007. + PG'07. 15th Pacific Conference on, pages 382–390. IEEE, 2007. +4. Images from [Wikipedia-HDR](https://en.wikipedia.org/wiki/High-dynamic-range_imaging) + +Exercises +--------- +1. Try all tonemap algorithms: [Drago](http://docs.opencv.org/master/da/d53/classcv_1_1TonemapDrago.html), + [Durand](http://docs.opencv.org/master/da/d3d/classcv_1_1TonemapDurand.html), + [Mantiuk](http://docs.opencv.org/master/de/d76/classcv_1_1TonemapMantiuk.html) and + [Reinhard](http://docs.opencv.org/master/d0/dec/classcv_1_1TonemapReinhard.html). +2. Try changing the parameters in the HDR calibration and tonemap methods. \ No newline at end of file diff --git a/doc/py_tutorials/py_photo/py_table_of_contents_photo.markdown b/doc/py_tutorials/py_photo/py_table_of_contents_photo.markdown index 9a8b03e900..1fd57b98ac 100644 --- a/doc/py_tutorials/py_photo/py_table_of_contents_photo.markdown +++ b/doc/py_tutorials/py_photo/py_table_of_contents_photo.markdown @@ -14,3 +14,7 @@ denoising etc. Do you have a old degraded photo with many black spots and strokes on it? Take it. Let's try to restore them with a technique called image inpainting. + +- @subpage tutorial_py_hdr + + Learn how to merge exposure sequence and process high dynamic range images. \ No newline at end of file