HDR-What is HDR Imaging & How to generate it?
What is an HDR image ??
High-dynamic-range imaging (HDRI) is a high dynamic range (HDR) technique used in imaging and photography to reproduce a greater dynamic range of luminosity than is possible with standard digital imaging or photographic techniques.
HDR vs LDR images
What is the Need of HDR imaging techniques??
But why do we need to increase the range of luminosity. The answer lies in our eyes -literally . The fact is that our eyes are many times more sensitive to our colorful world as compared to devices. This can also be evidenced from the image which clearly indicates difference between visible color gamut and sRGB gamut. That caused people to find solutions to this problem.
There are basically two ways to approach this problem.
- Improving Hardware: This branch of solution basically includes improving the hardware technologies of the camera , like increasing sensitivity of lens etc. But the basic problem with this is that hardware can only be improved to a certain extent. Also, generally special circumstances like static images backgrounds (i.e images shouldn’t change during the process)or taking multiple images are required in order to generate HDR images, which is not feasible.
- Image Processing based Techniques : These techniques are more feasible as they involve modifying a LDR or SDR image into HDR image. These techniques don’t require any special requirements, like static images or taking multiple snaps, all you need is one image and the rest will be done by your code.
HDR Image Generation Methods :
Presently, with the advent of deep learning techniques like CNNs, GANs etc, HDR image generated are almost similar to what we see, sometimes even better (LOL !!). But through this article I want to take you through the journey of how HDR imaging techniques developed.
Basically, after reading various techniques proposed by researchers, I group all the HDRI techniques into the following categories :
- Methods using Multiple Images
- Manual Methods using Single image
- Deep Learning methods using Single Image
HDRI Image Generation using Multiple Images:
Basic Principle:
The basic idea behind these type of techniques is to combine images of varying illuminations and then combine these images according to a particular scheme, called tone mapping. General method to generate these images of varied illuminations is to capture the images with different exposure duration. The longer the exposure, the more illuminated the image will be.
Then we combine these images with varied illuminations according to a particular scheme or a mathematical equation. One such method is to use tone mapping. The definition for tone mapping in wikipedia is given as :
Tone mapping is a technique used in image processing and computer graphics to map one set of colors to another to approximate the appearance of high-dynamic-range images in a medium that has a more limited dynamic range.
Some of the popular tone mapping techniques are :
- Drago
- Durand
- Reinhard
- Mantiuk
But there are numerous difficulties with these methods. Some of them include :
- The images should not change across the different exposures. If this happens, various defects such as ghosting occur.
- The method is highly dependent on the different images used to generate the images. If the images have biased exposures , then the HDR image will also not be proper.
- This method is definitely cumbersome and time consuming and non-versatile. Real time or human-populated setting can not be used to generated HDR images using this method
HDR Image generation using Single Image:
This category of HDR image generation comprises of 2 groups :
- Man-made methods like filters, tone mapping etc.
- Deep Learning methods like GANs etc
Man-made Methods:
These methods are based on the same principle as the methods using multiple images, i.e these methods also aim to develop multiple images with varied illuminations and then combining them to generate HDR images, just the difference being that these multiple images are made internally by the method.
As can be seen from the above image, the method firstly creates two images depicting the Illumination and reflectance of the original image. The illumination image can be understood as the grayscale version of the original image where the value of the pixel depicts the illumination of that pixel.Whereas the reflectance can be understood as the portion of the original image which describes the color characteristics.
Then this method generates images of various illuminations and the combine them to form an image which depicts the illumination in proportion to real world images (of higher illumination range).Afterwards, both the illumination and reflectance images is combined to form the final HDR image.
Another similar method is depicted in the above image, where with the help of filter, the original image is divide into Base and Detail layer (in contrast to illumination and reflectance images). Then different images are created from the base image and then combined to form a base image with a wide illumination range. The detail image preserves the fine edges and other such defining characteristics which should be preserved.
Another example of this category of HDR Image generation techniques can be evidenced by the following image :
These methods also have their pros and cons. Some of the advantages of these methods include :
- fast, can be used in real time ecosystems
- require less computational power
- can easily be manipulated according to users needs
But there are disadvantages of these methods also :
- The images generated are not smooth, their edges and other details may change.
- These methods can sometimes reduce the resolution of the images.
Deep Learning based HDR Image Generation Methods:
Now-a-days, with the advent of higher computation capabilities,deep learning methods, which involve calculation of million of parameters, have become realistic. Similarly, these deep learning methods can also be used to generate HDR images, and the images generated are indeed very good, sometimes even exceeding the original scenario in beauty or attractiveness.
One of the deep learning methods to generate HDR images can be seen from the above diagram.
The advantages of these deepl earing method is obvious, that it generally produces the best results.But as discussed earlier, these methods have their cons too:
- They require huge amount of training images, which may prove to be a difficult task.
- The time taken during training phase is huge (once it took me a month to train a model).
References :
- High Dynamic Range Imaging by Rafał K. Mantiuk, Karol Myszkowski and Hans-Peter Seidel, April 18, 2016
- A comparative review of tone-mapping algorithms for high dynamic range video by G. Eilertsen, R. K. Mantiuk, J. Unger
- Visual Salience And Stack Extension Based Ghost Removal For High-dynamic-range Imaging by Zijie Wang, Qin Liu, Takeshi Ikenaga
- Contextual and Variational Contrast Enhancement by Turgay Celik, Tardi Tjahjadi, IEEE TRANSACTIONS ON IMAGE PROCESSING, 2011
- LIME: Low-light image enhancement via illumination map estimation by X Guo, Y Li, H Ling IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 26, NO. 2, FEBRUARY 2017
- Naturalness preserved enhancement algorithm for non-uniform illumination images by S Wang, J Zheng, HM Hu, B Li, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 9, SEPTEMBER 2013
- A fast multi-scale retinex algorithm for color image enhancement by W Wang, B Li, J Zheng, S Xian, Proceedings of the 2008 International Conference on Wavelet Analysis and Pattern Recognition, Hong Kong, 30–31 Aug. 2008
- A fusion-based enhancing method for weakly illuminated images by Xue yang Fu, Delu Zeng,Yue Huang, Ying hao Liao,Xinghao Ding,John Paisley, European Association for Signal Processing ,2016
In this blog, I hoped to introduce the concept of HDR images and techniques for its generation. In next tutorial of this series, I have explained HDRI method which I developed namely : HDR image Geneartion method using ILP (Inverted Local Patterns)and its Saturation Compensation, which is an improvement over the method discussed in the paper “High performance High dynamic image generation by Inverted Local Patterns” by Shih-Chang Hsia, Ting-Tseng Kuo.
