Common Vision Blox Foundation Package Tool

C-Style	etBayerToRGB.dll
C++	Cvb::Foundation::BayerConversion
.Net API (C#, VB, F#)	Stemmer.Cvb.Foundation.BayerConversion
Python	cvb.foundation

Introduction

Image Processing Tool for Bayer to RGB conversion

An increasing number of cameras are being released that output colour images in a Bayer format.
The cameras have a single CCD with an integrated Bayer colour filter.
There are a number of reasons for having a single chip colour camera with a Bayer output, lower data rate, large selection of lenses, lower light levels.
The camera outputs raw Bayer data which is in the form of a monochrome image, the RGB colour information must be decoded from this raw data.

The BayerToRGB tool supports a Common Vision Blox compliant image e.g. either from a frame grabber, a camera or a stored image.
The format of the colour filter allows a number of different conversion methods to be used, for more information on the colour filter see Colour Mosaic Format.
The different methods each have their advantages and disadvantages, the most important of which are speed and accuracy.
With megapixel Bayer output cameras the volume of data is increased and therefore the conversion time is increased, the different conversion methods allow for either fast display or high accuracy conversion whenever necessary.
For further information on the conversion methods see RGB Conversion.

The BayerToRGB tool supports cameras with a high bit depth, up to a 16-Bit output is supported.
Although the tool can process images of greater than 8-Bits per plane the output image is in a 24-Bit RGB format.

There are 2 further important factors in converting Bayer format images, White Balance and Gamma.

• The White Balance will depend heavily on the light source being used. Every light source will have an individual spectral response and will therefore require a different white balance setup.
• Finally Gamma is an important aspect of colour display, the non-linear characteristics of the eye and/or monitors can be compensated with Gamma correction.

Features

• uses White Balance and Gamma factor for optimized result images
• optimized for speed
• supports up to 16 bit per Plane of the RGB image as input image
• output RGB image has 8 bit RGB per plane (24 bit RGB)
• is available as part of the CVB Foundation Package

Theory of operation

Colour Mosaic Format

The Bayer colour mosaic filter has a pattern as described below.
Following things can be seen in this image:

• firstly there are twice as many green pixels as red or blue, the reason for this is because it closely matches the spectral response of the eye,
• secondly in this example the first pixel is red, however it is important to remember that is not always the case, different cameras will output the data from a different point on the sensor, the image acquisition device (frame grabber or camera) will also define where the first pixel is digitized.

The BayerToRGB tool requires the user to specify the pixel mode for the image, this will be one of the following:

1. GreenBlue
2. BlueGreen
3. RedGreen
4. GreenRed

RGB Conversion

From the image we can also see that each pixel has one colour associated with it but the other two planes need to be calculated.
There are a number of different methods for the conversion each with advantages and disadvantages, these methods are outlined below:

RGB_Interpolate

This method offers the highest accuracy but also the slowest speed.
Interpolation of the nearest pixels is used to calculate the two unknown colour values for each position.
Depending on the position of the pixel either two or four pixels are used for the interpolation.
In this method full spatial resolution is maintained.

RGB_Neighbour

This method offers the fastest speed for a full resolution image but the colour accuracy is not as high as the RGB_Interpolate method.
The colour from a neighbour pixel is used for the two unknown colour values. This makes the conversion faster because there is no need for interpolation.
In this method full spatial resolution is maintained.

RGB_Half

This method is provided primarily for display.
This method creates an RGB image that is half the height and half the width of the source image, the colour accuracy is the same as the RGB_Neighbour method but the spacial accuracy is lost.
If the user needs a fast display for focusing or setup of the camera then this method will offer a full frame rate on a suitable PC.

BayerToRGB API

Each conversion method in the BayerToRGB tool is highly optimized for speed, approximate benchmarks (small ms values) for the system can be determined using the Example Application e.g. with a  megapixel 8-Bit image (1280, 1024), times should include white balance and gamma correction.

Example Application

Tool and Source Code are in the CVB Tutorial directory: cvbTutorial/BayerToRGB/VC/VCBayerToRGB

This example demonstrates how to use with the BayerToRGB Dll functions nearly all features of the tool: