|
Common Vision Blox 14.0
|
|
|
Common Vision Blox 14.0
|
Although Common Vision Blox comes with a variety of algorithms tailored to many different imaging and machine vision applications, there are sometimes good reasons to combine Common Vision Blox with other libraries in one application. Luckily, Common Vision Blox provides the necessary utilities to do so sometimes even at minimal or almost no CPU cost. This guide shows how to use Common Vision Blox together with libraries like OpenCV, IPP, Halcon or others.
To avoid lengthy wordings we will use the following abbreviations throughout this text:
| Acronym | Meaning |
|---|---|
| CVB | as always, CVB is short for Common Vision Blox |
| IPP | Short for "Intel Performance Primitives", a highly optimized and renowned imaging algorithm library |
| VPAT | Virtual Pixel Access Table – an image data access scheme available in CVB |
Although we have chosen OpenCV, IPP and Halcon as examples, it is evident that the methods and approaches outlined in this document are applicable to any library that provides access to the necessary data structures and information.
All products and libraries are property of their respective trademark holders and mention of these libraries/products does not imply an endorsement of Common Vision Blox by these companies or vice versa.
Please note that it is assumed that the reader has a firm grasp of C/C++, especially the use of pointers and pointer arithmetic – both are going to be used extensively throughout this document but will not be explained in detail. It is also assumed, that the reader is sufficiently familiar with Common Vision Blox, OpenCV, IPP or Halcon as none of those libraries will be explained beyond the extent immediately necessary for this guide.
With the exception of chapter 2, the chapters of this guide may be read independently or in sequence. Their summarized contents are:
The fundamental reason why it is not possible to simply pass images/image data between any pair of two machine vision libraries (or sometimes even between the Windows SDK or the Common Language Runtime and a machine vision library) is that there is no universally agreed format for the description of images that is understood and accepted by more than one library (or at best: by more than just a few libraries).
The main reasons for this are:
Not surprisingly the key to bridging the gap between any two libraries is to translate and/or transform the image model of library A to fit that of library B. Often (but not always) this can even be achieved without copying the image data. Of course, doing so requires knowledge of the data structures involved as well as the capabilities of the libraries involved.
Data type in this context refers to the data type of an individual pixel inside an image (i.e. if the memory address of a pixel is known, what data type will the pointer need to be cast to for dereferencing it properly).
CVB stores all the information it provides about the data type of an image’s pixel in a 32 bit value that is composed as follows:
| 31-12 | 11 | 10 | 9 | 8 | 7-0 |
|---|---|---|---|---|---|
| reserved | complex number? | overlay? | floating point number? | signed number? | bits per pixel |
The 32-bit value built after this recipe specifies the data type of an image plane's pixels. CVB in theory offers the possibility of constructing an image where the planes have different data types, but this not commonly used and one would in fact need to go to great lengths to construct such an image using CVB functions. In short: Such a case will not be considered in this document.
It is obvious that in situations where the data type can be specified freely (for example when calling CreateGenericImageDT or CreateImageFromPointer), it is easily possible to build nonsensical combinations like 7-bit floating point – the functions will not return an error but do as they’re told. Therefore, common sense should be applied when constructing a CVB data type descriptor.
Beyond the usual 8 bit unsigned pixels (0x00000008) two additional data type descriptors that are being used frequently are 0x0000000A and 0x0000000C which correspond to 10- and 12-bit unsigned integer data. As there are no built-in types with that size in any commonly used programming language it is up to the user to make sure that during operations on images of this type the range of valid values is not exceeded. Internally these formats are always stored in 16 bits per pixel and can be accessed like 16-bit integer data (i.e. through an unsigned short *). Note that the widespread notion of considering RGB a 24 bit per pixel format does not apply to CVB (instead, this would be considered an image with 3 planes at 8 bits per pixel).
If we ignore the reserved bits and the overlay mask bit (which has no correspondence in the OpenCV, IPP or Halcon pixel data types), then we end up using the following data type descriptors for the most commonly used pixel data types:
| Descriptor & 0x3FF | C/C++ data type |
|---|---|
0x00000008 | unsigned char |
0x00000108 | (signed) char |
0x00000010 | unsigned short |
0x00000110 | (signed) short |
0x00000020 | unsigned int |
0x00000120 | (signed) int |
0x00000220,0x00000320 | float |
0x00000240,0x00000340 | double |
CVB treats the pixel data type, the memory layout, and the color/channel interpretation of an image completely separate. Of course, monochrome images are just images with one channel (usually called "plane" in the CVB documentation). RGB images are simply images with three such channels, but so would be images in the HSI or Lab color space. That means that the user is responsible for keeping track of the color format he is currently working on. The Common Vision Blox Display, when confronted with an image with 3 or more planes, will simply assume that plane 0 contains the red pixels, 1 one contains the green pixels and plane 2 contains the blue pixels.
As already mentioned, CVB implements access to the image data through the so-called VPATs, two tables with offsets in x and y direction that can be combined with a base pointer to calculate the address of a pixel in memory. The base pointer and VPAT for an image plane can be queried using the function GetImageVPA:
This might look like an unnecessary overhead when compared to other libraries that often simply work on a base pointer and an increment in y direction to calculate a pixel’s address. But the VPATs not only allow CVB to easily cover practically all memory layouts that a 3rd party library may use – they also allow for neat tricks like merging two separate images into one (see function CreatePanoramicImageMap), rotating an image (see function CreateRotatedImageMap) or extracting a scaled portion of an image (see function CreateImageMap) – all without copying a single pixel.
Defining the lifetime of a large data object (like an image acquired from a camera that may easily contain several megabytes of pixel data) is not as trivial as it may sound, and there is more than just one approach available for this issue. Probably the two most commonly encountered patterns here are alloc/free (i.e., there’s a malloc-like allocation function and free-like cleanup function) and reference counting.
In CVB's C-API we have opted for the implementation of a reference counting model:
(Luckily, when working with CVB.Net, CVBpy or CVB++ these error-prone and daunting tasks are hidden from users by smart pointers.)
This is important to be aware of, because other libraries typically follow different approaches. IPP for example simply provides functions to allocate and free image memory (and it is the caller’s responsibility to make sure both are being called, and at the right times), while e.g., Sapera uses C++ objects and the image memory’s lifetime is tied to the lifetime of the containing Sapera object instance.
OpenCV typically describes the images it is working on using a structure called IplImage which was actually borrowed from the Intel Image Processing Library. The supported pixel data types for the OpenCV library are listed in the description of the depth member of the IplImage structure. The supported pixel data types are:
OpenCV supports images with 1 to 4 channels (nChannels member of IplImage). All channels share the same data type, and the memory layout may in theory be interleaved or planar, with interleaved being the predominant alternative (see description of the dataOrder member of IplImage).
The memory of an OpenCV image is accessible through the imageData pointer in the IplImage. This refers to the C interface of OpenCV. If you are using the C++ interface of OpenCV please to the documentation to map the description to the C++ interface. The data pointer together with the widthStep member (y increment) allow for the calculation of the address of each pixel in the image.
An OpenCV image comes into existence when it is created using one of the creation functions (like e.g., cvCreateImage) and can be destroyed using e.g. cvReleaseImage.
The image processing functions in the IPP (Intel® Integrated Performance Primitives for Intel® Architecture) are a little more basic than those of the OpenCV library. Unlike OpenCV, IPP does not use a structure to collect the information necessary to process image data – memorizing and organizing that data is solely the programmer’s responsibility.
On functions that process image data, IPP supports the following data types:
The complex data type is only used by a very limited set of functions. Like OpenCV, IPP supports images with 1 to 4 channels. In case of 4 channels, one channel may be used for transparency information (alpha channel).
Like OpenCV, IPP supports interleaved and planar data arrangement, with interleaved arrangement again being the predominant memory layout. The image data is being accessed through the base pointer and the vertical increment. Note that due to the fact that IPP contains many functions that have been optimized for Intel’s SSE units, it is necessary that a memory block for IPP to work on is aligned on a 32 byte boundary and that the start of each line of an image lies on a 4 byte boundary.
As IPP offers no structure to contain the image data, there is only a function available to allocate memory blocks that fit the requirements of IPP (ippiMalloc), and a function to free these memory blocks (ippiFree).
Halcon uses pixel types that are somewhat similar to CVB's data type descriptors mentioned earlier. These pixel types map to the C types uint8_t, int8_t, int32_t, and float as well as composed types
| Preprocessor define | Type ID | Meaning |
|---|---|---|
BYTE_IMAGE | (int)1 | 1 byte per pixel (0..255) |
INT4_IMAGE | (int)2 | Type: 4 bytes per pixel (int32_t) |
LONG_IMAGE | (int)2 | 4 byte per pixel (int32_t) |
FLOAT_IMAGE | (int)4 | 4 byte per pixel (float) |
DIR_IMAGE | (int)8 | edge orientation 0..180 |
CYCLIC_IMAGE | (int)16 | 0..255 cyclic |
INT1_IMAGE | (int)32 | -128..127 |
COMPLEX_IMAGE | (int)128 | 2 float images |
INT2_IMAGE | (int)512 | 2 bytes with sign |
UINT2_IMAGE | (int)1024 | Type: 2 bytes (unsigned short) |
VF_IMAGE | (int)2048 | Two float images |
INT8_IMAGE | (int)4096 | 8 byte per pixel (int64_t) |
Gray value images are represented by a rectangular image matrix. Several matrices (called channels) can be combined to a multi-channel image. Each channel is stored separately, i. e. in Halcon the channels are never interleaved in memory. The pixel data is stored as a one-dimensional array (image vector) of one of the pixel types. The indices of the array range from 0 to width*height-1
All of HALCON’s classes, i. e. not only HImage, HRegion, HTuple, HFramegrabber etc., but also the class HObject used when calling operators in the procedural approach, release their allocated resources automatically in their destructor. There is no need to call the operator ClearObj in HALCON/C++. Actually, if you do call it HALCON will at some point complain about already released memory. To explicitly release the resources before the instance gets out of scope, you can call the method .Clear() of the instance.
The data/image transfer from 3rd party libraries to CVB is probably the easier direction – simply because thanks to the VPAT image model and the fairly versatile data type descriptor CVB covers many memory layouts and data types encountered in libraries like OpenCV or IPP. It is also noteworthy that in practically all relevant cases the import can happen without copying the image data – all that is needed is a very thin VPAT wrapper (that can be built by the function CreateImageFromPointer) which takes only a fraction of a millisecond to construct. Probably the only potential obstacle is lifetime management.
As the function CreateImageFromPointer is pivotal for importing 3rd party images into CVB, let’s have a detailed look at it:
Its parameters are:
pImageMem Pointer to the pixel data. When importing from OpenCV the pointer can be directly copied from IplImage::imageData.MemSize pImageMem. This value will only be used for consistency checks and should be at least Width * Height * NumPlanes * sizeof(pixeldatatype). When importing from OpenCV, the value IplImage::imageSize may be used here.Width, Height IplImage::width and IplImage::height. NumPlanes IplImage::nChannels may be used directly as it corresponds to the CVB plane count interpretation. Likewise with Halcon's number of channels indicator. With IPP, it is necessary to interpret the image type as one of the following: 8u, 8s, 16u, 16s, 32s, 32f, 32sc, 32fc. Note that in the unlikely case of dealing with 32sc or 32fc it might be a good idea to double the number of planes and treat e.g., a one-channel complex image as a 2-plane image with plane 0 being real and plane 1 being imaginary part. | IPP Image Type | Number of Planes |
|---|---|
ipp<data_type>C1 | 1 |
ipp<data_type>C2 | 2 |
ipp<data_type>C3 | 3 |
ipp<data_type>C4 | 4 |
ipp<data_type>AC4 | 3 |
ipp<data_type>P3 | 3 |
ipp<data_type>P4 | 4 |
DataType | OpenCV | IPP | Halcon | CVB |
|---|---|---|---|
IPL_DEPTH_8U | ipp8u | BYTE_IMAGE | 0x00000008 |
IPL_DEPTH_8S | ipp8s | INT1_IMAGE | 0x00000108 |
IPL_DEPTH_16U | ipp16u | UINT2_IMAGE | 0x00000010 |
IPL_DEPTH_16S | ipp16s | INT2_IMAGE | 0x00000110 |
IPL_DEPTH_32S | ipp32s | not available | 0x00000120 |
IPL_DEPTH_32F | ipp32f | FLOAT_IMAGE | 0x00000320 |
IPL_DEPTH_64F | not available | not available | 0x00000340 |
PitchX, PitchY, PitchPlanep is the address of a pixel at the location (x, y, i) (with i being the plane index), then p + PitchX gives the address of the pixel at the location (x+1, y, i)p + PitchY gives the address of the pixel at the location (x, y+1, i)p + PitchPlane gives the address of the pixel at the location (x, y, i+1)
To determine the correct values for these pitches, it is necessary to look at the source image’s pixel data type as well as the source image’s memory layout! Luckily with OpenCV and IPP the number of possible permutations here is fairly limited.
Although OpenCV does in principle inherit the possibility to describe planar memory layouts from the IPL, it usually uses interleaved memory layouts only. With interleaved layout the pitch values need to be constructed as follows:
As the OpenCV library provides a lot of convenience functionality that IPP does not offer, we’ll only show a code sample for OpenCV here, but of course the principles apply to other 3rd party libraries as well. For the sake of brevity, error checking as well as using-, include- and linker statements have been omitted. Simply painting the image over the desktop is of course just a quick and dirty approach to displaying the image, but it reduces the amount of code necessary for displaying it to just one line.:
With IPP we need to take into account the possibility of planar memory layout (P3 and P4 variants of the image data types), as well as the special cases of the alpha channel (AC4) and the complex data types (32sc and 32fc):
C1, C2, C3, C4): NumPlanes refers to the value given to CreateImageFromPointer in the NumPlanes function parameter. stepBytes is the value returned by the ippMalloc function in the pStepBytes parameter (needs to memorized for later use) or generally speaking the y increment that is being used in IPP function calls. AC4) Planar formats (P3, P4)
Planar formats are only supported by CreateImageFromPointer if the distance in memory between the different planes is fixed. As there is currently no ippMalloc version for planar images it is the caller’s responsibility to make sure that this condition is met and that the y increments and the plane increments are calculated correctly and take into account the necessary padding bytes.
Complex formats (32sc, 32fc)
To treat IPP’s complex data formats properly one could come up with two valid approaches: One can either double the width and then simply access the image as if it were a regular image (keeping in mind that the even columns hold the real part and the odd columns hold the imaginary part of the pixel values), or one can construct an image with 2 planes and treat plane 0 as the real part and plane 1 as the imaginary part. The latter approach is only possible for complex formats with just one channel at least without building multiple images with CreateImageFromPointer and then concatenating them with CreateConcatenatedImage. But this case is probably so exotic that we should spare the reader the full description of it.
The approach of doubling the width is fairly straightforward and the rules mentioned above the IPP apply (with sizeof(pixel_type) being 4 of course). Just remember to multiply the Width parameter with 2 in that case. Building an image where the real part and the imaginary part are located in different planes requires – besides multiplying the NumPlanes parameter with 2 – a slight modification of the calculation of the increments:
PlaneOrder
This parameter gives the caller the opportunity to easily reorder the numbering of the image planes. As mentioned before, CVB usually interprets plane 0 as the red channel, plane 1 as the green channel and plane 2 as the blue channel where color processing or display is involved. If that fits the data provided by the 3rd party library (typically the case with the IPP for example), then PlaneOrder can simply be set to NULL. But OpenCV by default loads color images in BGR format – in this case the PlaneOrder parameter should be set up like this to create a proper CVB image:
ReleaseCallback, pUserData
The callback and user data pointer provided here can help make sure that the 3rd party image’s lifetime and the CVB image’s lifetime are properly managed. As the CVB image constructed by CreateImageFromPointer shares the source image’s memory, it is absolutely necessary that the source image exists at least as long as the CVB image constructed by CreateImageFromPointer. To make this easier, a callback pointer can be given in the ReleaseCallback parameter, and this callback will be invoked by CVB as soon as the reference counter of the image constructed by CreateImageFromPointer reaches zero. With this callback and the user data pointer it is even possible to transfer ownership (and thereby lifetime management) of the 3rd party image to CVB. For example it is possible to simply pass the IplImage* returned by e.g. cvLoadImage to the pUserData parameter and call cvReleaseImage on it in the release callback (see code example below). ReleaseCallback and/or pUserData may be set to NULL if the caller is not interested in knowing when the CVB image has been destroyed.
Of course it is also possible to decouple the CVB image completely from the source image it has been created from, but this requires copying the image data and is therefore a bit more time consuming. Probably the most convenient way to do it is to call CreateImageFromPointer first (as described above) and then using CreateDuplicateImageEx to create a decoupled copy of the source image.
In Halcon the operator get_image_pointer1 returns a pointer to the first channel of the image. Additionally, the image type (Type = byte, int2, uint2, etc.) and the image size (width and height) are returned. Consequently, a direct access to the image data via the pointer is possible. An image is stored by Halcon linearized in row major order, i.e., line by line. Note that the image types complex and vector_type are handled differently: complex images are interleaved, i.e., the real and the imaginary parts are alternating. In contrast, vector_field images consist of two matrices, one for the rows and one for the columns, which are stored one after the other. The number of image planes can be retrieved in Halcon using the function HImage::CountChannels(). Using both get_image_pointer1 and HImage::CountChannels(), all the required arguments for the CVB function CreateImageFromPointer can be calculated.
Transferring data from CVB images to 3rd party libraries is a lot more difficult than vice versa – especially if the transfer is supposed to be done without copying image data. Ironically it is the VPAT image model that made transferring 3rd party images into CVB so easy in the previous chapter. This makes the other direction potentially more complicated because a certain amount of work needs to be invested to make sure that the memory layout of the CVB image matches the requirements of the destination library.
For performance reasons, it is usually more desirable to interface CVB to another library without copying the image data. However, this is not always possible.
First of all the 3rd party library that is the target for the image data needs to support this approach, i.e., there needs to be some means of creating an image for that library around an already existing memory block (much like the CreateImageFromPointer function in CVB or the combination of cvCreateImageHeader and cvSetData in OpenCV). This is not always a given – a library that concerns itself exclusively or mostly with the acquisition of images is likely to require the buffers it works on to have been allocated by its own memory management. In short: If you want to transfer image data without copying them, the first step should be to make sure that the destination library actually supports this.
Then there is the matter of the CVB image model. Because the VPAT interface of CVB images supports far more memory layouts than a library relying on e.g., a base pointer and a fixed y increment, it is necessary to analyze the memory layout of a CVB image to find out if a data transfer without copying the image data is possible. The function for this is GetLinearAccess (exported by the CVCUtilities.dll). It analyses the VPAT on a given image’s plane and returns the x and y increments as well as the base pointer if the VPAT is organized linearly (i.e. if it uses fixed increments in x and y direction), otherwise the returned increments will be 0. The subsequent checks on the increments and the base pointer that are necessary of course depend on the 3rd party library’s requirements, so it is not possible to give a complete and generally applicable account here but for our sample cases (OpenCV and IPP) it is possible to give a recipe.
First of all, it is recommendable to determine whether or not the image format is in principle compatible with OpenCV. For this to be the case:
If these conditions are met, GetLinearAccess will need to be called on each plane of the image. In principle these conditions apply the logic behind the calculation of the increments for CreateImageFromPointer shown in chapter 3 to the CVB image. The results (base pointers and increments) will need to be harvested for the following comparisons:
IplImage structure does in fact have a channelSeq member, but according to the documentation this member is ignored by OpenCV. Of course, application of this criterion is optional: For those OpenCV functions that do not process color information the channel order is largely irrelevant.If all the required conditions are met, attaching an OpenCV image to the CVB image data is fairly straightforward: First an empty IplImage structure needs to be created as the recipient by calling cvCreateImageHeader, then the data members need to be set using cvSetData (the input for cvSetData can be taken directly from GetLinearAccess.
The only remaining issue to be sorted out now is the object lifetime. The situation is a bit dangerous in that both, the source CVB image and the wrapping OpenCV image rely on the same image data to be available. CVB has no means of yielding ownership of its image memory to another entity, so calling e.g., cvReleaseImage or cvReleaseData will result in an access violation as soon as the CVB image is being accessed after that (even if it is just through ReleaseObject). Vice versa, calling ReleaseObject on the CVB image and accessing the OpenCV image afterward will also result in an access violation. The only safe way to get rid of both, the CVB image and the OpenCV image (without producing a memory leak) is to only call cvReleaseImageHeader on the OpenCV side and (as usual) ReleaseObject on the CVB side. Of course the ReleaseObject call can happen a lot later than cvReleaseImageHeader – the latter will in fact destroy the the IplImage structure so that accidental continued use of the image is no longer possible.
Expressing all this in C/C++ could look e.g., like this:
Use of these functions is fairly straightforward:
In principle the same approach taken for OpenCV can be applied to IPP as well, therefore we will not re-iterate them in source code here again. First verify whether the image is in principle usable with IPP (the analysis is more less the same as for OpenCV, however IPP does not support 64 bit floating point images), analyze the memory layout (keeping in mind that some functions of IPP actually also support a planar layout) and use the information provided by GetLinearAccess to let IPP work on the image.
BGR arrangement is not an issue here, as most functions in IPP actually expect RGB layout, and lifetime management also is a non-issue as IPP uses no image structure of its own.
There is, however, one additional pitfall that did not apply to OpenCV: The IPP tries to use SSE optimized algorithms wherever possible. These do not only require that each line starts on a 4 byte boundary, they also require that the image starts on a 32 bit boundary. Both need to be verified using the data returned by GetLinearAccess, when calling an IPP function. If these conditions are not met, those IPP functions that expected them will return an error code. Two situations where this image data layout is usually given are CVB VIN drivers (as long as RotateImage has not been set) and BMP files.
Lifetime is also a non-issue with IPP. As there is – at least in the C API – no data structure for images (IPP usually works only on pointers and increments that the caller needs to keep inside his own data structures) there is nothing to keep in mind other than that the pointers and increments returned by GetLinearAccess should not be used any more once the image has been released with ReleaseObject.
Unlike in CVB, the Halcon image model is always planar in RGB cases. Therefore, it is only possible to convert monochrome CVB images to Halcon images without copying - for converting color CVB images to Halcon, copying is inevitable. The Halcon operator GenImage1Extern creates an image from a pointer to the pixels with storage management. As it has been explained already the pointer to the pixels can be queried with GetLinearAccess to let Halcon work on the image. The memory for the new image is not newly allocated by HALCON, and thus is not copied either. This means that the memory space that PixelPointer points to must be released by calling ReleaseObject on the CVB image object. This is done by the callback ClearProc provided by the caller. This procedure must have the following signature.
and will be called using __cdecl calling convention when deleting Image. If the memory shall not be released (in the case of frame grabbers or static memory) a procedure “without trunk” or the NULL-Pointer can be passed. Please remember that x and y increment of a CVB image should fulfill the same condition mentioned in OpenCV .
Although copying the image data from a CVB image to any 3rd party data structure will take significantly longer than the approach explained in the previous chapter (even on small images), there are typically two scenarios where that amount of time needs to be invested:
memcpy line by line as opposed to doing the whole image in one block is not much slower but saves a great deal of checks that would be necessary to properly treat padding bytes and, at the same time, automatically takes care of potential discrepancies between top-down or bottom-up arrangement.These two cases can differ significantly in terms of implementation and time required for copying the data. If the data in the CVB image is already arranged in a format that is usable by the target library, copying the data line by line will be very efficient. Otherwise, it may be necessary to copy each pixel individually through VPAT access, which obviously is more time-consuming.
To determine whether or not it becomes necessary to copy the image data through the VPAT, we can revert to the work that has been done in the previous chapter: If it would in principle be possible to transfer the data without copying, then the layout obviously is already ok and VPAT copy will not be necessary. Copying in this case is reduced to a loop over the image’s lines. For VPAT copy, there obviously needs to be two loops (one in vertical and one in horizontal direction). A bit of complexity is added by the fact that the amount of data to be copied for each pixel depends on the image’s data type, but templates go a long way to implementing copy loops that automatically apply to all supported data types. They can also greatly help reduce overhead from looping through the image planes. Note, however, that this benefit can only be reaped in the release build. In practice the code will look something like this:
With the OpenCV image created through cvCreateImage independent of the CVB image, it does not matter in what order they are destroyed. And unlike in the previous chapter, destruction of the OpenCV image must now happen through cvReleaseImage, otherwise a memory leak will occur.
Like in the previous chapter, the copy of CVB image data to IPP buffers is again pretty much covered by what has been said and coded for OpenCV. Again, the major differences are that the IPP does not support double-valued images and is working well with RGB arrangement of color images (therefore the special treatment in the template cvb_to_ocv_copy_vpat for 3-plane images can just be omitted). Due to the special requirements (image start on a 32 byte boundary, line start on a 4 byte boundary), import of CVB images into IPP is probably a little more likely to require copying image data.
As explained earlier, Halcon will always represent RGB images with planar memory layout. In CVB, RGB images usually have interleaved memory layout (as delivered by the hardware), so in most cases it will be necessary to copy the pixel data of RGB images from CVB to Halcon. The Halcon operator GenImageInterleaved does just that: It allocates memory for a three-channel planar image and copies the pixel data from an interleaved source. The following simple example shows how to convert a CVB RGB8 image into a Halcon image:
C/C++ has been used as the programming language for this description – mostly because when looking at any 3rd party imaging library, C++ is the most likely language to be supported. It is also because the pointer support in C/C++ is very straightforward and versatile. Of course, this does not mean that interoperation with 3rd party libraries is limited to C/C++.
The methods explained in this application note should be easily transferable to any language that supports the notion of pointers such as C#. With C# of course there’s the added complexity of managed versus unmanaged domain, but even that is not really a problem – it just requires proper use of the fixed and the unsafe keyword. Keep in mind however, that it is beneficial to keep the managed-unmanaged transitions to the necessary minimum as these transitions tend to consume a fair amount of CPU time.
Visual Basic.Net programmers can in theory also implement anything that has been shown in this application note. However, because the language does not support pointers directly, doing so requires heavy use of the System.Runtime.InteropServices.Marshal class’s methods and is certain to result in code that is much harder to implement and read, and runs a fair bit slower than the corresponding C# implementation. Therefore, our recommendation to VB.Net programmers is to implement their interoperation with 3rd party libraries in C# into a DLL that can then be used from within a VB.Net application.