Common Vision Blox Tool
Sample Image Lists are the format in which information required to build a classifier for Tools like Polimago is stored.
You are currently reading the documentation of the Common Vision Blox Sil.dll, which is part of the Common Vision Blox tool Polimago. The Sil.dll implements the functionality that is required for generating and manipulating a list of labeled samples, from which a classification algorithm like the one implemented in Polimago can then generate a classifier for a machine vision application. Therefore, the Sil.dll is not only the basis for the Polimago module for the TeachBench, it may also be used to write a customer-specific teach program (or an image database for an entirely different purpose).
For those readers who are already familiar with Common Vision Blox Manto: In Polimago, the implementation of the sample image list (which contains the list of samples from which to learn a classifier) has been separated from the implementation of the classification/regression algorithm, because the data base format used by Manto and Polimago is potentially usable for more than just those two tools. Consequently, the Sil.dll does not use license protection of any kind so that it may in the future be shipped with other Common Vision Blox modules as well. Existing Manto SILs can be converted into the new binary format used by the Sil.dll (see the function SilImportFromManto in the documentation of the Sil.dll).
Overview
To reflect the updates to the classification algorithm in Common Vision Blox Polimago, the sample image lists generated using the Sil.dll now support a wider range of data and label types than what has been available previously. A given sample image list (SIL) is now characterized by a label type and a data type:
- Possible label types are:
- Strings (zero-terminated)
- Fixed-length arrays/vectors of floating point values
- Possible data types are:
- Images (defined through width, height, dimension, data type, feature window and offset)
- Fixed-length arrays/vectors of floating point values
These characteristics determine the application a given SIL can be used for:
- SILs with string labels enable the generation of classification classifiers just like before in Manto:
From a SIL that uses string labels a classifier can be generated that, when pointed at a given area of interest inside an image, can tell you what object(s) is/are visible inside that area of interest or where this/those object(s) is/are located.
- SILs with floating point arrays labels, or vector labels, allow for a different mode of operation:
Instead of returning a discrete class number, a vector of floating point values will be returned as the result of using the classifier on a set of data (e.g. an area of interest inside an image). This vector can then be used to interpret the sample that the classifier has been looking at within the context of the other samples to determine e.g. a non-discrete quality index, a direction indication etc. - depending only on the interpretation/meaning of the vector labels that have been assigned to the samples in the SIL. Of course vector labels may use a length of 1 - the formalism remains the same.
Instead of image data, it is now also possible to use vector data for generating classifiers that work on something other than images (arbitrary sensor input), which allows for a whole new range of applications. However, currently the processing of non-image samples is not yet fully implemented inside the Polimago.dll, making the possibility to use non-image samples a theoretical one that will be addressed in the future.
The Sil.dll allows for the combination of either of the supported label types with either of the supported data types, giving the users of the tools complete flexibility over what they want to do with it. Of course this means that the API will have to cover all those combinations in a way that is compliant with the C-call conventions that are used throughout Common Vision Blox (which precludes the use of such convenient things like classes or function overloading). To achieve this, the API has decoupled the items that in the previous implementation of Manto were most of the time implicitly bound together:
- The sample image list (TSIL)
- The sample image list's label type (TSILLABELTYPE)
- The sample image list's data type (TSILDATATYPE)
- The sample image list's label entries (TSILLABEL)
- The sample image list's data entries (TSILDATA)
As a consequence the API may occasionally feel somewhat over the top when looking at the functions involved in binding these five entities together. To reduce this overhead, several convenience functions which are specific to a fixed combination of data type and label type have been added (like e.g. the SilAddXXXItem functions). Still, it is highly recommended to read through the following chapters to completely understand how the SIL.dll works.
Thread Safety
The Sil.dll does not use static variables internally, so all functions can be considered thread-safe in the sense that concurrent calls to different (or the same) function(s) of the Sil.dll will not lead to severe problems like access violations. However, most functions exported by the Sil.dll are not atomic: Calling these functions concurrently is likely to lead to unexpected results/failures (i.e. return of error codes) if e.g. one call removed an entry from the SIL that another, concurrent call tries to read data from. In other words: Operations on a SIL's internal structures may of course alter these structures - which may lead to the invalidation of handles or indices that have been retrieved earlier. ' These invalidations will usually not lead to severe problems like access violations, but they are likely to lead to unexpected results (mostly failures) of the function calls involved. This has to be taken into account when using the SIL.dll in a multi threaded application.
Structure of TSIL Objects
Internally the TSIL object consists of two lists: The label list and the data list.
As the name implies, the
- label list contains all labels (without any duplicate entries) that have been used inside the TSIL object:
The label list can be no longer than the data entry list and in all practical cases is actually significantly shorter. Each entry of the label list contains a list of data objects that this label has been assigned to.
- data list contains all data sets (images or floating point vectors) that have been added to the TSIL object (unlike the label list the data list may contain duplicates).
Each entry of the data list contains a reference to the label that data set has been given (in other words: there will usually be a 1:m relationship between a label and its data):
The lists can therefore effectively be traversed in both directions: It is possible to access entry i of the data list and retrieve the label that has been assigned to that entry or one may access the label list's entry j (i.e. the j-th class in the case of string labels) and retrieve the indices of the data list entries that belong to that class.
In addition to these two lists, the following objects/data are stored inside a TSIL object:
- TSILLABELTYPE indicator
A reference counted object that may be queried for information about the label type a TSIL object is using.
- TSILDATATYPE indicator
A reference counted object that may be queried for information about the data type a TSIL object is using.
- Comment
A zero terminated string with user-definable content.
- Creation Date & Modification Date
The system time at the moment the TSIL was created/last modified. Both are read-only properties of the TSIL.
- Mask Data
A set of data that has the same overall structure as TSIL's data entries (i.e. image data or floating point data of a type that matches the TSIL's data content) that can be convoluted over every sample before entering the learning/classification process. The Mask Data may be absent (which is the default).
Structure of XSIL Files
The SIL.dll uses a different, more easily parseable binary format for the data files it loads and saves than the previous Manto.dll. Therefore the default extension has changed from .sil to .xsil.
The XSIL file uses an approach derived from the Interchange File Format (IFF) (note that in the following sections "string" always means a fixed-size character string unless stated otherwise; if the actual string is shorter than the expected size it is to be filled up with null bytes):
- The file always starts with the 16 characters string "CVBXRIFF "
(that is "CVBXRIFF" plus 8 white spaces, or in bytes (hex): 43 56 42 58 52 49 46 46 20 20 20 20 20 20 20 20), identifying the file as a parseable binary file.
- That is followed by a 16 characters string that identifies the type of the file (empty string if generic CVBXRIFF container) - XSIL files have the string "MantoSil" here.
- Following this comes an arbitrary number of chunks, each of which is prefixed with
- a 8 byte character string identifying the type of content of the chunk that follows
- a 16 byte character string that may provide more insight about the purpose of the chunk for the parsing application
- an 8 byte unsigned integer number giving the size of the data that follows
(which is also the number of bytes that the reader must skip if it does not want to or know how to handle the data that follows)
- the actual data bytes
The following chunk identifiers are in use in XSIL files:
- COMPOSIT - identifies a chunk that consists of a group of other chunks, effectively partitioning the XSIL file into different scopes.
- CVIMG9 - identifies a chunk that contains a deserialized CVB image
(i.e. an array of bytes that has been produced with the Image Manager function ImageToMemory and may be restored with the Image Manager function MemoryToImage).
- TXTUTF8 - identifies a chunk that contains a (not necessarily null terminated!) string using UTF8 encoding.
- SYSTIME - identifies a chunk that contains a SYSTEMTIME structure that has been serialized.
- ARRYFL32 - identifies a chunk that contains an array of 32 bit floating point values.
- ARRYU32 - identifies a chunk that contains an array of 32 bit unsigned integers.
- DATATYPE - specific for Polimago (identification of the data type used in the SIL)
- LBLTYPE - specific for Polimago (identification of the label types used in the SIL)
The typical sequence of chunks in a SIL.dll XSIL file is as follows:
- COMPOSIT "SilData" - contains the list of data entries in the SIL
- sequence of either CVIMG9 chunks or ARRYFL32 chunks (each unnamed)
- COMPOSIT "SilLabels" - contains the list of labels in the SIL
- sequence of either ARRYFL32 or TXTUTF8 chunks (each unnamed)
- ARRYU32 "LabelRefs" - array of an even number of 32 bit ints that defines the mapping between labels and samples
- COMPOSIT "SilInfo2"
- TXTUTF8 - comment string
- SYSTIME "CreationTime" - creation time of the SIL
- SYSTIME "ModificationTime" - modification time of the SIL
- DATATYPE
- LBLTYPE
Differences to Manto
For those users that are familiar with Common Vision Blox Manto, this is a short overview of the features in which the Sil.dll differs from the image list handling functions in Common Vision Blox Manto:
- Sample Image Lists no longer have a list name.
The sample image list name served only display purposes and was mostly redundant to the image list comment (which is still available and accessible in the Sil.dll).
- More than one class may be excluded.
In the previous implementation, the excluded class was identified by its name, and therefore only one class could be excluded from the classification/regression results at any time. With the new Sil.dll, an arbitrary set of classes may be defined as "excluded", however Polimago will ignore the exclusion status as the concept of an explicitly defined excluded class is not present in Polimago. This only applies to classes that use string labels - vector labels cannot be excluded.
- Automatic class sorting
Unlike the previous version of Manto, which issued class indices in the order in which the classes had been added, Polimago will now keep the label lists sorted.
- No empty classes
It is no longer possible to have empty classes inside a SIL. Deleting the last data entry from a class will automatically also delete the class.
- No automatic instance generation settings.
As Polimago no longer supports the automatic instance generation feature of Common Vision Blox Manto, the settings that refer to automatic instance generation are no longer available in the Sil.dll.
- Feature windows may change.
Unlike the previous implementation, the Sil.dll provides functions to change the feature window within the limits defined by the image geometry (feature windows must still be inside the image).
