Memory Layout

When working with image data, different memory layouts are used to store pixel values efficiently, impacting performance and access patterns.

Linear and Contiguous Storage

Linear Layout: Image rows are stored sequentially, but each row may be followed by padding bytes. These layouts can be accessed with linear access.
Contiguous Layout: Image data is stored in a continuous memory block without gaps between rows. This allows for the most efficient memory array access (C++ only).

Interleaved vs. Planar Format

Interleaved Format: Each pixel is stored as a tightly packed set of components (e.g. RGB).
Planar Format: Each color channel is stored separately in memory, meaning all red values are stored first, followed by green, then blue.

Custom Layouts

In some cases, an image layout does not fit into any of these categories. This can (very rarely) happen due to specific hardware constraints. Additionally certain CVB functions e.g. mapping and simultaneously scaling an image with Cvb::Image::Map may result in a custom layout. When this occurs, image pixels can only be accessed using the Virtual Pixel Access Table (VPAT).

Linear Memory Layout

The following examples illustrate different linear and contiguous memory layouts for an RGB image:

Interleaved Contiguous

R G B  R G B  R G B  ...
R G B  R G B  R G B  ...
...

In this layout, pixel values are stored consecutively in memory, with each pixel containing all of its color components (e.g., RGB). Rows are stored directly after each other without padding.

Planar Contiguous

R R R R R R ...
R R R R R R ...
...
G G G G G G ...
G G G G G G ...
...
B B B B B B ...
B B B B B B ...
...

Here, color channels are stored separately. Each channel is stored without any gaps between rows (no padding). No extra bytes are inserted between rows or channels.

Interleaved Linear

R G B  R G B  R G B ... X X X  (X = padding)
R G B  R G B  R G B ... X X X
R G B  R G B  R G B ... X X X
...

This interleaved linear layout is similar to interleaved contiguous, but with padding bytes between rows (denotated as X). The row stride (distance between rows in memory) may be larger than the actual row width to ensure memory alignment.

Planar Linear

R R R  R R R  R R R ... X X X  (X = padding)
R R R  R R R  R R R ... X X X
...
G G G  G G G  G G G ... X X X  (X = padding)
G G G  G G G  G G G ... X X X
...
B B B  B B B  B B B ... X X X  (X = padding)
B B B  B B B  B B B ... X X X
...

In a planar linear layout, similar to the planar contiguous layout, the color channels are stored separately. However, padding bytes are introduced between rows to maintain a specific alignment.

VPAT Memory Layout

The VPAT (Virtual Pixel Access Table) system works well with interlaced images, even in complex cases like when odd and even fields are flipped. It defines a class of memory layouts that can be represented using VPA tables.

Linear representations are directly compatible with VPAT. The essential condition for VPAT compatibility is that the memory layout should allow scan lines (or columns) to be simple translations of each other.

The primary challenge in image processing is the fast computation and modification of the pixel function. For a linear memory representation of an image, this requires the rapid computation of the memory address:

M(X,Y) = M(0,0) + X ⋅ ΔX + Y ⋅ ΔY

In CVB, this is achieved with the help of tables that handle the multiplications and simplify complex issues, such as interlaced scan lines. To understand this, we introduce two functions (tables):

XTable(X) = M(0,0) + X ⋅ ΔX

for X=0,…,Width−1 and

YTable(Y) = Y ⋅ ΔY

for Y=0,…,Height−1.

The address is computed by:

M(X,Y) = XTable(X) + YTable(Y)

This allows for very rapid random access to pixels. The tables also allow for operations like reflecting or rotating the image, shrinking or enlarging it, without copying the underlying data and thus without any significant computation time. These operations require only O(Width+Height) time when programmed through the tables, compared to O(Width×Height) for direct computations on the image.

While the method with VPA tables can handle operations like rotation or scaling efficiently, it is slower than linear access because it requires additional computations to retrieve the table values for each pixel, compared to the direct memory address computation used in linear access.

Pixel Access

After covering the different memory layouts in the previous section, we now focus on how to efficiently read and write pixel values. The following sections introduce various pixel access methods for C++, .NET, and Python. Depending on the programming language and memory layout, different access methods offer varying levels of convenience, performance, and flexibility.

To learn more about pixel access in your preferred programming language, simply click on the corresponding buttons below.

C++

.NET

Python

Table of Contents

Memory Layout

Linear Memory Layout

VPAT Memory Layout

Pixel Access