frame_buffer

[Chinese]

1. Overview

The Frame Buffer module provides video frame buffer memory management functionality, primarily used for display and encoding scenarios. This module has the following core features:

1. Dual Heap Memory Management: Supports memory allocation from PSRAM’s DISPLAY heap and ENCODE heap respectively to meet different application requirements

2. Secure Memory Allocation: Uses 32-byte memory alignment to ensure efficient and compatible memory access

3. CRC Checksum Protection: Generates CRC-8 checksum for each allocated buffer to prevent duplicate freeing and illegal access

4. Memory Initialization: Newly allocated memory is initialized with a specific pattern (0xFB) for debugging and troubleshooting

5. Error Handling: Includes safety mechanisms such as null pointer checks and memory allocation failure handling

• For usage, you can refer to the DVP example project to see how to use frame buffer related API interfaces to implement image memory buffer management:

  • DVP CAMERA Sample Project

2. API functions

• frame_buffer_display_malloc() - Allocate display frame buffer from DISPLAY heap

• frame_buffer_display_free() - Free display frame buffer

• frame_buffer_encode_malloc() - Allocate encoding frame buffer from ENCODE heap

• frame_buffer_encode_free() - Free encoding frame buffer

Suitable for multimedia applications requiring efficient and secure memory management, such as video display systems and video encoding pipelines.

Note

This module depends on the PSRAM module, ensure it is initialized before use. PSRAM initialization consists of two parts: one is the initialization of the entire PSRAM hardware module, the other is the initialization of PSRAM’s DISPLAY heap and ENCODE heap. The initialization of DISPLAY heap and ENCODE heap is based on PSRAM initialization, ensure they are initialized before use. PSRAM module initialization is based on a voting mechanism, as modules using PSRAM are not limited to multimedia and can also use it as Heap. Voting interface reference: bk_pm_module_vote_psram_ctrl. The initialization of DISPLAY heap and ENCODE heap is implemented by calling pre-registered callback functions after PSRAM power-on initialization. Specific reference: bk_psram_frame_buffer_init.

3. Related Macro Definitions

The following macros define PSRAM memory region addresses and size configurations, corresponding to memory region definitions in the linker script:

• PSRAM_ADDR - PSRAM memory base address (0x60000000)

• PSRAM_MEM_SLAB_USER_SIZE - User heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_USER_SIZE)

• PSRAM_MEM_SLAB_AUDIO_SIZE - Audio heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_AUDIO_SIZE)

• PSRAM_MEM_SLAB_ENCODE_SIZE - Encoding heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_ENCODE_SIZE)

• PSRAM_MEM_SLAB_DISPLAY_SIZE - Display heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_DISPLAY_SIZE)

• psram_mem_slab - PSRAM memory mapping structure pointer for accessing various memory regions

Corresponding linker script memory region definitions:

• PSRAM_SLAB_USER - User heap memory region (Address: CONFIG_PSRAM_MEM_SLAB_USER_ADDR, Size: CONFIG_PSRAM_MEM_SLAB_USER_SIZE)

• PSRAM_SLAB_AUDIO - Audio heap memory region (Address: CONFIG_PSRAM_MEM_SLAB_AUDIO_ADDR, Size: CONFIG_PSRAM_MEM_SLAB_AUDIO_SIZE)

• PSRAM_SLAB_ENCODE - Encoding heap memory region (Address: CONFIG_PSRAM_MEM_SLAB_ENCODE_ADDR, Size: CONFIG_PSRAM_MEM_SLAB_ENCODE_SIZE)

• PSRAM_SLAB_DISPLAY - Display heap memory region (Address: CONFIG_PSRAM_MEM_SLAB_DISPLAY_ADDR, Size: CONFIG_PSRAM_MEM_SLAB_DISPLAY_SIZE)

These macros and memory region definitions together form the PSRAM memory management framework for managing memory allocation in different functional regions.

4. Heap Size Definitions

The following macros define PSRAM memory region size configurations, corresponding to memory region definitions in the linker script:

• PSRAM_MEM_SLAB_USER_SIZE - User heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_USER_SIZE)

• PSRAM_MEM_SLAB_AUDIO_SIZE - Audio heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_AUDIO_SIZE)

• PSRAM_MEM_SLAB_ENCODE_SIZE - Encoding heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_ENCODE_SIZE)

• PSRAM_MEM_SLAB_DISPLAY_SIZE - Display heap memory size configuration (CONFIG_PSRAM_MEM_SLAB_DISPLAY_SIZE)

These macros define PSRAM memory region sizes for configuring memory allocation and management.

Note

Heap size definition: To modify different heap sizes, simply modify the configuration file in the corresponding project. Reference app project: ./projects/app/partitions/bk7258/ram_regions.csv. Where PSRAM_MEM_SLAB_USER_SIZE, PSRAM_MEM_SLAB_AUDIO_SIZE, PSRAM_MEM_SLAB_ENCODE_SIZE, PSRAM_MEM_SLAB_DISPLAY_SIZE correspond to the sizes of memory regions, in bytes.

5. Related Data Structures

frame_buffer_t Structure

frame_buffer_t is the core data structure in frame buffer management, used to store video frame metadata and actual frame data:

typedef struct frame_buffer_t
{
    uint32_t flag;          // Flag bits
    uint8_t *frame;         // Frame data pointer
    uint32_t size;          // Frame data size
    uint8_t frame_crc;      // Frame data CRC checksum
    camera_type_t type;     // Camera type
    pixel_format_t fmt;     // Pixel format
    uint8_t crc;            // Structure CRC checksum
    uint32_t timestamp;     // Timestamp
    uint16_t width;         // Frame width
    uint16_t height;        // Frame height
    uint32_t length;        // Frame length
    uint32_t sequence;      // Sequence number
    uint32_t h264_type;     // H264 encoding type
};

Member Description:

• flag: Status flag bits, used to identify various states of the frame buffer

• frame: Pointer to actual frame data, storing video frame pixel data

• size: Total size of frame data (in bytes)

• frame_crc: CRC-8 checksum of frame data for data integrity verification

• type: Camera type (camera_type_t enumeration)

• fmt: Pixel format (pixel_format_t enumeration), supports multiple formats such as: - PIXEL_FMT_YUV420P, PIXEL_FMT_NV12, PIXEL_FMT_NV21 - PIXEL_FMT_RGB565, PIXEL_FMT_RGB888, PIXEL_FMT_ARGB8888 - PIXEL_FMT_GRAY, PIXEL_FMT_RAW, PIXEL_FMT_PNG

• crc: CRC checksum of the structure itself to prevent duplicate freeing

• timestamp: Timestamp, recording frame capture or processing time

• width: Frame width (in pixels)

• height: Frame height (in pixels)

• length: Actual data length of the frame

• sequence: Sequence number for frame order tracking

• h264_type: H264 encoding specific type information

This structure ensures memory safety through CRC checksum mechanism, preventing duplicate freeing and illegal access.

The following structures define PSRAM memory region structures for accessing various memory regions:

• psram_mem_slab_t - PSRAM memory mapping structure, containing memory mapping information for user heap, audio heap, encoding heap, and display heap

These structures define the layout and access methods of PSRAM memory regions for managing memory allocation in different functional regions.

6. Related Function Interfaces

6.1 Display Buffer Management Functions

frame_buffer_display_malloc

frame_buffer_t *frame_buffer_display_malloc(uint32_t size);

Function: Allocate display frame buffer from PSRAM’s DISPLAY heap

Parameters: - size: Requested frame data size (in bytes)

Return Value: - Success: Pointer to allocated frame_buffer_t structure - Failure: NULL

Implementation Details: 1. Allocate memory from PSRAM_HEAP_YUV heap, size is size + sizeof(frame_buffer_t) + 32 (32-byte alignment reserved) 2. Use os_memset to zero the frame_buffer_t structure 3. Calculate 32-byte aligned frame data pointer address 4. Set flag to FB_ALLOCATED_PATTERN (0x8338) 5. Calculate CRC-8 checksum of first 6 bytes and store in frame_crc 6. Return allocated frame buffer pointer

frame_buffer_display_free

void frame_buffer_display_free(frame_buffer_t *frame);

Function: Free display frame buffer

Parameters: - frame: Frame buffer pointer to free

Implementation Details: 1. Null pointer check, record error log if NULL 2. Check if flag is FB_ALLOCATED_PATTERN 3. Recalculate CRC checksum and compare with stored frame_crc 4. If verification passes, set flag to FB_FREE_PATTERN (0xF00F) and free memory 5. If verification fails, record duplicate free error log

6.2 Encoding Buffer Management Functions

frame_buffer_encode_malloc

frame_buffer_t *frame_buffer_encode_malloc(uint32_t size);

Function: Allocate encoding frame buffer from PSRAM’s ENCODE heap

Parameters: - size: Requested frame data size (in bytes)

Return Value: - Success: Pointer to allocated frame_buffer_t structure - Failure: NULL

Implementation Details: 1. Allocate memory from PSRAM_HEAP_ENCODE heap, size calculation same as display buffer 2. Memory initialization and alignment processing same as display buffer 3. Same flag setting and CRC checksum mechanism

frame_buffer_encode_free

void frame_buffer_encode_free(frame_buffer_t *frame);

Function: Free encoding frame buffer

Parameters: - frame: Frame buffer pointer to free

Implementation Details: 1. Free logic identical to display buffer 2. Use same safety checks and CRC verification mechanism

6.3 Key Features

1. Memory Alignment: All frame data pointers are 32-byte aligned (ALIGN_BITS=5)

2. Secure Allocation: Use specific allocation pattern (FB_ALLOCATED_PATTERN) to identify valid buffers

3. Duplicate Free Prevention: Prevent memory duplicate freeing through CRC-8 checksum

4. Error Handling: Detailed error logging for debugging issues

5. Memory Initialization: Newly allocated memory is zero-initialized

6.4 Usage Example

// Allocate display buffer
frame_buffer_t *display_frame = frame_buffer_display_malloc(1920 * 1080 * 2);
if (display_frame) {
    // Use frame buffer
    // ...
    // Free buffer
    frame_buffer_display_free(display_frame);
}

// Allocate encoding buffer
frame_buffer_t *encode_frame = frame_buffer_encode_malloc(1280 * 720 * 1.5);
if (encode_frame) {
    // Use frame buffer
    // ...
    // Free buffer
    frame_buffer_encode_free(encode_frame);
}