DVP

[Chinese]

1. Overview

This section mainly describes the usage and adaptation methods of DVP, as well as related macro definitions.

2. Supported Features

  1. Supports maximum resolution 1280X720 input (output);

  2. Only supports Sensor output standard YUV422 data format (big-endian mode): YUYV/UYVY/YYUV/UVYY;

  3. Default VSYNC/HSYNC input is high active, and supports inversion

  4. Supports direct output of YUV422 data to BUFF;

  5. Supports direct output of JPEG encoded data, or YUV422 & JPEG data separately to BUFF;

  6. Supports direct output of H264 encoded data, or YUV422 & H264 data separately to BUFF;

  7. Supports dynamic adjustment of encoding bit rate (JPEG or H264);

  8. Supports regenerating IDR frames at any time for H264 encoding

  9. Supports Sensor data reception error detection

Users need to enable the corresponding functions by configuring parameters according to their application scenarios.

4. Adapting New Camera

1.To adapt a new DVP camera, you need to add the corresponding DVP driver file in the peripheral files, which mainly includes the configuration table for I2C to configure sensor registers. Different resolutions require different tables. The specific sensor structure is as follows.

2.The way to output the specified resolution and frame rate of the sensor is to set the sensor’s registers through the configuration table. The scheme is to configure three tables to achieve, which are: - Initialization table: used to initialize the sensor’s registers. - Frame rate table: used to set the sensor’s frame rate. - Resolution table: used to set the sensor’s resolution.

The configuration of the related table content needs to be provided by the sensor manufacturer.

3.After configuring the relevant parameters according to the above instructions, you need to register the sensor detection function to the system, which can be referred to the implementation method in ./projects/dvp_example/ap/dvp_test/src/dvp_gc0001_test.c. Register the following structure content.

const dvp_sensor_config_t *gc0001_detect_sensor(void)
{
    LOGI("%s\n", __func__);
    if (gc0001_detect())
    {
        return &dvp_sensor_gc0001;
    }

    return NULL;
}

BK_CAMERA_SENSOR_DETECT_SECTION(gc0001_detect_sensor);

4.After completing the above steps, you need to add the sensor configuration file to the compilation system, which can be referred to the implementation method in ./projects/dvp_example/ap/CMakeLists.txt.

set(GLOBAL_FUNCTION_SYMBOLS "")

list(APPEND srcs
    # other srcs
    dvp_test/src/dvp_gc0001_test.c
)

list(APPEND GLOBAL_FUNCTION_SYMBOLS "gc0001_detect_sensor")

foreach(symbol ${GLOBAL_FUNCTION_SYMBOLS})
    target_link_libraries(${COMPONENT_LIB} INTERFACE "-u ${symbol}")
endforeach()

5.After completing the above steps, the added sensor detection method will be registered in the segment after compilation. When the user starts, the sensor will be automatically detected. If the sensor is detected, the sensor’s detection function will be called.

6.The sensor detection method segment will be registered in the ld file by default, but some projects may have their own ld files, and do not use the ld file of the SDK by default, then you need to add the following content to the ld file:

.sensor_detect :
{
__camera_sensor_detect_array_start = .;
KEEP(*(.camera_sensor_detect_function_list.*))
__camera_sensor_detect_array_end = .;
} > FLASH

7.The specific sensor configuration structure is as follows:

/**
* @brief DVP sensor configuration structure
* @details Configuration parameters and function pointers for DVP sensor
*/
typedef struct
{
    char *name;                     /**< sensor name */
    media_ppi_t def_ppi;            /**< sensor default resolution */
    frame_fps_t def_fps;            /**< sensor default frame rate */
    mclk_freq_t  clk;               /**< sensor working clock when configuring frame rate and resolution */
    pixel_format_t fmt;             /**< sensor input data format */
    sync_level_t vsync;             /**< sensor vertical sync signal active level */
    sync_level_t hsync;             /**< sensor horizontal sync signal active level */
    uint16_t id;                    /**< sensor type (sensor_id_t) */
    uint16_t address;               /**< sensor register address written via I2C */
    uint16_t fps_cap;               /**< frame rates supported by sensor */
    uint16_t ppi_cap;               /**< resolutions supported by sensor */

    /**
    * @brief Auto-detect the used DVP sensor
    * @return Returns true if sensor is detected, false otherwise
    */
    bool (*detect)(void);

    /**
    * @brief Initialize DVP sensor
    * @return Returns 0 on success, error code on failure
    */
    int (*init)(void);

    /**
    * @brief Set sensor resolution
    * @param ppi Resolution to set
    * @return Returns 0 on success, error code on failure
    */
    int (*set_ppi)(media_ppi_t ppi);

    /**
    * @brief Set sensor frame rate
    * @param fps Frame rate to set
    * @return Returns 0 on success, error code on failure
    */
    int (*set_fps)(frame_fps_t fps);

    /**
    * @brief Power down or reset sensor
    * @return Returns 0 on success, error code on failure
    */
    int (*power_down)(void);

    /**
    * @brief Read sensor register
    * @param reg Register address
    * @param val Pointer to store register value
    * @return Returns 0 on success, error code on failure
    */
    int (*read_register)(uint32_t reg, uint32_t *val);

    /**
    * @brief Write to sensor register
    * @param reg Register address
    * @param val Register value to write
    * @return Returns 0 on success, error code on failure
    */
    int (*write_register)(uint32_t reg, uint32_t val);
} dvp_sensor_config_t;
Description of some parameters:

  • clk: Camera input clock, default is 24MHz, need to be configured according to camera specification;

  • fmt: The format of data output from camera to chip, currently only supports YUV420, the order needs to be synchronized with camera output order, default is YUYV;

  • vsync: Camera output vsync active level, some cameras output valid data when vsync is low, need to be synchronized with camera vsync output level, default is high level active;

  • hsync: Camera output vsync active level, some cameras output valid data when vsync is low, need to be synchronized with camera hsync output level, default is high level active;

  • address: I2C slave address for configuring camera registers, need to be configured according to camera specification;

  • fps_cap: The frame rate table supported by camera, needs to configure corresponding registers to achieve;

  • ppi_cap: The resolution table supported by camera, needs to configure corresponding registers to achieve;

5. DVP Camera Usage Flow

5.1 DVP Camera Power on

Before using the DVP sensor, you need to power it on. For the BK7258 platform, you need to configure CONFIG_DVP_CTRL_POWER_GPIO_ID as the VDD pin of the DVP sensor, the default control pin is GPIO_28, and you need to set it to output mode and set its level to high.

/**
* @brief DVP camera power on
*/
{
    #define GPIO_INVALID_ID           (0xFF)
    #ifdef CONFIG_DVP_CTRL_POWER_GPIO_ID
    #define DVP_POWER_GPIO_ID CONFIG_DVP_CTRL_POWER_GPIO_ID
    #else
    #define DVP_POWER_GPIO_ID GPIO_INVALID_ID
    #endif

    if (DVP_POWER_GPIO_ID != GPIO_INVALID_ID)
    {
        GPIO_UP(DVP_POWER_GPIO_ID);
    }
}

5.2 Create DVP Camera Controller

Call the following interface to create a DVP camera controller:

avdk_err_t bk_camera_dvp_ctlr_new(bk_camera_ctlr_handle_t *handle, bk_dvp_ctlr_config_t *config);

Description: - Creates DVP camera controller and returns the controller handle. - The input configuration contains two parts:

Part 1: bk_dvp_config_t, including DVP camera IO mapping, I2C configuration, resolution and frame rate configuration to be output, image format configuration to be output, etc. Part 2: bk_dvp_callback_t, through this callback function to apply for space to store target data, and return image data to the user, the user manages the target data by themselves.

Note

  1. In the DVP callback function, no blocking operations or long operations should be performed, otherwise the hardware interrupt will be delayed, leading to abnormal image data. In the following section of creating DVP camera controller, a specific example will be given, in which the callback functions only perform simple frame buffer application and release operations, users can refer to the implementation.

  2. If working in encoding mode, the memory will be applied according to the resolution size by default, and configured in open, and released in delete. If you do not want to repeat the application and release of this memory, it is recommended not to delete the controller.

Parameters: - handle: DVP camera controller handle. - config: DVP camera controller configuration parameters.

Related data structures: bk_dvp_ctlr_config_t, bk_dvp_config_t, bk_dvp_callback_t. Reference path: ap/include/components/dvp_camera_types.h. bk_dvp_config_t supports the following default configuration:

/**
* @brief DVP camera configuration for 640x480 resolution at 30 FPS with MJPEG format
* @details Predefined configuration macro for common DVP camera settings
*/

/* Explanation of parameters in macro definition:
 * i2c_config.id - I2C controller ID
 * i2c_config.scl_pin - I2C clock pin
 * i2c_config.sda_pin - I2C data pin
 * i2c_config.baud_rate - I2C baud rate (100kHz)
 * i2c_config.flags - I2C flags (no special configuration)
 * reset_pin - Reset pin (0xFF means not used)
 * pwdn_pin - Power down pin (0xFF means not used)
 * io_config.data_width - Data bit width (8-bit)
 * io_config.data_pin - Data pin array
 * io_config.vsync_pin - Vertical sync pin
 * io_config.hsync_pin - Horizontal sync pin
 * io_config.xclk_pin - External clock pin
 * io_config.pclk_pin - Pixel clock pin
 * clk_source - Clock source (24MHz)
 * width - Image width (pixels)
 * height - Image height (pixels)
 * fps - Frame rate (frames/second)
 * img_format - Image format (MJPEG)
 * user_data - User data pointer (none)
 */
#define BK_DVP_864X480_30FPS_MJPEG_CONFIG()\
{\
    .i2c_config = {\
        .id = 1,\
        .scl_pin = 0,\
        .sda_pin = 1,\
        .baud_rate = 100000,\
        .flags = 0,\
    },\
    .reset_pin = 0xFF,\
    .pwdn_pin = 0xFF,\
    .io_config = {\
        .data_width = SENSOR_BITS_WIDTH_8BIT,\
        .data_pin = {32, 33, 34, 35, 36, 37, 38, 39},\
        .vsync_pin = 31,\
        .hsync_pin = 30,\
        .xclk_pin = 27,\
        .pclk_pin = 29,\
    },\
    .clk_source = MCLK_24M,\
    .width = 640,\
    .height = 480,\
    .fps = 30,\
    .img_format = IMAGE_MJPEG,\
    .user_data = NULL,\
}

// It is recommended that users select idle nodes from their own maintained linked lists according to the image format for the requested frames.
// Maintain two linked lists with the complete operation below: one free list and one ready list.
// Take a node from the free list each time when applying, add the node to the ready list after application is complete,
// take from the ready list when needed, and directly release to the free list after use.
// For details, you can refer to: ``./projects/dvp_example/dvp_test/src/dvp_frame_list.c``
/**
 * @brief Allocate different types of frame buffers based on image format and initialize frame structure members
 *
 * @param format Image format (YUV or other formats like MJPEG/H264, etc.)
 * @param size Buffer size to allocate (bytes)
 * @return Returns frame buffer pointer on success, NULL on failure
 */
static frame_buffer_t *dvp_frame_malloc(image_format_t format, uint32_t size)
{
    // from dvp frame free queue malloc format type frame buffer
    frame_buffer_t *frame = dvp_frame_queue_malloc(format, size);

    if (frame)
    {
        frame->sequence = 0;
        frame->length = 0;
        frame->timestamp = 0;
        frame->size = size;
    }

    return frame;
}

/**
 * @brief Notify DVP camera that data processing is complete and release frame buffer
 *
 * @param format Image format (YUV or other formats like MJPEG/H264, etc.)
 * @param frame Pointer to frame buffer containing frame data to be released
 * @param result Processing result, 0 indicates success, non-zero indicates failure. Can be stored in user-maintained linked list when successful; recommend direct release when failed
 */
static void dvp_frame_complete(image_format_t format, frame_buffer_t *frame, int result)
{
    if (frame->sequence % 30 == 0)
    {
        LOGD("%s: seq:%d, length:%d, format:%s, h264_type:%x, result:%d\n", __func__, frame->sequence,
            frame->length, (format == IMAGE_MJPEG) ? "mjpeg" : (format == IMAGE_H264) ? "h264" : "YUV", frame->h264_type, result);
    }

    if (result != AVDK_ERR_OK)
    {
        // error frame, free frame buffer to free queue
        dvp_frame_queue_free(format, frame);
    }
    else
    {
        // frame complete, add to ready queue, wait for user use
        dvp_frame_queue_complete(format, frame);
    }
}

static const bk_dvp_callback_t dvp_cbs = {
    .malloc = dvp_frame_malloc,
    .complete = dvp_frame_complete,
};

// create dvp controller
{
    bk_dvp_ctlr_config_t dvp_ctrl_config = {
        .config = BK_DVP_864X480_30FPS_MJPEG_CONFIG(),
        .cbs = &dvp_cbs,
    };

    // you can change the config here
    //dvp_ctrl_config.config.img_format = output_format;
    //dvp_ctrl_config.config.width = ppi >> 16;
    //dvp_ctrl_config.config.height = ppi & 0xFFFF;
    // new
    ret = bk_camera_dvp_ctlr_new(&s_dvp_handle, &dvp_ctrl_config);
}

5.3 Open DVP Camera

Call the following interface to open the DVP camera:

avdk_err_t bk_camera_open(bk_camera_ctlr_handle_t handle);
Description:

Opens DVP camera controller and returns error code. After opening the DVP camera, the registered callback function will continuously apply for data space for storage. After opening the DVP camera, it will start collecting data and call the registered callback function to return the collected data to the user. The user needs to pay attention to whether the frame is valid and choose to use or release it.

Parameters:

handle: DVP camera controller handle.

5.4 DVP Ioctl

Call the following interface to perform DVP IOCTL operations:

avdk_err_t bk_camera_ioctl(bk_camera_ctlr_handle_t handle, uint32_t cmd, void *arg)
Description:

DVP IOCTL operations, supported commands are: - DVP_IOCTL_CMD_H264_IDR_RESET: Regenerate IDR frame, must ensure that the working mode of DVP is H264_MODE or H264_YUV_MODE mode, for it to take effect. - DVP_IOCTL_CMD_SENSOR_WRITE_REGISTER: Write sensor register. - DVP_IOCTL_CMD_SENSOR_READ_REGISTER: Read sensor register.

Parameters:

handle: DVP camera controller handle. cmd: Command code, supported commands shown above. arg: Parameter, user-provided parameters, different for different commands.

5.5 Close DVP Camera

Call the following interface to close the DVP camera:

avdk_err_t bk_camera_close(bk_camera_ctlr_handle_t handle);
Description:

Closes DVP camera controller. After closing the DVP camera, the registered callback function will stop applying for data space and also stop collecting data.

Parameters:

handle: DVP camera controller handle.

5.6 Deregister DVP Camera Controller

Call the following interface to deregister the DVP camera controller:

avdk_err_t bk_camera_delete(bk_camera_ctlr_handle_t handle);
Description:

Deregisters DVP camera controller. Releases the memory space applied by the controller. It can only be called when dvp is successfully closed.

Parameters:

handle: DVP camera controller handle.

5.7 DVP Power Off

Pull down CONFIG_DVP_CTRL_POWER_GPIO_ID.

{
    // power off dvp camera
    if (DVP_POWER_GPIO_ID != GPIO_INVALID_ID)
    {
        GPIO_DOWN(DVP_POWER_GPIO_ID);
    }
}
Parameters:

handle: DVP camera controller handle.

6. DVP Support Scheme

Because DVP supports outputting various formats of data, users need to choose the appropriate output format according to their own needs.

  • DVP directly outputs YUV images, and the display module is used for display.

  • DVP directly outputs MJPEG images, and the WiFi module is used for image transmission.

  • DVP directly outputs H264 images, and the WiFi module is used for image transmission.

  • DVP directly outputs MJPEG&YUV images, and the WiFi module is used for image transmission, and the display module is used for display.

  • DVP directly outputs H264&YUV images, and the WiFi module is used for image transmission, and the display module is used for display.