The path of demos: ./components/media/cli/media_cli.c``, ./components/media/camera/camera_act.c, ./components/media/transfer/transfer_act.c, ./components/media/lcd/lcd_act.c
The following process takes the board as a softap as an example to introduce the use of apk, after the mobile phone is connected to the ap enabled by the board:
Figure 1 is a schematic diagram of the app
Figure 2 is the main interface of the app
Among them, choose 1 to see Figure 3.
1: Settings menu;
2: Update apk and rollback apk menu;
Figure 3 is the setting interface menu
Among them:
1: For the setting interface as shown in Figure 4;
2: button to return to the main interface;
Figure 4 is the real setting interface
Among them, the setting instructions are as follows, after the setting is completed, return to the main interface of Figure 2
1: Set the solution, currently support video_transfer and doorbell, select doorbell here;
2: Set the data transmission mode, currently supports UDP and TCP, and UDP is selected by default;
3: Set the ip address of the peer, the default is 192.168.0.1 in ap mode, and it is not required to be modified, and in sta mode, it is set to the ip address of the peer;
4: Set the camera type, currently supports DVP and UVC, set according to the type of camera you use;
5: Set the output resolution of the camera;
6: Set the resolution of the LCD output, according to the LCD screen resolution you use;
Figure 5 is the function enable setting interface
Among them, the function enable description is as follows:
1: Switch video image transmission;
2: switch voice;
3: Photo switch, currently not supported;
4: Switch LCD screen display;
Note
Set the peer IP address in Figure 4. When the board is softap, the default is 192.168.0.1. When the board is used as a staion, the mobile phone and the board are connected to the same ap,
and the filled IP address can be passed through the command `` ip`` to get.
In addition, the app also supports the function of mobile phone image transfer, that is, the command video_transfer-a|sssidkey, but step 1 in Figure 4 must be set to video_transfer mode.
The apk download address: http://dl.bekencorp.com/apk/RealtimeVideo.apk
The doorbell project can add new screens according to the needs of users to meet different product needs. For the new driver screen driver code,
please refer to ./middleware/driver/lcd Add lcd_xxx.c under the path, such as lcd_st7282.c;
The main configuration process of the newly added driver code is as follows:
For RGB screen, hsync/vsync proch needs to be modified according to lc
The screen resolution is undefined and needs to be defined in ./include/driver/media_types.h.
The newly added screen ID needs to be defined in ./include/driver/lcd_types.h, as follows:
A screen similar to lcd_gc9503v needs to be initialized, and the initialization function needs to be registered in its structure:
.init=lcd_gc9503v_init,
The lcd_gc9503v_init function is generally provided by the screen manufacturer. It needs the hardware interface to simulate the SPI or I2C interface,
so it is necessary to initialize the GPIO and adapt the corresponding SPI or I2C protocol according to the initialization command.
Currently, the SPI3-wire and 4-wire protocols have been adapted in the SDK.
So far, the adaptation of the screen driver has been completed.
The next simple adaptation is to ensure that the user enters a command or passes a parameter (screen resolution or name), and the corresponding device can be found in the SDK:
Obtain the judgment of the LCD device name, and add the judgment of the new screen name in media_cli.c
please refer to html/bk7256/zh_CN/latest/examples/video/dma2d.html It introduces in detail the supported types of the data format of the foreground icon. This document explains the common data of ARGB8888 and RGB565.
Prepare png or jpg images (also known as foreground images) of small icons that need to be fused
Convert png to rgba8888 data, this conversion can maintain the transparency of png, suitable for blending icons whose background is transparent
Toolpath: components/media/tools/ffmpeg_bat/png2argb Access all png images under this path, double-click run.bat.
Or convert jpg to rgb565 (big endian) data. During the conversion process, all opaque jpg pixels will be converted to opaque by default, which is suitable for blending icons with similar backgrounds and icon backgrounds, or blending that does not require background frames.
Toolpath: components/media/tools/ffmpeg_bat/jpeg2rgb565 Access all png images under this path, double-click run.bat.
If you want to fuse the foreground icon at a fixed position on the LCD screen, you also need to set the coordinates of the foreground fusion
For example: if you want to merge the foreground icon (16x36) of the version number from the 500th line of the screen (480x800), the center position can be set as follows:
After preparing the picture data and the blending position, the next step is to configure the actual blending parameters, please refer to the API interface in lcd_act.c: lcd_blend_handler
For example, the wifi icon to be blended starts to blend at the upper right corner of the screen, that is, the screen coordinates (lcd_width-logo_width, 0), and its lcd_blend.pbg_addr needs to set the address offset (frame->frame + (frame->width - WIFI_LOGO_W) * 2.
How to set the offset is described in detail in the DMA2D use case guide.
The cameras used in the application process are not only those currently supported, but also need to be adapted to other dvp cameras or uvc cameras.
The following is a separate description of how to adapt to the two different types of cameras.
Adaptation of dvp camera
The dvp camera configures the output of the camera through I2C communication, mainly to configure the value of the sensor register to achieve the expected image effect (resolution, frame rate, etc.)
Refer to the driver code: middleware/driver/camera/dvp_gc0328c.c, first you need to adapt the parameter structure of the dvp camera: dvp_sensor_config_t;
typedef struct
{
char *name; /**< camera name */
media_ppi_t def_ppi; /**< The default resolution of the camera, generally used resolution */
sensor_fps_t def_fps; /**< The camera's default frame rate, usually the commonly used frame rate*/
uint16 id; /**< camera type (enumeration value, you need to add it yourself), refer to the enumeration type sensor_id_t */
uint8 clk; /**< The input MCLK specified by the camera protocol, and this MCLK is separated from the CLK of the JPEG module and needs to be configured by yourself */
/**@example
* JPEG_96M_MCLK_24M: Indicates that the camera protocol stipulates that the MCLK input is 24MHz,
and the clock of the JPEG module is 96MHz at this time, and 96MHz can be divided by four to get 24MHz
* It should also be noted that the working clock of JPEG is divided in CLK (480MHz and 320MHz), the frequency division coefficient range F=[0, 15],
the frequency division calculation formula JPEG_CLK=CLK/(1+F);
* JPEG only supports frequency division: 0:4 frequency division, 1:6 frequency division, 2:2 frequency division, 3:3 frequency division
**/
uint16 address; /**< The address of the camera through the I2C configuration register, generally the datasheet will tell */
uint16 fps_cap; /**< The camera supports the configured frame rate, and outputs different frame rates according to requirements */
uint16 ppi_cap; /**< The camera supports the configured resolution, and outputs different resolutions according to different scenarios */
bool (*detect)(const dvp_camera_i2c_callback_t *cb); /**< The camera function is automatically detected, which is to read whether the camera ID (such as CHIP_ID) is consistent with the current camera*/
int (*init)(const dvp_camera_i2c_callback_t *cb); /**< Configure the camera initialization register table, other adjustments (such as: resolution, frame rate, white balance, etc.) must be based on this */
int (*set_ppi)(const dvp_camera_i2c_callback_t *cb, media_ppi_t ppi); /**< Set the camera resolution register table, generally support different resolution output */
int (*set_fps)(const dvp_camera_i2c_callback_t *cb, sensor_fps_t fps); /**< The register for setting the camera frame rate, generally supports the output of different frame rates */
int (*power_down)(const dvp_camera_i2c_callback_t *cb); /**< set register to configure camera enable */
int (*dump_register)(const dvp_camera_i2c_callback_t *cb, media_ppi_t ppi); /**< Debug interface, view all register configuration values */
void (*read_register)(bool enable); /**< Enable the register check interface, check the value of the configuration register is consistent with the expected value during the configuration process*/
} dvp_sensor_config_t;
2) Refer to the enable camera driver code: middleware/driver/camera/dvp_camera.c, in the function: bk_dvp_camera_driver_init(),
it may be necessary to add the MCLK input configuration of the new camera;
Note: 480MHz is selected by default in the above JPEG gaze, and there is currently no open SDK interface to configure and select 480MHz or 320MHz.
If you need to choose 320MHz, please refer to the JPEG driver code: middleware/driver/jpeg_enc/jpeg_driver.c.
The only thing that uvc needs to adapt to is the resolution it supports. The resolution of uvc output is ever-changing.
Currently, only some conventional resolutions are adapted. If customers have special resolutions, they need to add them by themselves.
1) Currently, customers are not supported to add new resolutions independently, and they will be modified later,
because the current addition of new resolutions requires developers to give customers a new libusb.a file. Replace the path: components/bk_libs/bk7256_app/libs/libusb.a
2) After replacing the new libusb.a file, refer to the header file: include/driver/media_types.h,
the parameters in the enumeration type media_ppi_t need to be added, if not.
Add a new resolution to the command line
If you need to use the cli command that comes with doorbell, you need to make the newly added resolution take effect, otherwise skip this step
Refer to the doorbell command line: components/media/cli/media_cli.c, adapt the new command, add a new resolution in the function: get_string_to_ppi();
Attention
The width of the camera resolution must be divisible by 16 (864/16), and the height must be divisible by 8 (480/8). Otherwise hardware decoding will fail.
When the pixels of the screen and the picture are opposite, for example, the screen is 480X800 and the camera is 800X480, the image can be rotated and displayed.
Rotated images are currently adapted as follows:
The values for each parameter are entered as follows:
//ConfigurationofblendLocationsframe_addr_offset=((start_y+VERSION_POSTION_Y)*frame->width+start_x+VERSION_POSTION_X)*2;lcd_font_config.pbg_addr=(uint8_t*)(frame->frame+frame_addr_offset);//addressoffsetlcd_font_config.bg_offline=frame->width-CLOCK_LOGO_W;lcd_font_config.xsize=CLOCK_LOGO_W;///Thewidthofthefusion/blendareaisdeterminedaccordingtothewidthofChinesecharacterslcd_font_config.ysize=CLOCK_LOGO_H;///TheheightofthefusionareaisdeterminedbytheheightoftheChinesecharacterslcd_font_config.str_num=2;///Mergeseveralstringsatonce#if 1 ///font yuv data to bg yuv imageif(frame->fmt==PIXEL_FMT_VUYY)///FusionChinesecharactersintoYUVdatalcd_font_config.font_format=FONT_VUYY;elselcd_font_config.font_format=FONT_YUYV;#else ///font rgb data to bg yuv imagelcd_font_config.font_format=FONT_RGB565;///FusionChinesecharactersintoRGB565data#endif///Whitefont,fontsize,blendingintothestartingcoordinatesofthearealcd_font_config.str[0]=(font_str_t){(constchar*)("MU, 27℃"),FONT_WHITE,font_digit_black24,0,2};lcd_font_config.str[1]=(font_str_t){(constchar*)("2022-12-12 VL"),FONT_WHITE,font_digit_black24,0,26};lcd_font_config.bg_data_format=frame->fmt;///backgrounddataformatlcd_font_config.bg_width=frame->width;///backgroundimagesizelcd_font_config.bg_height=frame->height;lcd_driver_font_blend(&lcd_font_config);
