Network Transfer Module
Module Introduction
Network Transfer is a unified network transmission abstraction layer component for network transmission of multimedia data (audio, video) and control commands. This component adopts layered architecture and modular design, decoupling upper-layer multimedia applications from underlying transmission protocols through standardized interfaces, enabling developers to flexibly switch or customize transmission solutions (such as Agora RTC, VolcEngine RTC, etc.).
Core Features
Unified Abstract Interface: Provides consistent API for upper-layer applications, shielding underlying transmission protocol differences
Pluggable Protocol: Supports dynamic registration of transmission solutions, convenient for integrating custom protocols
Multiple Backend Support: Supports multiple RTC backends such as Agora RTC, VolcEngine RTC
Event-driven Mechanism: Real-time notification of state changes and data reception to application layer through callback mechanism
Auto Routing: Audio data automatically routed to audio engine, video data automatically routed to network
Module Architecture
The Network Transfer module architecture is as follows:
Application Layer (Application)
↓
Network Transfer API (network_transfer.h)
↓
RTC Backend Adapter Layer
├── Agora RTC (bk_agora_api.c)
└── VolcEngine RTC (bk_volc_api.c)
↓
RTC SDK
├── Agora RTC SDK
└── VolcEngine RTC SDK
↓
Network Layer
Workflow
Initialization Flow
The Network Transfer initialization flow is as follows:
ntwk_trans_init()
↓
1. Check if Already Initialized
- If initialized, return directly
↓
2. Select RTC Backend Based on Configuration
#if CONFIG_AGORA_IOT_SDK
- Set Agora RTC callback functions
* audio_tx_cb = bk_agora_rtc_audio_data_send
* video_tx_cb = bk_agora_rtc_video_data_send
* start_cb = bk_agora_start
* stop_cb = bk_agora_stop
* pre_config_cb = bk_agora_pre_config
* update_cb = bk_agora_update_agent
- network_type = NETWORK_TYPE_AGORA_RTC
#elif CONFIG_VOLC_RTC_EN
- Set VolcEngine RTC callback functions
* audio_tx_cb = bk_byte_rtc_audio_data_send
* video_tx_cb = bk_byte_rtc_video_data_send
* start_cb = bk_byte_start
* stop_cb = bk_byte_stop
* pre_config_cb = bk_byte_pre_config
* update_cb = bk_byte_update_agent
- network_type = NETWORK_TYPE_VOLC_RTC
#endif
↓
3. Execute Pre-configuration Callback
- pre_config_cb(user_data)
- Register CLI commands, etc.
↓
4. Set Initialization Flag
- initialized = true
↓
Initialization Complete
Start Flow
Detailed flow of ntwk_trans_start():
1. Check Initialization Status
- If not initialized, return error
- If already started, return directly
↓
2. Call Start Callback
- start_cb(user_data)
- For Agora: bk_agora_start()
* Start Agent
* Start RTC
- For VolcEngine: bk_byte_start()
↓
3. Set Start Flag
- is_started = true
↓
Start Complete
Audio Send Flow
Detailed flow of ntwk_trans_send_audio():
1. Parameter Validation
- Check initialization status
- Check data pointer and size
- Check audio type validity
↓
2. Call Audio Send Callback
- audio_tx_cb(data, size, audio_type)
- For Agora: bk_agora_rtc_audio_data_send()
* Check connection status
* Map encoder type
* agora_rtc_send_audio_data()
- For VolcEngine: bk_byte_rtc_audio_data_send()
↓
3. Return Send Result
↓
Send Complete
Video Send Flow
Detailed flow of ntwk_trans_send_video():
1. Parameter Validation
- Check initialization status
- Check frame buffer validity
- Check frame data pointer and length
↓
2. Call Video Send Callback
- video_tx_cb(frame)
- For Agora: bk_agora_rtc_video_data_send()
* Check connection status
* I-frame filtering (H.264 only)
* Frame rate control (500ms interval)
* agora_rtc_send_video_data()
- For VolcEngine: bk_byte_rtc_video_data_send()
↓
3. Return Send Result
↓
Send Complete
Audio Receive Flow
Detailed flow of ntwk_trans_recv_audio():
1. Parameter Validation
- Check initialization status
- Check data pointer and size
↓
2. Write to Audio Engine
- audio_engine_write_data(data, size, 0)
- Audio engine automatically decodes and plays
↓
Receive Complete
Stop Flow
Detailed flow of ntwk_trans_stop():
1. Check Status
- If not initialized, return error
- If not started, return error
↓
2. Clear Start Flag
- is_started = false
↓
3. Call Stop Callback
- stop_cb(user_data)
- For Agora: bk_agora_stop()
* Stop RTC
* Stop Agent
- For VolcEngine: bk_byte_stop()
↓
Stop Complete
Important Interfaces
Initialization Interface
/**
* @brief Initialize network transfer module
*
* Selects RTC backend (Agora or VolcEngine) based on configuration,
* sets corresponding callback functions, and executes pre-configuration.
*
* @return int
* - 0: Success
* - < 0: Failure
*/
int ntwk_trans_init(void);
/**
* @brief Deinitialize network transfer module
*
* Stops network transfer and clears global context.
*
* @return int
* - 0: Success
* - < 0: Failure
*/
int ntwk_trans_deinit(void);
Start and Stop Interface
/**
* @brief Start network transfer
*
* Calls the start callback function of underlying RTC backend.
*
* @param user_data User data pointer, passed to start callback function
* @return int
* - 0: Success
* - < 0: Failure
*/
int ntwk_trans_start(void *user_data);
/**
* @brief Stop network transfer
*
* Calls the stop callback function of underlying RTC backend.
*
* @param user_data User data pointer, passed to stop callback function
* @return int
* - 0: Success
* - < 0: Failure
*/
int ntwk_trans_stop(void *user_data);
/**
* @brief Update network transfer
*
* Calls the update callback function of underlying RTC backend, used for updating Agent configuration, etc.
*
* @param user_data User data
* @param update_info Update information
* @return int
* - 0: Success
* - < 0: Failure
*/
int ntwk_trans_update(void *user_data, void *update_info);
Data Send Interface
/**
* @brief Send audio data to network
*
* @param data Audio data pointer
* @param size Audio data size
* @param audio_type Audio encoder type
* @return int
* - >= 0: Number of bytes sent
* - < 0: Failure
*/
int ntwk_trans_send_audio(const uint8_t *data, size_t size,
audio_enc_type_t audio_type);
/**
* @brief Send video frame to network
*
* @param frame Video frame buffer pointer
* @return int
* - 0: Success
* - < 0: Failure
*/
int ntwk_trans_send_video(frame_buffer_t *frame);
Data Receive Interface
/**
* @brief Receive audio data and write to audio engine
*
* This function writes received audio data to audio engine for decoding and playback.
*
* @param data Audio data pointer
* @param size Audio data size
* @return int
* - 0: Success
* - < 0: Failure
*/
int ntwk_trans_recv_audio(const uint8_t *data, size_t size);
Query Interface
/**
* @brief Get current network transfer type
*
* @return network_type_t Network type enumeration value
*/
network_type_t ntwk_trans_get_network_type(void);
/**
* @brief Get audio encoder type
*
* Gets the encoder type currently used from audio engine.
*
* @return audio_enc_type_t Audio encoder type enumeration value
*/
audio_enc_type_t ntwk_trans_get_audio_encoder_type(void);
Data Structures
network_type_t: Network type enumeration
typedef enum {
NETWORK_TYPE_VOLC_RTC = 0, // VolcEngine RTC
NETWORK_TYPE_AGORA_RTC, // Agora RTC
NETWORK_TYPE_MAX
} network_type_t;
ntwk_trans_ctx_t: Network transfer module context structure
typedef struct {
network_type_t network_type; // Network type
void *network_config; // Network-specific configuration
audio_tx_callback_t audio_tx_cb; // Audio send callback
video_tx_callback_t video_tx_cb; // Video send callback
ntwk_trans_start_callback_t start_cb; // Start callback
ntwk_trans_stop_callback_t stop_cb; // Stop callback
ntwk_trans_pre_config_callback_t pre_config_cb; // Pre-configuration callback
ntwk_trans_update_callback_t update_cb; // Update callback
void *user_data; // User data
bool is_started; // Whether started
bool initialized; // Initialization flag
} ntwk_trans_ctx_t;
Callback Function Types:
// Audio send callback
typedef int (*audio_tx_callback_t)(uint8_t *data_ptr, size_t data_len,
audio_enc_type_t audio_type);
// Video send callback
typedef int (*video_tx_callback_t)(frame_buffer_t *frame);
// Start callback
typedef bk_err_t (*ntwk_trans_start_callback_t)(void *user_data);
// Stop callback
typedef bk_err_t (*ntwk_trans_stop_callback_t)(void *user_data);
// Pre-configuration callback
typedef bk_err_t (*ntwk_trans_pre_config_callback_t)(void *user_data);
// Update callback
typedef bk_err_t (*ntwk_trans_update_callback_t)(void *user_data,
void *update_info);
Main Macro Definitions
Kconfig Configuration Macros
Basic Configuration:
// Enable network transfer module
CONFIG_BK_NETWORK_TRANSFER=y
RTC Backend Selection:
// Use Agora RTC
CONFIG_AGORA_IOT_SDK=y
// Or use VolcEngine RTC
// CONFIG_VOLC_RTC_EN=y
Other Configuration:
// Enable WebSocket transmission (AI scenario)
CONFIG_BK_WSS_TRANS=y
// WebSocket transmission does not use PSRAM
CONFIG_BK_WSS_TRANS_NOPSRAM=y
// Enable Sensenova (Agora)
CONFIG_SENSENOVA_ENABLE=y
// Start Agent from Beken server (default)
CONFIG_STARTUP_AGENT_FROM_BK_SERVER=y
// Or start Agent from custom server
// CONFIG_STARTUP_AGENT_FROM_BK_SERVER=n
Usage Examples
Basic Usage
#include "network_transfer.h"
#include "audio_engine.h"
#include "video_engine.h"
// Audio read callback (receive data from audio engine)
static int audio_read_callback(unsigned char *data, unsigned int len, void *args)
{
// Send audio data to network
return ntwk_trans_send_audio(data, len,
ntwk_trans_get_audio_encoder_type());
}
int main(void)
{
// 1. Initialize audio engine
audio_engine_init();
// 2. Initialize video engine
video_engine_init();
// 3. Initialize network transfer
int ret = ntwk_trans_init();
if (ret != 0) {
LOGE("Network transfer init failed\n");
return -1;
}
// 4. Start network transfer
char device_id[65] = "your_device_id";
ret = ntwk_trans_start(device_id);
if (ret != 0) {
LOGE("Network transfer start failed\n");
return -1;
}
// 5. Check network type
network_type_t type = ntwk_trans_get_network_type();
if (type == NETWORK_TYPE_AGORA_RTC) {
LOGI("Using Agora RTC\n");
} else if (type == NETWORK_TYPE_VOLC_RTC) {
LOGI("Using VolcEngine RTC\n");
}
// 6. Audio data will be automatically sent through audio_read_callback
// Video data will be automatically sent through video_engine
// 7. Received audio data will be automatically played (through ntwk_trans_recv_audio)
// 8. Stop network transfer
ntwk_trans_stop(device_id);
// 9. Deinitialize
ntwk_trans_deinit();
return 0;
}
Update Agent Configuration
// Update Agent configuration (e.g., switch multimodal)
void update_agent_mode(const char *mode)
{
int ret = ntwk_trans_update(device_id, (void *)mode);
if (ret != 0) {
LOGE("Update agent failed\n");
}
}
Notes
Initialization Order: Recommend initializing audio engine and video engine first, then network transfer
Backend Selection: Only one RTC backend can be selected (Agora or VolcEngine), configured through Kconfig
Callback Functions: Callback functions are automatically set by module internally based on configuration, application layer does not need to manually set
Data Routing: - Audio data: Automatically sent through audio_engine’s read_callback - Video data: Automatically sent through video_engine’s transfer task - Receive audio: Automatically written to audio_engine through ntwk_trans_recv_audio
State Check: Should ensure network transfer is started before sending data (by checking return value)
Error Handling: All interfaces have error return values, should check return values and handle errors
Resource Management: Must call
ntwk_trans_stop()andntwk_trans_deinit()after useThread Safety: Network transfer module uses thread-safe mechanisms internally, but callback functions should avoid long blocking
Agent Management: Agent start and stop are managed by underlying RTC backend, network transfer module only responsible for calling
Configuration Dependency: Ensure corresponding RTC backend is correctly configured (Agora requires App ID, Token, etc.)