An assistance device for a search dog with a GPS sensor and multimedia transmission

Client

A European software and hardware company focused on IoT devices.

Challenge

The customer approached us to finalise their product — an assistance device for a search dog with a GPS sensor and multimedia transmission.

Our task was to develop a mobile application for Android and ensure the transfer of audio-video streams between a smartphone, a server and a wearable device for a search dog. The device should identify the dog's location and support visual and audio monitoring by transmitting data to a cloud server via a mobile internet connection and receiving commands from the operator.

Solution

Also, our engineering team ensured the multimedia transmission. For remote control and monitoring of the device, we used MQTT, a lightweight signalling protocol that was designed to keep a reliable connection in networks with limited resources.

MQTT enables a device to notify the server when it connects and report the status. On the other hand, the server can control a device (e.g. start or stop streaming). QoS2 allows the reliable delivery of only one copy of a message. 

^{Figure 1. Server-device interaction over MQTT}

On the device side, a stream from the camera goes to the MJPEG encoder, and a stream from a microphone goes to the MP3 encoder respectively. 

We chose MJPEG codec to ensure stable device operation when the mobile internet was unstable: when network bandwidth drops, we will still see complete frames, and only FPS will be decreased. 

MP3 format was used because it had a good quality-bitrate ratio. FFMPEG did video and audio stream encoding and decoding.

^{Figure 2. Mobile client-device interaction}

Our engineers chose RTP for real-time audio and video delivery. The protocol provided facilities for jitter compensation and detection of packet loss and out-of-order delivery. RTP was used in conjunction with the RTP Control Protocol (RTCP). While RTP carried the media streams (e.g., audio and video), RTCP monitored transmission statistics and quality of service (QoS), as well as supported the synchronisation of multiple streams. 

The device sends video and audio streams to the server. Also, the user can have an audio channel to the device. To deliver streams between the mobile client and the server, we used WebRTC (according to the customer's requirement).

If network bandwidth is insufficient for video delivery, the server will receive a video stream with low FPS (even with FPS < 1). In order not to lose video data, the device can detect such situations and record video to the internal storage. When the device detects enough bandwidth, it uploads local records to the server via HTTPS.

The technologies used in the project:

Business Value

The client received a new product for their target audience. The solution can identify the dog's location coordinates, receive commands from the operator, transmit video and audio to the cloud server via a mobile internet connection.