An assistance device for a search dog. Mobile application development and multimedia transmission for the GPS device designed to find lost people
MQTT for remote control and monitoring. A lightweight protocol that ensures a reliable connection in networks with limited resources
MJPEG codec. To ensure stable device operation when the mobile internet is unstable
An assistance device for a search dog. Mobile application development and multimedia transmission for the GPS device designed to find lost people
MQTT for remote control and monitoring. A lightweight protocol that ensures a reliable connection in networks with limited resources
MJPEG codec. To ensure stable device operation when the mobile internet is unstable

An assistance device for a search dog with a GPS sensor and multimedia transmission

Client

A European software and hardware company focused on IoT devices.

 

Challenge

The customer approached us to finalise their product — an assistance device for a search dog with a GPS sensor and multimedia transmission.

Our task was to develop a mobile application for Android and ensure the transfer of audio-video streams between a smartphone, a server and a wearable device for a search dog. The device should identify the dog's location and support visual and audio monitoring by transmitting data to a cloud server via a mobile internet connection and receiving commands from the operator.

 

Solution

 

We developed an Android app to interact with a wearable device that search dogs wear on their chests. Their device consists of a camera, microphone, audio speaker and GPS sensor in a compact enclosure so that operators can see and hear what is happening around the dogs and communicate with them.
 

Also, our engineering team ensured the multimedia transmission. For remote control and monitoring of the device, we used MQTT, a lightweight signalling protocol that was designed to keep a reliable connection in networks with limited resources.

MQTT enables a device to notify the server when it connects and report the status. On the other hand, the server can control a device (e.g. start or stop streaming). QoS2 allows the reliable delivery of only one copy of a message. 

 

 


Figure 1. Server-device interaction over MQTT

 

On the device side, a stream from the camera goes to the MJPEG encoder, and a stream from a microphone goes to the MP3 encoder respectively. 

We chose MJPEG codec to ensure stable device operation when the mobile internet was unstable: when network bandwidth drops, we will still see complete frames, and only FPS will be decreased. 

MP3 format was used because it had a good quality-bitrate ratio. FFMPEG did video and audio stream encoding and decoding.



Figure 2. Mobile client-device interaction

 

Our engineers chose RTP for real-time audio and video delivery. The protocol provided facilities for jitter compensation and detection of packet loss and out-of-order delivery. RTP was used in conjunction with the RTP Control Protocol (RTCP). While RTP carried the media streams (e.g., audio and video), RTCP monitored transmission statistics and quality of service (QoS), as well as supported the synchronisation of multiple streams. 

The device sends video and audio streams to the server. Also, the user can have an audio channel to the device. To deliver streams between the mobile client and the server, we used WebRTC (according to the customer's requirement).

 


 

If network bandwidth is insufficient for video delivery, the server will receive a video stream with low FPS (even with FPS < 1). In order not to lose video data, the device can detect such situations and record video to the internal storage. When the device detects enough bandwidth, it uploads local records to the server via HTTP.

The technologies used in the project: 

  • Programming language: C++14, 
  • CI/CD: Jenkins, 
  • Code review system: GitHub, 
  • Version control: git, 
  • Libraries: curl, FFmpeg, Boost, MQTT, libjpeg, RTP.

 

Business Value

The client received a new product for their target audience. The solution can identify the dog's location coordinates, receive commands from the operator, transmit video and audio to the cloud server via a mobile internet connection.

Other cases

Tell us about your project

All submitted information will be kept confidential.