Skip to content

Add Ground Station Streaming: hand video/telemetry ports to an external app #125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mengelh wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add Ground Station Streaming: hand video/telemetry ports to an external app #125

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 129 additions & 5 deletions app/src/main/java/com/openipc/pixelpilot/VideoActivity.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -662,6 +662,9 @@ private void showSettingsMenu(View anchor) {
// UDP Forwarding submenu
setupUdpForwardingSubMenu(popup);

// Ground Station Streaming submenu
setupGroundStationStreamingSubMenu(popup);

// Help submenu
setupHelpSubMenu(popup);

Expand Down Expand Up @@ -1162,12 +1165,112 @@ private void setupHelpSubMenu(PopupMenu popup) {

/**
* Starts the native Mavlink service and posts an initial Runnable to the Handler.
*
* When telemetry streaming is enabled, wfb-ng's own mavlink_aggregator
* already forwards the raw MAVLink stream to 127.0.0.1:14550 (see
* WfbngLink::initAgg). Skip this app-internal OSD parser so it doesn't
* compete for that port.
*/
private void setupMavlink() {
if (isTelemetryStreamingEnabled()) {
return;
}
MavlinkNative.nativeStart(this);
handler.post(runnable);
}

// ----------------------------------------------------------------------------
// GROUND STATION STREAMING
// ----------------------------------------------------------------------------
// Two independent toggles, since a ground station app might only care about
// one channel (e.g. a video-only OSD box, or a telemetry-only companion
// computer). wfb-ng's own aggregators forward raw RTP/H264 and MAVLink to
// 127.0.0.1:5600/14550 unconditionally (see WfbngLink::initAgg) regardless
// of these settings -- they only decide whether this app's own in-app
// consumers compete for those same ports, so any UDP-based ground station
// app (QGroundControl, Mission Planner, a custom OSD, ...) works, not just
// QGroundControl specifically.

private static final String PREF_STREAM_VIDEO = "gs_stream_video";
private static final String PREF_STREAM_TELEMETRY = "gs_stream_telemetry";

public boolean isVideoStreamingEnabled() {
return getSharedPreferences("general", MODE_PRIVATE).getBoolean(PREF_STREAM_VIDEO, false);
}

public boolean isTelemetryStreamingEnabled() {
return getSharedPreferences("general", MODE_PRIVATE).getBoolean(PREF_STREAM_TELEMETRY, false);
}

/**
* True while either channel is being streamed out -- used to keep the
* USB/wfb-ng pipeline alive across pause/resume instead of stopping it
* (see onPause()/onStop()). Both channels share one adapter, so it can't
* be paused for one and kept alive for the other.
*/
private boolean isGroundStationStreamingActive() {
return isVideoStreamingEnabled() || isTelemetryStreamingEnabled();
}

/**
* Persists the new video-streaming mode and applies it immediately
* instead of requiring a restart -- starting or stopping the in-app video
* player right away.
*/
private void setVideoStreamingEnabled(boolean enabled) {
getSharedPreferences("general", MODE_PRIVATE).edit().putBoolean(PREF_STREAM_VIDEO, enabled).apply();
if (enabled) {
videoPlayer.stop();

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot Oct 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

6. Video handoff can freeze the settings menu 🐞 Bug ☼ Reliability

setVideoStreamingEnabled(true) calls videoPlayer.stop() directly from the menu callback, and
native stop synchronously joins both receiver threads. If either receiver is still processing a
packet or has not exited its receive loop, the main thread waits for it before the menu interaction
can finish.
Agent Prompt 
## Issue description
The video-streaming menu action joins native receiver threads on the Android main thread.
## Fix Focus Areas
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1220-1229]
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1253-1261]
- app/videonative/src/main/cpp/VideoPlayer.cpp[190-204]
## Recommended Fix
Perform potentially blocking receiver teardown on a worker thread, and update the menu state on the main thread once the receiver has stopped and released its port.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

videoPlayer.stopAudio();
Comment on lines +1222 to +1224

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot Oct 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

4. Video handoff leaves recordings stalled 🐞 Bug ≡ Correctness

setVideoStreamingEnabled(true) stops the receivers without stopping an active DVR session or
changing its recording indicator. If the user switches to ground-station video during a recording,
the DVR receives no more decoded frames while its timer and recording UI continue until they
manually stop it.
Agent Prompt 
## Issue description
Switching video to an external ground station stops the source of DVR frames but leaves an active recording and its UI running.
## Fix Focus Areas
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1220-1229]
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1448-1504]
## Recommended Fix
Finalize any active DVR session before stopping the video receiver, and prevent starting an in-app DVR session while its video source is disabled.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +1222 to +1224

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot Oct 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

5. Video handoff keeps stale detection running 🐞 Bug ➹ Performance

setVideoStreamingEnabled(true) stops video reception without stopping the object-detection loop or
hiding its overlay. With detection enabled, that loop continues sampling the video view and running
inference even though the in-app video source has been disabled.
Agent Prompt 
## Issue description
Object detection continues consuming resources after video streaming disables the in-app video receiver.
## Fix Focus Areas
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1220-1229]
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[2181-2226]
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[2229-2317]
## Recommended Fix
Stop and clear the detection loop when video streaming is enabled; restart it when in-app video is restored if detection remains configured.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

} else {
videoPlayer.start();
updateUdpForwardingState();
videoPlayer.startAudio();
}
}

/**
* Persists the new telemetry-streaming mode and applies it immediately --
* starting or stopping the in-app MAVLink OSD parser right away.
*/
private void setTelemetryStreamingEnabled(boolean enabled) {
getSharedPreferences("general", MODE_PRIVATE).edit().putBoolean(PREF_STREAM_TELEMETRY, enabled).apply();
if (enabled) {
MavlinkNative.nativeStop(this);

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot Oct 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. External apps cannot bind the telemetry port 🐞 Bug ≡ Correctness

setTelemetryStreamingEnabled(true) calls nativeStop(), which signals the MAVLink listener to
exit but never closes its socket bound to port 14550. When telemetry was already running in-app,
that socket remains open after the listener exits, so the ground station cannot bind the port the
toggle is meant to free.
Agent Prompt 
## Issue description
Enabling telemetry streaming stops the listener thread but leaves its UDP socket bound, preventing another app from binding port 14550.
## Fix Focus Areas
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1236-1244]
- app/mavlink/src/main/cpp/mavlink.cpp[52-101]
- app/mavlink/src/main/cpp/mavlink.cpp[318-325]
## Recommended Fix
Ensure the native listener closes its socket on every exit path, and complete listener shutdown and socket release before treating the telemetry handoff as complete.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

handler.removeCallbacks(runnable);
} else {
MavlinkNative.nativeStart(this);
handler.post(runnable);
Comment on lines +1239 to +1243

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot Sep 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Telemetry never returns in-app 🐞 Bug ≡ Correctness

setTelemetryStreamingEnabled() calls nativeStop() and later nativeStart(), but the native stop
signal is only incremented and is never reset before a replacement listener starts. Every
enable-then-disable cycle therefore launches a listener that exits immediately, while a rapid cycle
can additionally bind before the old socket releases port 14550.
Agent Prompt 
## Issue description
Make the native MAVLink listener safely reusable so disabling ground-station telemetry reliably restores the in-app telemetry display.

## Issue Context
The new toggle stops and restarts the listener, but `nativeStop()` permanently increments a process-global signal and does not wait for the existing socket thread to finish. `nativeStart()` neither resets that signal nor synchronizes against the previous listener.

## Fix Focus Areas
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1236-1244]
- app/mavlink/src/main/cpp/mavlink.cpp[57-58]
- app/mavlink/src/main/cpp/mavlink.cpp[377-388]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

}
}

/**
* Settings submenu with the two streaming toggles.
*/
private void setupGroundStationStreamingSubMenu(PopupMenu popup) {
SubMenu gsMenu = popup.getMenu().addSubMenu("Ground Station Streaming");

MenuItem videoItem = gsMenu.add("Video");
videoItem.setCheckable(true);
videoItem.setChecked(isVideoStreamingEnabled());
videoItem.setOnMenuItemClickListener(item -> {
boolean newState = !item.isChecked();
item.setChecked(newState);
setVideoStreamingEnabled(newState);
return true;
});

MenuItem telemetryItem = gsMenu.add("Telemetry");
telemetryItem.setCheckable(true);
telemetryItem.setChecked(isTelemetryStreamingEnabled());
telemetryItem.setOnMenuItemClickListener(item -> {
boolean newState = !item.isChecked();
item.setChecked(newState);
setTelemetryStreamingEnabled(newState);
return true;
});
}

// ----------------------------------------------------------------------------
// BATTERY RECEIVER
// ----------------------------------------------------------------------------
Expand Down Expand Up @@ -1545,7 +1648,18 @@ protected void onPause() {

videoPlayer.stop();
videoPlayer.stopAudio();
wfbLinkManager.stopAdapters();
// While streaming to QGroundControl the whole point is for wfb-ng to keep
// decoding and forwarding to 127.0.0.1:5600/14550 while this Activity is
// backgrounded. Stopping/restarting the USB adapter on every pause/resume
// also hits a native teardown race in devourer's RtlJaguarDevice
// destructor (rtw_hal_deinit on a device that hasn't finished bringing up
// yet) when the two happen in quick succession. A real Foreground Service
// (Phase 4) is still needed for guaranteed survival under memory pressure
// -- this only keeps the pipeline alive across ordinary Activity
// lifecycle transitions.
if (!isGroundStationStreamingActive()) {
wfbLinkManager.stopAdapters();
}
Comment on lines +1660 to +1662

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot Sep 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Exiting leaves the radio pipeline running 🐞 Bug ☼ Reliability

The new lifecycle conditions skip stopAdapters() whenever either persisted streaming preference is
enabled, but the activity has no destruction cleanup for the retained native link, USB connection,
thread, or callback timer. Finishing or recreating the activity therefore leaves its old pipeline
holding the adapter and activity context, and reopening constructs another link that attempts to use
the same statically tracked device.
Agent Prompt 
## Issue description
Preserve streaming during temporary background transitions without orphaning the USB and native pipeline when the activity is actually destroyed.

## Issue Context
The activity now skips adapter shutdown in both pause and stop, while `WfbNgLink` owns per-instance native state, threads, USB connections, and a timer retaining the activity. Add explicit terminal cleanup or move ownership into a lifecycle-appropriate service; distinguish ordinary backgrounding from finishing or recreation.

## Fix Focus Areas
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1660-1662]
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[1676-1680]
- app/wfbngrtl8812/src/main/java/com/openipc/wfbngrtl8812/WfbNgLink.java[53-67]
- app/src/main/java/com/openipc/pixelpilot/WfbLinkManager.java[209-233]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


// Stop VPN service
Log.w(TAG, "onPause: stopping service");
Expand All @@ -1559,7 +1673,11 @@ protected void onStop() {
MavlinkNative.nativeStop(this);
handler.removeCallbacks(runnable);
unregisterReceivers();
wfbLinkManager.stopAdapters();
// See the comment in onPause() -- streaming mode keeps the USB/wfb-ng
// pipeline running across ordinary lifecycle transitions.
if (!isGroundStationStreamingActive()) {
wfbLinkManager.stopAdapters();
}
videoPlayer.stop();
videoPlayer.stopAudio();
super.onStop();
Expand All @@ -1576,9 +1694,15 @@ protected void onResume() {
wfbLinkManager.refreshAdapters();

wfbLinkManager.startAdapters();
videoPlayer.start();
updateUdpForwardingState();
videoPlayer.startAudio();
// While video streaming is enabled, skip the in-app video/audio
// receiver: wfb-ng's own video_aggregator already forwards RTP/H264 to
// 127.0.0.1:5600 (see WfbngLink::initAgg), and the ground station app
// binds that port instead.
if (!isVideoStreamingEnabled()) {
videoPlayer.start();
updateUdpForwardingState();
videoPlayer.startAudio();
}

SharedPreferences prefs = getSharedPreferences("general", MODE_PRIVATE);
boolean odEnabled = prefs.getBoolean("od_enabled", false);
Expand Down
Loading