diff --git a/CHANGELOG.md b/CHANGELOG.md index 2375ab3..1ecd0cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,12 @@ Versions after 2.17 are `MAJOR.MINOR.PATCH` (2.17 is followed by 2.17.1 or 2.18. `docs/testing.md`, "Versions and releases"); the headings below keep the numbers as they were released. +## 2.18.0 — 2026-10-19 + +- Mirror in the camera settings, next to Turn: flips the picture left to right in the single + view and the grid, for H.264, H.265 and MJPEG; a mirrored H.264/H.265 slot is drawn with + OpenGL ES (PR #246, closes #226). + ## 2.17 — 2026-10-17 - Turn in the camera settings (Auto, 0°, 90°, 180°, 270°), stored per slot; H.264/H.265 are diff --git a/README.md b/README.md index 9e693dd..7551eeb 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ A beta channel is available on [Google Play](https://play.google.com/apps/intern ``` rtsp://username:password@ip_address:554/stream=0 ``` - Under the URL you can choose the transport; Auto (UDP, falling back to TCP) is the default for a new slot. Below it, **Turn** turns the picture of a camera mounted sideways or upside down. Other URL types: [Transports](docs/transports.md). + Under the URL you can choose the transport; Auto (UDP, falling back to TCP) is the default for a new slot. Below it, **Turn** turns the picture of a camera mounted sideways or upside down, and **Mirror** flips it left to right. Other URL types: [Transports](docs/transports.md). 4. Press **Save**. The app connects straight away and reconnects automatically if the stream drops. ## Controls diff --git a/README.ru.md b/README.ru.md index d1b7d8a..7741cf2 100644 --- a/README.ru.md +++ b/README.ru.md @@ -65,7 +65,7 @@ ``` rtsp://username:password@ip_address:554/stream=0 ``` - Под адресом можно выбрать транспорт; у нового слота по умолчанию Auto (UDP с откатом на TCP). Ниже настройка **Turn** поворачивает картинку камеры, установленной боком или вверх ногами. Другие виды адресов — в [Transports](docs/transports.md) (на английском). + Под адресом можно выбрать транспорт; у нового слота по умолчанию Auto (UDP с откатом на TCP). Ниже настройка **Turn** поворачивает картинку камеры, установленной боком или вверх ногами, а **Mirror** отражает её слева направо. Другие виды адресов — в [Transports](docs/transports.md) (на английском). 4. Нажмите **Save**. Приложение сразу подключится и само переподключится, если поток прервётся. ## Управление diff --git a/README.zh-CN.md b/README.zh-CN.md index cab2fcf..f23354a 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -65,7 +65,7 @@ ``` rtsp://username:password@ip_address:554/stream=0 ``` - 地址下方可选择传输方式，新槽位默认为 Auto（优先 UDP，必要时回退到 TCP）。其下的 **Turn** 可旋转画面，适用于侧装或倒装的摄像机。其他地址格式见 [Transports](docs/transports.md)（英文）。 + 地址下方可选择传输方式，新槽位默认为 Auto（优先 UDP，必要时回退到 TCP）。其下的 **Turn** 可旋转画面，适用于侧装或倒装的摄像机；**Mirror** 可将画面左右镜像。其他地址格式见 [Transports](docs/transports.md)（英文）。 4. 点击 **Save**。应用会立即连接；视频流中断时会自动重连。 ## 操作说明 diff --git a/android/app/build.gradle b/android/app/build.gradle index f6bbe65..ab89f64 100755 --- a/android/app/build.gradle +++ b/android/app/build.gradle @@ -23,8 +23,8 @@ android { applicationId "com.openipc.decoder" minSdk 21 targetSdk 36 - versionCode 42 - versionName "2.17" + versionCode 43 + versionName "2.18.0" buildConfigField "String", "GIT_HASH", "\"${getGitHash()}\"" } diff --git a/android/app/src/main/java/com/openipc/decoder/Decoder.java b/android/app/src/main/java/com/openipc/decoder/Decoder.java index 33896c1..07a1a91 100755 --- a/android/app/src/main/java/com/openipc/decoder/Decoder.java +++ b/android/app/src/main/java/com/openipc/decoder/Decoder.java @@ -140,6 +140,8 @@ public class Decoder extends Activity private final Transport.Mode[] mTransports = new Transport.Mode[CAM_COUNT]; /** Per slot (#201): degrees counter-clockwise, -1 for Auto (from the URL). */ private final int[] mRotates = new int[CAM_COUNT]; + /** Per slot (#226): the picture flipped left↔right. */ + private final boolean[] mMirrors = new boolean[CAM_COUNT]; private int mActive; private volatile String mHost; private String mVersion; @@ -212,12 +214,9 @@ protected void onCreate(Bundle savedInstanceState) { // --- Create managers --- videoDecoder = new MediaCodecManager(new MediaCodecManager.Listener() { @Override public void onResolutionChanged(int w, int h) { - // The decoder turns the picture (#201, Android 6+ as in MediaCodecManager): a - // quarter turn swaps its sides - RtspClient c = rtspClient; - int r = c != null && Build.VERSION.SDK_INT >= Build.VERSION_CODES.M - ? c.rotate() : 0; - if (r == 90 || r == 270) updateResolution(h, w); + // A quarter turn by the decoder (#201, Android 6+) or by the mirror's GL path + // (#226, any version) swaps the picture's sides + if (videoDecoder.swapsSides()) updateResolution(h, w); else updateResolution(w, h); } @Override public void onDecoderStarted() { removeClearOverlay(); } @@ -591,7 +590,7 @@ private void restartClean() { @Override public void onShowSettings() { - menuManager.showUrlEditor(mActive, mHosts, mTransports, mRotates); + menuManager.showUrlEditor(mActive, mHosts, mTransports, mRotates, mMirrors); } @Override @@ -606,13 +605,17 @@ public void onShowSettings() { private void configureClient() { rtspClient.configure(mHost, mUserAgent, mTransports[mActive]); rtspClient.setSlotTurn(mRotates[mActive]); + rtspClient.setSlotMirror(mMirrors[mActive]); } - private void createDecoder() { + /** A decoder for the stream of video thread {@code gen}, with that stream's turn and mirror. */ + private void createDecoder(int gen) { videoDecoder.setSurface(mVideoSurface); RtspClient c = rtspClient; videoDecoder.setRotation(c != null ? c.rotate() : 0); - videoDecoder.createDecoder(codecH265); + videoDecoder.setMirror(c != null && c.mirror()); + // Restarted with new settings meanwhile: this decoder is dropped, the new thread makes one + videoDecoder.createDecoder(codecH265, () -> gen == listenerGen); } private void closeDecoder() { videoDecoder.closeDecoder(); } @@ -652,6 +655,7 @@ private void loadSettings() { // Auto for a slot that has never stored one (#112) mTransports[i] = Transport.Mode.stored(pref.getString("transport_" + i, null)); mRotates[i] = pref.getInt("rotate_" + i, -1); + mMirrors[i] = pref.getBoolean("mirror_" + i, false); // Plain URLs from older versions are re-saved encrypted right away if (credentials.needsMigration(stored)) { if (migrate == null) migrate = pref.edit(); @@ -715,6 +719,7 @@ private void saveSettings() { credentials.sealForSlot(mHosts[i], mUnreadable[i], DEFAULT_URL)); edit.putString("transport_" + i, mTransports[i].name()); edit.putInt("rotate_" + i, mRotates[i]); + edit.putBoolean("mirror_" + i, mMirrors[i]); } edit.apply(); @@ -1025,6 +1030,7 @@ private void createQuadCells() { quadCells[i] = new QuadCell(i, url, mode, quadViews[i], framePool, mUserAgent, index -> runOnUiThread(() -> removeQuadOverlay(index))); quadCells[i].setSlotTurn(i < CAM_COUNT ? mRotates[i] : -1); + quadCells[i].setSlotMirror(i < CAM_COUNT && mMirrors[i]); quadCells[i].start(); } quadAudio = getSharedPreferences("settings", MODE_PRIVATE).getInt("quad_audio", -1); @@ -1629,6 +1635,10 @@ private Map debugState() { // Turned counter-clockwise, "size" is the turned one (#200, #201) int rotate = rtspClient != null ? rtspClient.rotate() : 0; if (rotate != 0) single.put("rotate", rotate); + if (rtspClient != null && rtspClient.mirror()) { + single.put("mirror", true); + if (videoDecoder.mirrorFailed()) single.put("mirror_failed", true); + } m.put("single", single); } QuadCell[] cells = quadCells; @@ -1674,6 +1684,10 @@ private Map debugState() { Map hb = cell.heartbeatStats(); if (hb != null) c.put("heartbeat", hb); if (cell.rotate() != 0) c.put("rotate", cell.rotate()); + if (cell.mirror()) { + c.put("mirror", true); + if (cell.mirrorFailed()) c.put("mirror_failed", true); + } } list.add(c); } @@ -2291,7 +2305,7 @@ private boolean drawJpeg(Frame frame, int gen) { Surface s = mVideoSurface; RtspClient client = rtspClient; int rotate = client != null ? client.rotate() : 0; - int[] size = jpegRenderer.render(frame, s, rotate, + int[] size = jpegRenderer.render(frame, s, rotate, client != null && client.mirror(), () -> gen == listenerGen && s == mVideoSurface); if (size != null) jpegShown(size[0], size[1]); return size != null; @@ -2380,7 +2394,13 @@ && drawJpeg(buffer, gen)) { framePool.recycle(buffer); } else if (buffer != null) { if (videoDecoder.decodeFrame(buffer, codecH265, 0)) { - createDecoder(); + // Restarted meanwhile (new settings): the new thread makes the + // decoder, with the new turn and mirror (#226) + if (gen != listenerGen) { + framePool.recycle(buffer); + break; + } + createDecoder(gen); // The frame that triggered creation (often the SPS) goes in too videoDecoder.decodeFrame(buffer, codecH265, 0); } diff --git a/android/app/src/main/java/com/openipc/decoder/GlMirror.java b/android/app/src/main/java/com/openipc/decoder/GlMirror.java new file mode 100644 index 0000000..a3848df --- /dev/null +++ b/android/app/src/main/java/com/openipc/decoder/GlMirror.java @@ -0,0 +1,337 @@ +/* + * Copyright (c) OpenIPC https://openipc.org Prosperity Public License 3.0.0 + * + * GlMirror.java — draws a decoder's output mirrored (and turned) onto a surface with OpenGL ES + * + */ + +package com.openipc.decoder; + +import android.graphics.SurfaceTexture; +import android.opengl.EGL14; +import android.opengl.EGLConfig; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; +import android.opengl.GLES11Ext; +import android.opengl.GLES20; +import android.os.Handler; +import android.os.HandlerThread; +import android.util.Log; +import android.view.Surface; + +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.FloatBuffer; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; + +/** + * A slot with Mirror on (#226): the decoder renders into a SurfaceTexture, and every frame is + * drawn onto the view's surface flipped left↔right and turned like the slot's Turn. A + * SurfaceView cannot be mirrored with setScaleX(-1) (the system composes it), and + * MediaFormat.KEY_ROTATION has no mirror, so this path is used only for mirrored slots; the + * others keep the decoder drawing straight onto the SurfaceView (#76). + * + * Everything GL runs on one thread of its own. Release the decoder before this, so its output + * surface is gone when the EGL surface leaves the view's surface. + */ +final class GlMirror implements SurfaceTexture.OnFrameAvailableListener { + private static final String TAG = "OpenIPCDecoder"; + private static final long SYNC_TIMEOUT_MS = 2_000; + + private static final String VERTEX = + "uniform mat4 uMvp;\n" + + "uniform mat4 uSt;\n" + + "attribute vec4 aPos;\n" + + "attribute vec4 aTex;\n" + + "varying vec2 vTex;\n" + + "void main() {\n" + + " gl_Position = uMvp * aPos;\n" + + " vTex = (uSt * aTex).xy;\n" + + "}\n"; + private static final String FRAGMENT = + "#extension GL_OES_EGL_image_external : require\n" + + "precision mediump float;\n" + + "varying vec2 vTex;\n" + + "uniform samplerExternalOES sTex;\n" + + "void main() {\n" + + " gl_FragColor = texture2D(sTex, vTex);\n" + + "}\n"; + /** Triangle strip over the whole surface: x, y, then u, v. */ + private static final float[] QUAD = { + -1f, -1f, 0f, 0f, + 1f, -1f, 1f, 0f, + -1f, 1f, 0f, 1f, + 1f, 1f, 1f, 1f, + }; + + private final HandlerThread thread; + private final Handler handler; + private final float[] mvp; + private final float[] st = new float[16]; + private final FloatBuffer quad; + + private EGLDisplay display = EGL14.EGL_NO_DISPLAY; + private EGLContext context = EGL14.EGL_NO_CONTEXT; + private EGLSurface window = EGL14.EGL_NO_SURFACE; + private int program; + private int texture; + private int uMvp, uSt, aPos, aTex; + private SurfaceTexture surfaceTexture; + private Surface input; + private volatile boolean released; + /** A frame could not be shown (EGL error after setup): logged once, reported in stats. */ + private volatile boolean drawFailed; + /** eglInitialize() succeeded: tearDown() balances it with eglTerminate(). */ + private boolean initialized; + + private GlMirror(int turnCcw, boolean mirror) { + mvp = transform(turnCcw, mirror); + quad = ByteBuffer.allocateDirect(QUAD.length * 4).order(ByteOrder.nativeOrder()) + .asFloatBuffer(); + quad.put(QUAD).position(0); + thread = new HandlerThread("gl-mirror"); + thread.start(); + handler = new Handler(thread.getLooper()); + } + + /** + * The position transform: turned {@code turnCcw} degrees counter-clockwise, then flipped + * left↔right on the screen when {@code mirror}, so Mirror always swaps the sides of the + * picture as it is shown, whatever the turn. + */ + static float[] transform(int turnCcw, boolean mirror) { + // Quarter turns are exact; column-major, as glUniformMatrix4fv takes it + int t = ((turnCcw % 360) + 360) % 360; + float c = t == 0 ? 1f : t == 180 ? -1f : 0f; + float s = t == 90 ? 1f : t == 270 ? -1f : 0f; + float f = mirror ? -1f : 1f; + return new float[] { + f * c, s, 0f, 0f, + -f * s, c, 0f, 0f, + 0f, 0f, 1f, 0f, + 0f, 0f, 0f, 1f, + }; + } + + /** Where the point (x, y) of the full-surface quad lands with {@code m} (for tests). */ + static float[] map(float[] m, float x, float y) { + return new float[] {m[0] * x + m[4] * y + m[12], m[1] * x + m[5] * y + m[13]}; + } + + /** Set up on {@code target}; throws when EGL or GL cannot. Decode into {@link #input()}. */ + static GlMirror create(Surface target, int turnCcw, boolean mirror) throws Exception { + GlMirror g = new GlMirror(turnCcw, mirror); + try { + g.runSync(() -> g.setUp(target)); + } catch (Exception e) { + g.release(); + throw e; + } + return g; + } + + /** A frame could not be drawn since setup (Qodo on #246). */ + boolean drawFailed() { return drawFailed; } + + private void drawFailed(String why) { + if (released) return; + if (!drawFailed) Log.w(TAG, "Mirror frame not shown: " + why); + drawFailed = true; + } + + /** The surface the decoder renders into. */ + Surface input() { return input; } + + /** Release GL and the thread; waits until the view's surface is free again. */ + void release() { + // A frame already queued on the GL thread is no longer drawn (Qodo on #246) + released = true; + try { + runSync(this::tearDown); + } catch (Exception e) { + Log.w(TAG, "Mirror release: " + e.getMessage()); + } + thread.quitSafely(); + } + + private interface Task { + void run() throws Exception; + } + + private void runSync(Task task) throws Exception { + final Exception[] failure = new Exception[1]; + CountDownLatch done = new CountDownLatch(1); + if (!handler.post(() -> { + try { + task.run(); + } catch (Exception e) { + failure[0] = e; + } finally { + done.countDown(); + } + })) { + throw new IllegalStateException("mirror thread stopped"); + } + if (!done.await(SYNC_TIMEOUT_MS, TimeUnit.MILLISECONDS)) { + throw new IllegalStateException("mirror thread does not answer"); + } + if (failure[0] != null) throw failure[0]; + } + + private void setUp(Surface target) { + display = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY); + int[] version = new int[2]; + if (display == EGL14.EGL_NO_DISPLAY || !EGL14.eglInitialize(display, version, 0, version, 1)) { + throw new IllegalStateException("EGL display: " + eglError()); + } + initialized = true; + int[] attribs = { + EGL14.EGL_RED_SIZE, 8, EGL14.EGL_GREEN_SIZE, 8, EGL14.EGL_BLUE_SIZE, 8, + EGL14.EGL_RENDERABLE_TYPE, EGL14.EGL_OPENGL_ES2_BIT, + EGL14.EGL_SURFACE_TYPE, EGL14.EGL_WINDOW_BIT, + EGL14.EGL_NONE, + }; + EGLConfig[] configs = new EGLConfig[1]; + int[] count = new int[1]; + if (!EGL14.eglChooseConfig(display, attribs, 0, configs, 0, 1, count, 0) || count[0] < 1) { + throw new IllegalStateException("EGL config: " + eglError()); + } + context = EGL14.eglCreateContext(display, configs[0], EGL14.EGL_NO_CONTEXT, + new int[] {EGL14.EGL_CONTEXT_CLIENT_VERSION, 2, EGL14.EGL_NONE}, 0); + if (context == EGL14.EGL_NO_CONTEXT) throw new IllegalStateException("EGL context: " + eglError()); + window = EGL14.eglCreateWindowSurface(display, configs[0], target, + new int[] {EGL14.EGL_NONE}, 0); + if (window == EGL14.EGL_NO_SURFACE) throw new IllegalStateException("EGL surface: " + eglError()); + if (!EGL14.eglMakeCurrent(display, window, window, context)) { + throw new IllegalStateException("EGL current: " + eglError()); + } + + program = link(compile(GLES20.GL_VERTEX_SHADER, VERTEX), + compile(GLES20.GL_FRAGMENT_SHADER, FRAGMENT)); + uMvp = GLES20.glGetUniformLocation(program, "uMvp"); + uSt = GLES20.glGetUniformLocation(program, "uSt"); + aPos = GLES20.glGetAttribLocation(program, "aPos"); + aTex = GLES20.glGetAttribLocation(program, "aTex"); + + int[] tex = new int[1]; + GLES20.glGenTextures(1, tex, 0); + texture = tex[0]; + GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, texture); + GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, + GLES20.GL_LINEAR); + GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, + GLES20.GL_LINEAR); + GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, + GLES20.GL_CLAMP_TO_EDGE); + GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, + GLES20.GL_CLAMP_TO_EDGE); + + surfaceTexture = new SurfaceTexture(texture); + surfaceTexture.setOnFrameAvailableListener(this, handler); + input = new Surface(surfaceTexture); + } + + @Override + public void onFrameAvailable(SurfaceTexture t) { + // On the GL thread (the listener's handler) + if (released || t != surfaceTexture) return; + try { + surfaceTexture.updateTexImage(); + surfaceTexture.getTransformMatrix(st); + int[] size = new int[2]; + if (!EGL14.eglQuerySurface(display, window, EGL14.EGL_WIDTH, size, 0) + || !EGL14.eglQuerySurface(display, window, EGL14.EGL_HEIGHT, size, 1)) { + drawFailed("surface: " + eglError()); + return; + } + GLES20.glViewport(0, 0, size[0], size[1]); + GLES20.glClearColor(0f, 0f, 0f, 1f); + GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT); + GLES20.glUseProgram(program); + GLES20.glActiveTexture(GLES20.GL_TEXTURE0); + GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, texture); + GLES20.glUniformMatrix4fv(uMvp, 1, false, mvp, 0); + GLES20.glUniformMatrix4fv(uSt, 1, false, st, 0); + quad.position(0); + GLES20.glVertexAttribPointer(aPos, 2, GLES20.GL_FLOAT, false, 16, quad); + GLES20.glEnableVertexAttribArray(aPos); + quad.position(2); + GLES20.glVertexAttribPointer(aTex, 2, GLES20.GL_FLOAT, false, 16, quad); + GLES20.glEnableVertexAttribArray(aTex); + GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4); + if (!EGL14.eglSwapBuffers(display, window)) drawFailed("swap: " + eglError()); + } catch (RuntimeException e) { + // The surface went away meanwhile (view or activity change): the owner releases us + drawFailed(e.getMessage()); + } + } + + private void tearDown() { + released = true; + if (surfaceTexture != null) { + surfaceTexture.setOnFrameAvailableListener(null); + surfaceTexture.release(); + surfaceTexture = null; + } + if (input != null) { + input.release(); + input = null; + } + if (display != EGL14.EGL_NO_DISPLAY) { + if (program != 0) GLES20.glDeleteProgram(program); + if (texture != 0) GLES20.glDeleteTextures(1, new int[] {texture}, 0); + EGL14.eglMakeCurrent(display, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE, + EGL14.EGL_NO_CONTEXT); + // Destroying the EGL surface frees the view's surface for a plain decoder again + if (window != EGL14.EGL_NO_SURFACE) EGL14.eglDestroySurface(display, window); + if (context != EGL14.EGL_NO_CONTEXT) EGL14.eglDestroyContext(display, context); + // Android counts eglInitialize() per display: this only drops our reference, the + // other mirrored cells and the system renderer keep theirs (Qodo on #246) + if (initialized) EGL14.eglTerminate(display); + EGL14.eglReleaseThread(); + } + initialized = false; + window = EGL14.EGL_NO_SURFACE; + context = EGL14.EGL_NO_CONTEXT; + display = EGL14.EGL_NO_DISPLAY; + program = 0; + texture = 0; + } + + private static int compile(int type, String source) { + int shader = GLES20.glCreateShader(type); + GLES20.glShaderSource(shader, source); + GLES20.glCompileShader(shader); + int[] ok = new int[1]; + GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, ok, 0); + if (ok[0] == 0) { + String log = GLES20.glGetShaderInfoLog(shader); + GLES20.glDeleteShader(shader); + throw new IllegalStateException("shader: " + log); + } + return shader; + } + + private static int link(int vertex, int fragment) { + int p = GLES20.glCreateProgram(); + GLES20.glAttachShader(p, vertex); + GLES20.glAttachShader(p, fragment); + GLES20.glLinkProgram(p); + GLES20.glDeleteShader(vertex); + GLES20.glDeleteShader(fragment); + int[] ok = new int[1]; + GLES20.glGetProgramiv(p, GLES20.GL_LINK_STATUS, ok, 0); + if (ok[0] == 0) { + String log = GLES20.glGetProgramInfoLog(p); + GLES20.glDeleteProgram(p); + throw new IllegalStateException("program: " + log); + } + return p; + } + + private static String eglError() { + return "0x" + Integer.toHexString(EGL14.eglGetError()); + } +} diff --git a/android/app/src/main/java/com/openipc/decoder/JpegRenderer.java b/android/app/src/main/java/com/openipc/decoder/JpegRenderer.java index deb38f8..e9b258c 100644 --- a/android/app/src/main/java/com/openipc/decoder/JpegRenderer.java +++ b/android/app/src/main/java/com/openipc/decoder/JpegRenderer.java @@ -44,15 +44,17 @@ final class JpegRenderer { /** * Shows one frame on {@code s}, turned {@code rotate} degrees counter-clockwise (0, 90, - * 180 or 270, #200). Returns its {width, height} as shown (turned) when it is on screen, + * 180 or 270, #200), then flipped left↔right on the screen when {@code mirror} (#226). Returns its {width, height} as shown (turned) when it is on screen, * null when not (yet). {@code current} is false once the stream or the surface has been * replaced. */ - int[] render(Frame frame, Surface s, int rotate, JpegCodec.Current current) { + int[] render(Frame frame, Surface s, int rotate, boolean mirror, JpegCodec.Current current) { if (s == null || !s.isValid() || !current.holds()) return null; // The hardware JPEG decoder first, where there is one (#132); it turns the frame itself - // from Android 6 on (#201), before that a turned frame is drawn by the app (#200) - boolean hwTurns = rotate == 0 || Build.VERSION.SDK_INT >= Build.VERSION_CODES.M; + // from Android 6 on (#201), before that a turned frame is drawn by the app (#200). It + // cannot mirror: a mirrored frame is drawn by the app (#226) + boolean hwTurns = !mirror + && (rotate == 0 || Build.VERSION.SDK_INT >= Build.VERSION_CODES.M); if (hwTurns && !hwFailed && !drawn && JpegCodec.candidate() != null) { int[] size = JpegAssembler.size(frame.data(), frame.length()); if (size != null) { @@ -90,6 +92,8 @@ int[] render(Frame frame, Surface s, int rotate, JpegCodec.Current current) { Canvas c = s.lockCanvas(null); try { c.drawColor(Color.BLACK); + // Flipped on the screen, after the turn (#226) + if (mirror) c.scale(-1f, 1f, c.getWidth() / 2f, c.getHeight() / 2f); if (rotate == 0) { c.drawBitmap(b, null, new Rect(0, 0, c.getWidth(), c.getHeight()), paint); } else { diff --git a/android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java b/android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java index 99b0119..f66ce7d 100644 --- a/android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java +++ b/android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java @@ -60,6 +60,12 @@ class MediaCodecManager { private volatile int rotation; /** The turn the current decoder applies: {@link #rotation} where Android 6+ can, else 0. */ private volatile int appliedRotation; + /** Mirror the picture (#226): the decoder renders through {@link GlMirror}. */ + private volatile boolean mirror; + /** The GL path of the current decoder (decoderLock), released after the decoder. */ + private volatile GlMirror glMirror; + /** The current decoder draws mirrored. */ + private volatile boolean mirrored; /** Frames rendered so far (#82): with the frames received this gives the real decode rate. */ private volatile int framesOut; @@ -269,6 +275,28 @@ void setRotation(int degreesCcw) { rotation = ((degreesCcw % 360) + 360) % 360; } + /** See {@link #mirror}; takes effect with the next decoder. */ + void setMirror(boolean on) { + mirror = on; + } + + /** Mirror is on, but the current decoder draws unmirrored: the GL path failed (#226). */ + boolean mirrorFailed() { + GlMirror g = glMirror; + return mirror && decoder != null && (!mirrored || (g != null && g.drawFailed())); + } + + /** Detach the GL path of the current decoder (decoderLock held), like {@link #takeSlot}. */ + private GlMirror takeMirror() { + GlMirror g = glMirror; + glMirror = null; + return g; + } + + private static void releaseMirror(GlMirror g) { + if (g != null) g.release(); + } + /** Enable or disable decoding; disabling also releases the decoder. */ /** * Enable or disable decoding. Never blocks the caller (the UI thread): disabling only @@ -283,13 +311,17 @@ void setEnabled(boolean enabled) { if (target == null) return; new Thread(() -> { String slot; + GlMirror gl; synchronized (decoderLock) { if (decoder != target) return; // already replaced or released decoder = null; slot = takeSlot(); + gl = takeMirror(); retry.onSuccess(); } releaseNow(target); + // After the decoder: its output must be gone before GL leaves the surface + releaseMirror(gl); releaseSlot(slot); }, "decoder-release").start(); } @@ -299,8 +331,11 @@ private static void releaseNow(MediaCodec codec) { try { codec.release(); } catch (Exception ignored) {} } - private static void releaseAsync(MediaCodec codec) { - new Thread(() -> releaseNow(codec), "decoder-release").start(); + private static void releaseAsync(MediaCodec codec, GlMirror gl) { + new Thread(() -> { + releaseNow(codec); + releaseMirror(gl); + }, "decoder-release").start(); } /** Release the decoder on purpose (codec switch, camera switch): no back-off. */ @@ -315,6 +350,9 @@ void closeDecoder() { private void closeDecoderLocked() { synchronized (decoderLock) { + // A creation still running (another thread, older settings) is discarded when it + // ends: it must not replace the decoder the caller will make next (#226) + lifecycle++; retry.onSuccess(); MediaCodec codec = decoder; // The next stream must not look live because of this one's last frame @@ -322,6 +360,7 @@ private void closeDecoderLocked() { if (codec == null) return; decoder = null; String slot = takeSlot(); + GlMirror gl = takeMirror(); Log.i(TAG, "Close video decoder"); try { codec.stop(); @@ -329,6 +368,7 @@ private void closeDecoderLocked() { } catch (Exception e) { Log.e(TAG, "Decoder close exception", e); } + releaseMirror(gl); releaseSlot(slot); } } @@ -338,6 +378,15 @@ private static long nowMs() { } void createDecoder(boolean codecH265) { + createDecoder(codecH265, null); + } + + /** + * Like {@link #createDecoder(boolean)}; {@code current} (may be null) is checked under the + * lock when the decoder is installed: false once the caller's stream was restarted with + * other settings, and the decoder made with the old ones is dropped (Qodo on #246). + */ + void createDecoder(boolean codecH265, JpegCodec.Current current) { final int gen; synchronized (decoderLock) { if (decoder != null || !enabled || !retry.mayCreate(nowMs())) return; @@ -379,6 +428,8 @@ void createDecoder(boolean codecH265) { final String slot = software ? softwareSlot(type) : type; MediaCodec local; + GlMirror gl = null; + int turnApplied = 0; try { if (software) { Log.i(TAG, "Start software video decoder " + softwareName @@ -390,19 +441,38 @@ void createDecoder(boolean codecH265) { learnLimits(local, type); } int turn = rotation; - if (turn != 0 && Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) { + Surface target = s; + if (mirror) { + // Mirrored (#226): drawn with GL, which turns the picture as well + try { + gl = GlMirror.create(s, turn, true); + target = gl.input(); + Log.i(TAG, "Mirrored picture" + (turn != 0 ? ", turned " + turn + + "\u00b0 counter-clockwise" : "") + " (drawn with OpenGL ES)"); + } catch (Exception e) { + Log.w(TAG, "Mirror unavailable, the picture is shown as it comes: " + + e.getMessage()); + gl = null; + } + } + if (gl == null && turn != 0 && Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) { // The decoder turns its output on the surface: clockwise degrees here (#201) format.setInteger(MediaFormat.KEY_ROTATION, (360 - turn) % 360); - } else { + } else if (gl == null) { turn = 0; } - appliedRotation = turn; + turnApplied = turn; try { - local.configure(format, s, null, 0); + local.configure(format, target, null, 0); local.start(); if (listener != null) listener.onDecoderStarted(); } catch (Exception e) { - local.release(); + try { + local.release(); + } finally { + releaseMirror(gl); + gl = null; + } throw e; } } catch (Exception e) { @@ -419,12 +489,17 @@ void createDecoder(boolean codecH265) { synchronized (decoderLock) { // Disabled meanwhile (pause/stop) or another thread was faster - if (decoder != null || !enabled || gen != lifecycle) { + if (decoder != null || !enabled || gen != lifecycle + || (current != null && !current.holds())) { InstanceBudget.release(slot); - releaseAsync(local); + releaseAsync(local, gl); return; } decoder = local; + glMirror = gl; + mirrored = gl != null; + // Only the decoder that stays sets these: a creation that lost must not (#226) + appliedRotation = turnApplied; // A software decoder holds a slot of its own budget, not of the hardware one heldMime = slot; softwareInUse = software; @@ -499,11 +574,13 @@ boolean decodeFrame(Frame buffer, boolean codecH265, long watchdogRef) { } catch (Exception e) { decoder = null; String slot = takeSlot(); + GlMirror gl = takeMirror(); retry.onFailure(nowMs()); Log.e(TAG, "Codec exception, retry in " + DecoderRetry.delayMs(retry.failures()) + " ms: " + e.getMessage()); try { codec.stop(); } catch (Exception ignored) {} try { codec.release(); } catch (Exception ignored) {} + releaseMirror(gl); releaseSlot(slot); if (listener != null) listener.onDecoderFailed(); } diff --git a/android/app/src/main/java/com/openipc/decoder/MenuManager.java b/android/app/src/main/java/com/openipc/decoder/MenuManager.java index 7a63b80..7687e08 100644 --- a/android/app/src/main/java/com/openipc/decoder/MenuManager.java +++ b/android/app/src/main/java/com/openipc/decoder/MenuManager.java @@ -27,6 +27,7 @@ import android.webkit.HttpAuthHandler; import android.webkit.WebView; import android.webkit.WebViewClient; +import android.widget.CheckBox; import android.widget.EditText; import android.widget.HorizontalScrollView; import android.widget.LinearLayout; @@ -542,7 +543,7 @@ private EditText createEdit(String title) { } void showUrlEditor(int mActive, String[] mHosts, Transport.Mode[] mTransports, - int[] mRotates) { + int[] mRotates, boolean[] mMirrors) { AlertDialog.Builder builder = new AlertDialog.Builder(activity); builder.setTitle("Camera " + (mActive + 1) + " URL"); @@ -589,6 +590,11 @@ void showUrlEditor(int mActive, String[] mHosts, Transport.Mode[] mTransports, HorizontalScrollView turnRow = new HorizontalScrollView(activity); turnRow.addView(turn); content.addView(turnRow); + // Mirror of the picture (#226): left↔right, after the turn + CheckBox mirror = new CheckBox(activity); + mirror.setText("Mirror"); + mirror.setChecked(mMirrors[mActive]); + content.addView(mirror); builder.setView(content); final int slot = mActive; @@ -599,6 +605,7 @@ void showUrlEditor(int mActive, String[] mHosts, Transport.Mode[] mTransports, if (checked != null) mTransports[slot] = (Transport.Mode) checked.getTag(); View turned = turn.findViewById(turn.getCheckedRadioButtonId()); if (turned != null) mRotates[slot] = (Integer) turned.getTag(); + mMirrors[slot] = mirror.isChecked(); if (listener != null) listener.onSaveSettings(); // triggers save + reconnect }); builder.setNegativeButton("Cancel", (dialog, which) -> dialog.cancel()); diff --git a/android/app/src/main/java/com/openipc/decoder/QuadCell.java b/android/app/src/main/java/com/openipc/decoder/QuadCell.java index 46a9350..2135ad6 100644 --- a/android/app/src/main/java/com/openipc/decoder/QuadCell.java +++ b/android/app/src/main/java/com/openipc/decoder/QuadCell.java @@ -442,6 +442,11 @@ static boolean isKeyFrameNal(int type, boolean h265) { /** Degrees counter-clockwise the frames are turned (#200, #201), 0 for none. */ int rotate() { return session.rotate(); } void setSlotTurn(int rotate) { session.setSlotTurn(rotate); } + /** The slot's Mirror (#226). */ + boolean mirror() { return session.mirror(); } + /** Mirror is on, but the video decoder draws unmirrored (the GL path failed). */ + boolean mirrorFailed() { return !jpeg && decoder.mirrorFailed(); } + void setSlotMirror(boolean on) { session.setSlotMirror(on); } /** Transport in use, with the mode after a fallback (#112). */ String transportLabel() { return Transport.label(session.transportInUse(), transport); } @@ -698,6 +703,7 @@ private void decode(Frame buffer) { if (s == null || !s.isValid()) return; decoder.setSurface(s); decoder.setRotation(session.rotate()); + decoder.setMirror(session.mirror()); decoder.createDecoder(codecH265); // Feed the frame that triggered creation (often the SPS) to the new decoder decoder.decodeFrame(buffer, codecH265, 0); @@ -737,7 +743,7 @@ private void decodeJpeg(Frame frame) { return; } Surface s = surface; - int[] size = jpegRenderer.render(frame, s, session.rotate(), + int[] size = jpegRenderer.render(frame, s, session.rotate(), session.mirror(), () -> running && jpeg && s == surface); if (size == null) return; lastJpegDecodeMs = now; diff --git a/android/app/src/main/java/com/openipc/decoder/RtspClient.java b/android/app/src/main/java/com/openipc/decoder/RtspClient.java index eabc4c6..885fb0e 100644 --- a/android/app/src/main/java/com/openipc/decoder/RtspClient.java +++ b/android/app/src/main/java/com/openipc/decoder/RtspClient.java @@ -98,6 +98,9 @@ void configure(String host, String userAgent, Transport.Mode transport) { /** Degrees counter-clockwise the MJPEG frames are turned (#200), 0 for none. */ int rotate() { return session.rotate(); } void setSlotTurn(int rotate) { session.setSlotTurn(rotate); } + /** The slot's Mirror (#226). */ + boolean mirror() { return session.mirror(); } + void setSlotMirror(boolean on) { session.setSlotMirror(on); } void start() { nalAssembler.reset(); diff --git a/android/app/src/main/java/com/openipc/decoder/RtspSession.java b/android/app/src/main/java/com/openipc/decoder/RtspSession.java index fbd1789..1ae256c 100644 --- a/android/app/src/main/java/com/openipc/decoder/RtspSession.java +++ b/android/app/src/main/java/com/openipc/decoder/RtspSession.java @@ -710,6 +710,12 @@ int rotate() { return t >= 0 ? t : rotate; } + /** The slot's Mirror setting (#226): the picture flipped left↔right. */ + private volatile boolean slotMirror; + + boolean mirror() { return slotMirror; } + void setSlotMirror(boolean on) { slotMirror = on; } + /** The slot's own turn (#201): -1 leaves it to the URL or the toy's default. */ void setSlotTurn(int rotate) { int t = rotate == 0 || rotate == 90 || rotate == 180 || rotate == 270 ? rotate : -1; diff --git a/android/app/src/test/java/com/openipc/decoder/GlMirrorTest.java b/android/app/src/test/java/com/openipc/decoder/GlMirrorTest.java new file mode 100644 index 0000000..416d39e --- /dev/null +++ b/android/app/src/test/java/com/openipc/decoder/GlMirrorTest.java @@ -0,0 +1,47 @@ +/* + * Copyright (c) OpenIPC https://openipc.org Prosperity Public License 3.0.0 + * + * GlMirrorTest.java — where the mirrored and turned picture lands (#226) + * + */ + +package com.openipc.decoder; + +import static org.junit.Assert.assertArrayEquals; + +import org.junit.Test; + +public class GlMirrorTest { + private static final float E = 1e-6f; + + /** The picture's right edge centre (1, 0) and top edge centre (0, 1), as shown. */ + private static void lands(int turn, boolean mirror, float[] right, float[] top) { + float[] m = GlMirror.transform(turn, mirror); + assertArrayEquals(right, GlMirror.map(m, 1f, 0f), E); + assertArrayEquals(top, GlMirror.map(m, 0f, 1f), E); + } + + @Test + public void mirrorSwapsLeftAndRight() { + lands(0, false, new float[] {1f, 0f}, new float[] {0f, 1f}); + lands(0, true, new float[] {-1f, 0f}, new float[] {0f, 1f}); + } + + @Test + public void turnIsCounterClockwise() { + // 90° ↺: the right edge goes up, the top goes left + lands(90, false, new float[] {0f, 1f}, new float[] {-1f, 0f}); + lands(180, false, new float[] {-1f, 0f}, new float[] {0f, -1f}); + lands(270, false, new float[] {0f, -1f}, new float[] {1f, 0f}); + lands(-90, false, new float[] {0f, -1f}, new float[] {1f, 0f}); + } + + @Test + public void mirrorComesAfterTheTurn() { + // Turned first, then flipped on the screen: the top stays on the screen's top half + // for 0° and 180°, and goes right instead of left for 90° + lands(90, true, new float[] {0f, 1f}, new float[] {1f, 0f}); + lands(180, true, new float[] {1f, 0f}, new float[] {0f, -1f}); + lands(270, true, new float[] {0f, -1f}, new float[] {-1f, 0f}); + } +} diff --git a/docs/debug.md b/docs/debug.md index 083d0ef..0a6df3e 100644 --- a/docs/debug.md +++ b/docs/debug.md @@ -171,7 +171,7 @@ Single view and U view: every cell of the grid as the debug page shows it (3×3 | `codec`, `size` | when found | | `fps`, `kbps`, `key_frames` | measured during the check | -Toy drone slots ([toys.md](toys.md#flow-ufo)) add a `flowufo` object per cell and in `single` (`jieli` for `jieli://` slots, [toys.md](toys.md#hf--jieli-toys)), and a `heartbeat` object for RTSP slots that send the KY UFO heartbeat, and `rotate` (degrees counter-clockwise) where the picture is turned ([toys.md](toys.md#ky-ufo-rtsp-toys)). +Toy drone slots ([toys.md](toys.md#flow-ufo)) add a `flowufo` object per cell and in `single` (`jieli` for `jieli://` slots, [toys.md](toys.md#hf--jieli-toys)), and a `heartbeat` object for RTSP slots that send the KY UFO heartbeat, and `rotate` (degrees counter-clockwise) where the picture is turned ([toys.md](toys.md#ky-ufo-rtsp-toys)), and `mirror: true` where it is mirrored ([usage.md](usage.md)), with `mirror_failed: true` if the device could not mirror the video (it is shown unmirrored) or a mirrored frame could not be drawn. ## Log diff --git a/docs/usage.md b/docs/usage.md index 4622346..2447b96 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -17,7 +17,8 @@ 1. Open the menu: tap the screen, or press **OK** on a remote. 2. Choose a slot (**1**–**8**), then **Settings**. 3. Enter the camera URL, for example `rtsp://user:password@192.168.1.10:554/stream=0`, and choose the transport under it (**TCP / UDP / Auto / Multicast**, a new slot starts on Auto). URL formats: [transports.md](transports.md). - Below them, **Turn** (**Auto**, **0°**, **90° ↺**, **180°**, **270° ↺**, counter-clockwise) turns the picture of a camera mounted sideways or upside down ([#201](https://github.com/OpenIPC/decoder/issues/201)). **Auto** leaves it to the URL's `?rotate=` and the toys' defaults. It applies in the single view and in the grid, to H.264, H.265 and MJPEG, and to snapshots and screenshots; the decoder turns the picture (Android 6+; before that only MJPEG is turned). A mirror setting is not there yet: the video surface cannot be mirrored without a separate drawing path. + Below them, **Turn** (**Auto**, **0°**, **90° ↺**, **180°**, **270° ↺**, counter-clockwise) turns the picture of a camera mounted sideways or upside down ([#201](https://github.com/OpenIPC/decoder/issues/201)). **Auto** leaves it to the URL's `?rotate=` and the toys' defaults. It applies in the single view and in the grid, to H.264, H.265 and MJPEG, and to snapshots and screenshots; the decoder turns the picture (Android 6+; before that only MJPEG is turned). + **Mirror** flips the picture left to right, after the turn ([#226](https://github.com/OpenIPC/decoder/issues/226)), in the single view and in the grid, for H.264, H.265 and MJPEG, and in snapshots and screenshots. A mirrored H.264/H.265 slot is drawn with OpenGL ES instead of straight by the decoder (this path also turns it, on any Android version); a mirrored MJPEG slot is decoded in software. 4. **Save**. The app connects at once and reconnects by itself when the stream drops (exponential back-off, 1 s up to 30 s). The URL dialog is titled `Camera N URL`. The picture is fitted to the screen whatever its shape, without cropping. URLs are stored encrypted, see [security.md](security.md#credential-store). @@ -39,7 +40,7 @@ The menu is one grid: the eight slots on top, two rows of four below. Every word | Item | Action | |--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **1**–**8** | Switch to a camera slot (single view). The number of the active slot shows the connection quality: green under 100 ms of jitter, yellow 100–300 ms, red above. A grey number is an empty slot | -| **Settings** | URL, transport and turn of the active slot. Single view only, grey elsewhere | +| **Settings** | URL, transport, turn and mirror of the active slot. Single view only, grey elsewhere | | **WebUI** | The camera's web interface, see [WebUI](#webui). Single view only, grey elsewhere | | **Grid** | The view: [multi-view](#multi-view) (2×2 or 3×3) or the single camera. Does not change the playback mode | | **Slides** | The playback mode: every frame, or key frames only, see [Slides](#slides). Does not change the view |