docs: update docs

2025-04-21 03:55:04 +00:00 · 2020-01-16 18:33:08 +08:00 · 2020-01-16 18:33:08 +08:00 · 2db8a5710f
commit 2db8a5710f
parent 3e725e1675
5 changed files with 364 additions and 7 deletions
--- a/README.md
+++ b/README.md
@ -12,6 +12,17 @@ A single application can support up to 16 Android devices to connect at the same

 Supports three major desktop platforms, GNU/Linux, Windows and MacOS.

+It focuses on:
+
+ - **lightness** (native, displays only the device screen)
+ - **performance** (30~60fps)
+ - **quality** (1920×1080 or above)
+ - **low latency** ([35~70ms][lowlatency])
+ - **low startup time** (~1 second to display the first image)
+ - **non-intrusiveness** (nothing is left installed on the device)
+
+[lowlatency]: https://github.com/Genymobile/scrcpy/pull/646
+
 ![win](screenshot/win.png)

 ![mac](screenshot/mac.jpg)
@ -150,6 +161,15 @@ Real-time mouse and keyboard control Android device
 - Install apk: drag and drop apk to the video window to install
 - Transfer files: Drag files to the video window to send files to Android devices
 - Background recording: record only, no display interface
+- Copy-paste
+
+    It is possible to synchronize clipboards between the computer and the device, in
+    both directions:
+
+    - `Ctrl`+`c` copies the device clipboard to the computer clipboard;
+    - `Ctrl`+`Shift`+`v` copies the computer clipboard to the device clipboard;
+    - `Ctrl`+`v` _pastes_ the computer clipboard as a sequence of text events (but
+    breaks non-ASCII characters).

 ## TODO
 [TODO](docs/TODO.md)
@ -157,6 +177,9 @@ Real-time mouse and keyboard control Android device
 ## FAQ
 [FAQ](docs/FAQ.md)

+## DEVELOP
+[DEVELOP](docs/DEVELOP.md)
+
 ## Why develop QtScrcpy?
 There are several reasons for this, and the proportions are arranged from large to small:
 1. In the process of learning Qt, you need a project to combat
@ -176,9 +199,10 @@ Try to provide all the dependencies and make it easy to compile.

 ### Android (If you do not need to modify the requirements, you can use the built-in scrcpy-server directly)
 1. Set up an Android development environment on the target platform
-2. Open the server project in the project root directory using Android Studio
-3. Build it
-4. After compiling apk, rename it to scrcpy-server and replace third_party/scrcpy-server
+2. Open server project in project root with Android Studio
+3. The first time you open it, if you do not have the corresponding version of gradle, you will be prompted to find gradle, whether to upgrade gradle and create it. Select Cancel. After canceling, you will be prompted to select the location of the existing gradle. You can also cancel it (it will download automatically).
+4. Edit the code as needed, but of course you do n’t need to.
+4. After compiling the apk, rename it to scrcpy-server and replace third_party/scrcpy-server.

 ## Licence
 Since it is based on scrcpy, respect its Licence
--- a/README_zh.md
+++ b/README_zh.md
@ -12,6 +12,18 @@ QtScrcpy可以通过USB(或通过TCP/IP)连接Android设备，并进行显示和

 同时支持GNU/Linux，Windows和MacOS三大主流桌面平台

+它专注于:
+
+ - **精致** (仅显示设备屏幕)
+ - **性能** (30~60fps)
+ - **质量** (1920×1080以上)
+ - **低延迟** ([35~70ms][低延迟])
+ - **快速启动** (1s内就可以看到第一帧图像)
+ - **非侵入性** (不在设备上安装任何软件)
+
+[低延迟]: https://github.com/Genymobile/scrcpy/pull/646
+
+
 ![win](screenshot/win.png)

 ![mac](screenshot/mac.jpg)
@ -158,6 +170,12 @@ Mac OS平台，你可以直接使用我编译好的可执行程序:
 - 安装apk：拖拽apk到视频窗口即可安装
 - 传输文件：拖拽文件到视频窗口即可发送文件到Android设备
 - 后台录制：只录制，不显示界面
+- 复制粘贴
+
+    在计算机和设备之间双向同步剪贴板：
+    - `Ctrl` + `c`将设备剪贴板复制到计算机剪贴板；
+    - `Ctrl` + `Shift` + `v`将计算机剪贴板复制到设备剪贴板；
+    - `Ctrl` +`v` 将计算机剪贴板作为一系列文本事件发送到设备（不支持非ASCII字符）。

 ## TODO
 [后期计划](docs/TODO.md)
@ -165,6 +183,9 @@ Mac OS平台，你可以直接使用我编译好的可执行程序:
 ## FAQ
 [常见问题说明](docs/FAQ.md)

+## 开发者
+[开发者相关](docs/DEVELOP.md)
+
 ## 为什么开发QtScrcpy？
 综合起来有以下几个原因，比重从大到小排列：
 1. 学习Qt的过程中需要一个项目实战一下
@ -177,7 +198,7 @@ Mac OS平台，你可以直接使用我编译好的可执行程序:
 尽量提供了所有依赖资源，方便傻瓜式编译。

 ### PC端
-1. 目标平台上搭建Qt开发环境(Qt >= 5.9.7, vs >= 2015 (不支持mingw))
+1. 目标平台上搭建Qt开发环境(Qt >= 5.9.7, vs >= 2015 (**不支持mingw**))
 2. 克隆该项目
 3. 使用QtCreator打开项目根目录all.pro
 4. 编译，运行即可
--- a/config/config.ini
+++ b/config/config.ini
@ -1,7 +1,13 @@
 [common]
+# 窗口标题
 WindowTitle=QtScrcpy
+# 录制文件路径
 RecordPath=
+# 最大fps（仅支持Android 10以上）
 MaxFps=60
+# scrcpy-server的版本号（不要修改）
 ServerVersion=1.12.1
+# 是否显示手机皮肤，0不显示
 UseSkin=1
+# 视频解码方式：-1 自动，0 软解，1 dx硬解，2 opengl硬解
 UseDesktopOpenGL=-1
--- a/docs/DEVELOP.md
+++ b/docs/DEVELOP.md
@ -0,0 +1,300 @@
+# scrcpy for developers
+
+## Overview
+
+This application is composed of two parts:
+ - the server (`scrcpy-server`), to be executed on the device,
+ - the client (the `scrcpy` binary), executed on the host computer.
+
+The client is responsible to push the server to the device and start its
+execution.
+
+Once the client and the server are connected to each other, the server initially
+sends device information (name and initial screen dimensions), then starts to
+send a raw H.264 video stream of the device screen. The client decodes the video
+frames, and display them as soon as possible, without buffering, to minimize
+latency. The client is not aware of the device rotation (which is handled by the
+server), it just knows the dimensions of the video frames.
+
+The client captures relevant keyboard and mouse events, that it transmits to the
+server, which injects them to the device.
+
+
+
+## Server
+
+
+### Privileges
+
+Capturing the screen requires some privileges, which are granted to `shell`.
+
+The server is a Java application (with a [`public static void main(String...
+args)`][main] method), compiled against the Android framework, and executed as
+`shell` on the Android device.
+
+[main]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Server.java#L123
+
+To run such a Java application, the classes must be [_dexed_][dex] (typically,
+to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
+`classes.dex`, pushed to the device in `/data/local/tmp`, then it can be run
+with:
+
+    adb shell CLASSPATH=/data/local/tmp/classes.dex \
+        app_process / my.package.MainClass
+
+_The path `/data/local/tmp` is a good candidate to push the server, since it's
+readable and writable by `shell`, but not world-writable, so a malicious
+application may not replace the server just before the client executes it._
+
+Instead of a raw _dex_ file, `app_process` accepts a _jar_ containing
+`classes.dex` (e.g. an [APK]). For simplicity, and to benefit from the gradle
+build system, the server is built to an (unsigned) APK (renamed to
+`scrcpy-server`).
+
+[dex]: https://en.wikipedia.org/wiki/Dalvik_(software)
+[apk]: https://en.wikipedia.org/wiki/Android_application_package
+
+
+### Hidden methods
+
+Although compiled against the Android framework, [hidden] methods and classes are
+not directly accessible (and they may differ from one Android version to
+another).
+
+They can be called using reflection though. The communication with hidden
+components is provided by [_wrappers_ classes][wrappers] and [aidl].
+
+[hidden]: https://stackoverflow.com/a/31908373/1987178
+[wrappers]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers
+[aidl]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/aidl/android/view
+
+
+### Threading
+
+The server uses 3 threads:
+
+ - the **main** thread, encoding and streaming the video to the client;
+ - the **controller** thread, listening for _control messages_ (typically,
+   keyboard and mouse events) from the client;
+ - the **receiver** thread (managed by the controller), sending _device messges_
+   to the clients (currently, it is only used to send the device clipboard
+   content).
+
+Since the video encoding is typically hardware, there would be no benefit in
+encoding and streaming in two different threads.
+
+
+### Screen video encoding
+
+The encoding is managed by [`ScreenEncoder`].
+
+The video is encoded using the [`MediaCodec`] API. The codec takes its input
+from a [surface] associated to the display, and writes the resulting H.264
+stream to the provided output stream (the socket connected to the client).
+
+[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
+[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
+[surface]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L68-L69
+
+On device [rotation], the codec, surface and display are reinitialized, and a
+new video stream is produced.
+
+New frames are produced only when changes occur on the surface. This is good
+because it avoids to send unnecessary frames, but there are drawbacks:
+
+ - it does not send any frame on start if the device screen does not change,
+ - after fast motion changes, the last frame may have poor quality.
+
+Both problems are [solved][repeat] by the flag
+[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
+
+[rotation]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90
+[repeat]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
+[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
+
+
+### Input events injection
+
+_Control messages_ are received from the client by the [`Controller`] (run in a
+separate thread). There are several types of input events:
+ - keycode (cf [`KeyEvent`]),
+ - text (special characters may not be handled by keycodes directly),
+ - mouse motion/click,
+ - mouse scroll,
+ - other commands (e.g. to switch the screen on or to copy the clipboard).
+
+Some of them need to inject input events to the system. To do so, they use the
+_hidden_ method [`InputManager.injectInputEvent`] (exposed by our
+[`InputManager` wrapper][inject-wrapper]).
+
+[`Controller`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Controller.java#L81
+[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
+[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
+[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
+[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
+
+
+
+## Client
+
+The client relies on [SDL], which provides cross-platform API for UI, input
+events, threading, etc.
+
+The video stream is decoded by [libav] (FFmpeg).
+
+[SDL]: https://www.libsdl.org
+[libav]: https://www.libav.org/
+
+### Initialization
+
+On startup, in addition to _libav_ and _SDL_ initialization, the client must
+push and start the server on the device, and open two sockets (one for the video
+stream, one for control) so that they may communicate.
+
+Note that the client-server roles are expressed at the application level:
+
+ - the server _serves_ video stream and handle requests from the client,
+ - the client _controls_ the device through the server.
+
+However, the roles are reversed at the network level:
+
+ - the client opens a server socket and listen on a port before starting the
+   server,
+ - the server connects to the client.
+
+This role inversion guarantees that the connection will not fail due to race
+conditions, and avoids polling.
+
+_(Note that over TCP/IP, the roles are not reversed, due to a bug in `adb
+reverse`. See commit [1038bad] and [issue #5].)_
+
+Once the server is connected, it sends the device information (name and initial
+screen dimensions). Thus, the client may init the window and renderer, before
+the first frame is available.
+
+To minimize startup time, SDL initialization is performed while listening for
+the connection from the server (see commit [90a46b4]).
+
+[1038bad]: https://github.com/Genymobile/scrcpy/commit/1038bad3850f18717a048a4d5c0f8110e54ee172
+[issue #5]: https://github.com/Genymobile/scrcpy/issues/5
+[90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e
+
+
+### Threading
+
+The client uses 4 threads:
+
+ - the **main** thread, executing the SDL event loop,
+ - the **stream** thread, receiving the video and used for decoding and
+   recording,
+ - the **controller** thread, sending _control messages_ to the server,
+ - the **receiver** thread (managed by the controller), receiving _device
+   messages_ from the client.
+
+In addition, another thread can be started if necessary to handle APK
+installation or file push requests (via drag&drop on the main window) or to
+print the framerate regularly in the console.
+
+
+
+### Stream
+
+The video [stream] is received from the socket (connected to the server on the
+device) in a separate thread.
+
+If a [decoder] is present (i.e. `--no-display` is not set), then it uses _libav_
+to decode the H.264 stream from the socket, and notifies the main thread when a
+new frame is available.
+
+There are two [frames][video_buffer] simultaneously in memory:
+ - the **decoding** frame, written by the decoder from the decoder thread,
+ - the **rendering** frame, rendered in a texture from the main thread.
+
+When a new decoded frame is available, the decoder _swaps_ the decoding and
+rendering frame (with proper synchronization). Thus, it immediatly starts
+to decode a new frame while the main thread renders the last one.
+
+If a [recorder] is present (i.e. `--record` is enabled), then its muxes the raw
+H.264 packet to the output video file.
+
+[stream]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/stream.h
+[decoder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/decoder.h
+[video_buffer]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/video_buffer.h
+[recorder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/recorder.h
+
+```
+                                   +----------+      +----------+
+                              ---> | decoder  | ---> |  screen  |
+             +---------+     /     +----------+      +----------+
+ socket ---> | stream  | ----
+             +---------+     \     +----------+
+                              ---> | recorder |
+                                   +----------+
+```
+
+### Controller
+
+The [controller] is responsible to send _control messages_ to the device. It
+runs in a separate thread, to avoid I/O on the main thread.
+
+On SDL event, received on the main thread, the [input manager][inputmanager]
+creates appropriate [_control messages_][controlmsg]. It is responsible to
+convert SDL events to Android events (using [convert]). It pushes the _control
+messages_ to a queue hold by the controller. On its own thread, the controller
+takes messages from the queue, that it serializes and sends to the client.
+
+[controller]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/controller.h
+[controlmsg]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/control_msg.h
+[inputmanager]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/input_manager.h
+[convert]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/convert.h
+
+
+### UI and event loop
+
+Initialization, input events and rendering are all [managed][scrcpy] in the main
+thread.
+
+Events are handled in the [event loop], which either updates the [screen] or
+delegates to the [input manager][inputmanager].
+
+[scrcpy]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c
+[event loop]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c#L201
+[screen]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/screen.h
+
+
+## Hack
+
+For more details, go read the code!
+
+If you find a bug, or have an awesome idea to implement, please discuss and
+contribute ;-)
+
+
+### Debug the server
+
+The server is pushed to the device by the client on startup.
+
+To debug it, enable the server debugger during configuration:
+
+```bash
+meson x -Dserver_debugger=true
+# or, if x is already configured
+meson configure x -Dserver_debugger=true
+```
+
+Then recompile.
+
+When you start scrcpy, it will start a debugger on port 5005 on the device.
+Redirect that port to the computer:
+
+```bash
+adb forward tcp:5005 tcp:5005
+```
+
+In Android Studio, _Run_ > _Debug_ > _Edit configurations..._ On the left, click on
+`+`, _Remote_, and fill the form:
+
+ - Host: `localhost`
+ - Port: `5005`
+
+Then click on _Debug_.
--- a/docs/FAQ.md
+++ b/docs/FAQ.md
@ -1,9 +1,15 @@
-# 可以看到画面，但无法控制
+# Frequently Asked Questions
+一些经常问的问题

-## 小米手机
-检查是否USB调试里打开了允许模拟点击
+如果在此文档没有解决你的问题，描述你的问题，截图软件控制台中打印的日志，一起发到QQ群里提问。
+
+## 可以看到画面，但无法控制
+有些手机(小米等手机)需要额外打开控制权限，检查是否USB调试里打开了允许模拟点击

 ![image](image/USB调试(安全设置).jpg)

+## 错误信息Could not open video stream
+导致这个错误的原因有很多，最简单的解决方法是在分辨率设置中，选择一个较低的分辨率
+
 ## 声音
 [关于转发安卓声音到PC的讨论](https://github.com/Genymobile/scrcpy/issues/14#issuecomment-543204526)