Skip to content

Device Mirror experimental ​

The Device Mirror shows the live screen of an Android emulator or device, an iOS simulator, or a physical iPhone inside the host window. On Android devices and iOS simulators you can tap, swipe, type and press hardware buttons on the mirrored screen; a physical iPhone takes input too once you set your development team, which is experimental. Every device can take screenshots and recordings, kept per device so the ones from a test run are easy to find again. An AI agent can do the same over MCP.

It needs no app: nothing is added to the app you debug, and it is listed at the top of the sidebar, above the app picker, usable as soon as the host starts with nothing connected — see Plugins that need no app.

The Device Mirror showing a Pixel 9 emulator's screen, with the device picker and the device's buttons above it and the text field belowThe Device Mirror showing a Pixel 9 emulator's screen, with the device picker and the device's buttons above it and the text field below

Setup ​

Install the host plugin ​

Install Device Mirror from Settings → Plugins → Add Plugins → Official Plugins. To install it by Maven coordinates instead, use com.kitakkun.jetwhale:jetwhale-device-mirror:<version>, released with the host under the host's version.

Tools on your machine ​

The plugin drives the command-line tools you already use for devices, and looks for them where a terminal would find them (see How the tools are found). When a tool is missing, the device list says which one and what it would enable.

To mirrorYou need
Android emulators and devicesadb from the Android SDK platform-tools
iOS simulators (macOS)Xcode alone: simctl for screenshots and recordings, and an XCTest runner for live video and input — see Input on iOS
iPhones and iPads connected by USB (macOS)Xcode alone for live video, screenshots and recordings, with the iPhone set up as in iPhones over USB; for input, also iproxy (brew install libimobiledevice) and a development team — see Input on iOS
Live video from Android devices and iPhonesffmpeg: brew install ffmpeg, winget install ffmpeg or apt install ffmpeg

Android devices and iPhones send their screen as H.264, which the plugin decodes with the ffmpeg command. Without it an Android device is shown through screenshots a few times a second, and an iPhone cannot be mirrored. iOS simulators, and Android emulators that expose their own gRPC screen stream, need no ffmpeg; an emulator without that stream is decoded like a device, and so is a foldable emulator folded through adb shell cmd device_state state.

iPhones over USB ​

The plugin reads a physical iPhone's screen the way QuickTime Player does: macOS offers the screen of an iPhone connected by USB as a camera. For its live view, screenshots and recordings, the iPhone must be:

  • Connected by a cable. An iPhone paired only over Wi-Fi is not listed.
  • Trusting this Mac. Answer Trust on the iPhone when you first connect it.
  • Unlocked, with its screen on.

The first time you open an iPhone, the plugin builds a small capture helper with Xcode's Swift compiler, which takes a few seconds and happens again only when Xcode or the helper changes. macOS then asks whether JetWhale Debugger may access the camera; allow it. You can change the answer later in System Settings → Privacy & Security → Camera. When the host runs from a terminal or an IDE, macOS asks on behalf of that app instead. If the mirror says that macOS stopped the capture helper, reinstall JetWhale Debugger from the latest release: an older install does not declare that it uses the camera, so macOS cannot ask.

While the plugin reads its screen, the iPhone shows a 9:41 status bar and plays its sound on this Mac, as it does for QuickTime Player. Both return to normal about 30 seconds after nothing shows the iPhone any more.

Input needs more: a development team, iproxy and Developer Mode — see Input on a physical iPhone.

How the tools are found ​

An app started from Finder, the Dock or a desktop entry does not inherit your shell's PATH. So on macOS and Linux the plugin looks for each tool in these places, in order:

  1. For adb only, the Android SDK: ANDROID_HOME, ANDROID_SDK_ROOT, then the SDK's default location.
  2. The host's own PATH.
  3. The directories these tools are usually installed in, whether or not a PATH lists them:
    • /opt/homebrew/bin and /usr/local/bin, Homebrew's on Apple silicon and on Intel Macs: ffmpeg, iproxy from libimobiledevice, and adb from the android-platform-tools cask
    • ~/.local/bin, where you can link a tool that only your login shell finds, as shown below
  4. The PATH your login shell sets up, only when a tool is still missing after these.

On Windows an app gets your PATH however it is started, so the plugin uses it as it is.

To read the login shell's PATH, the plugin runs your shell ($SHELL, or /bin/zsh on macOS and /bin/sh elsewhere when it is unset) as an interactive login shell, as a terminal does. It does so at most once per run of the host, the first time it looks for devices; the host does the same for its own adb. The shell runs your startup files, .zshrc and .bashrc included, so whatever they start runs again. When the shell has not answered within 5 seconds, exits with an error or prints no PATH, the plugin goes on with what it found before.

The tools the plugin starts get the directories it searched as their PATH, so a tool that looks for others on PATH finds them too, as an asdf shim finds the program it stands for.

If your startup files live in a folder macOS protects, such as Documents, Desktop or iCloud Drive — for example when .zshrc is a link to a file kept in Documents — macOS asks whether JetWhale Debugger may access that folder when the shell starts. The shell waits for the answer, so an answer given after 5 seconds takes effect from the next run of the host. If you choose Don't Allow, the shell cannot read those files, and the tools they put on PATH, through asdf for example, are not found. Either make the tool reachable from one of the directories in step 3, for example by linking it in a terminal:

shell
mkdir -p ~/.local/bin
ln -s "$(command -v ffmpeg)" ~/.local/bin/ffmpeg

or allow the access later under System Settings → Privacy & Security → Files & Folders. Then restart the host.

If a tool is reported missing although it is installed, open a new terminal and check that command -v <tool> prints its path. If it prints nothing, add the tool's directory to PATH in your shell's startup files. If it prints a path, link it into ~/.local/bin as above, or check that your startup files finish within 5 seconds without asking for input. Then restart the host, which looks for the tools again.

Using it ​

Devices and the grid ​

The device's name at the top left opens the device picker: every device, Android and iOS in two groups, with its kind, its OS version (iOS only) and a dot for what the mirror last saw — showing its screen, screen off, unavailable, or (hollow) not watched yet. A device appears once it is booted or connected.

The picker's first entry, All devices, opens the grid: every device as a tile, at its own shape, with a screenshot refreshed every 1.5 seconds and how long ago it was taken. Only tiles on screen are captured, two at a time at most, and nothing streams meanwhile. Click a tile, or press Enter on it, to open that device; hovering one offers Open and Screenshot.

The device grid: a Pixel 9 emulator and an iPhone 16 simulator showing their screens, and a Pixel 7 whose screen is offThe device grid: a Pixel 9 emulator and an iPhone 16 simulator showing their screens, and a Pixel 7 whose screen is off

The grid's toolbar acts on every device at once. The camera button saves one screenshot per device. The record button starts recording every device that can record; while any device records it turns into Stop all, with a count of the devices recording. Until you tick Don't show again, it asks before starting, because recording several devices at once is heavy on this machine — iOS simulators most, since they encode their video on the Mac.

Live view and input ​

The mirrored screen fills the pane at its own aspect ratio. A click is a tap and a drag is a swipe, at the matching point on the device, and the field under the screen types into whatever has focus. When an Android device rotates or a foldable folds, the view follows within a couple of seconds. Switching back to a device shows its last frame at once, dimmed, until its stream reconnects.

When no live video is available, the mirror falls back to screenshots and says why above the text field; it tries the video again after 5 seconds, then 15, then once a minute. An iOS simulator also shows screenshots while its XCTest runner starts, and switches to the runner's video once it is up.

The toolbar's buttons are grouped — navigation, volume, screen power, captures — and wrap as a whole in a narrow window. Android has Home, Back, Recent apps, Power, Volume up and Volume down; a simulator that takes input has Home, Recent apps and Power; an iPhone that takes input has Home, Power, Volume up and Volume down. On Android, Screen off and Wake turn the screen off and on, and Wake lifts a lock screen that has no PIN, pattern or password. Stats under the screen shows frames received and shown per second, the longest gap between frames, and how long decoding, copying and drawing take.

Input on iOS ​

Taps, swipes, text and buttons reach an iOS simulator or iPhone through an XCTest runner: a small UI-test bundle that Xcode builds on this Mac, installs on the device and keeps running while it sends the input. It is the way WebDriverAgent drives iOS, and it reaches the whole screen, the home screen and system alerts included. A simulator's live video comes from the same runner, about 30 frames a second. It runs on iOS 17 or later, the oldest that Xcode 27's XCTest runs on.

  • The first use builds it, which takes ten to twenty seconds once per Xcode version; after that a runner starts in about three seconds. The mirror starts it as soon as it shows the device, so the first tap seldom waits; on a simulator, screenshots stand in for the video meanwhile.
  • One runner per device, shared with any other plugin that drives iOS. It stops by itself after five minutes without input while nothing streams its screen, unless an agent holds it with keepRunnerAlive, and when the host quits. Its build and state live under the host's app data, in xctest-runner/.
  • On a simulator, the runner is the only way in. When it cannot be built or started, or the simulator runs an iOS older than 17, the simulator takes no input and is shown through screenshots, and the mirror says why. Recent apps opens the app switcher with the swipe up from the bottom edge that a Face ID iPhone takes.
  • In landscape, a simulator's live view and screenshots turn with it, and a click, or the MCP tap and swipe, takes their pixels; the runner turns them into the simulator's own portrait points.

Input on a physical iPhone experimental ​

Driving a physical iPhone follows Apple's and Appium's documentation for running an XCTest runner on a device, and has not been tried on one yet. It needs:

  1. Your development team, which signs the runner. Choose Set team… in the banner under the iPhone's screen and enter the ten-character team ID shown under Membership in your Apple Developer account. Xcode must be signed in to that team (Xcode → Settings → Accounts); a free personal team works.
  2. iproxy, which forwards the runner's port over USB: brew install libimobiledevice.
  3. On the iPhone: trust this Mac, turn on Developer Mode (Settings → Privacy & Security → Developer Mode, then restart), turn on Enable UI Automation (Settings → Developer), and keep it unlocked while it is driven. Signing also needs the iPhone online.

The banner under an iPhone says what is missing. A failure that only shows when the runner starts — signing, Developer Mode, UI Automation, a locked iPhone — comes back as a notice when you first tap.

Captures ​

Screenshot and Record in the toolbar save into the device's captures; while a recording runs, Record turns into a red counter, and clicking it stops the recording. Each device records on its own, so several can record at once. The notice that names a saved file offers Open, which shows it in the Captures panel, and Copy.

Every capture is kept in a folder for its device, with one folder per day:

text
captures/
  Pixel-9-3fa2c1d0/
    2026-09-25/
      123005-screenshot.png
      123005-screenshot.png.json
      123412-recording.mp4
      123412-recording.mp4.json
  iPhone-17-9b04e7aa/
    …

The short code after the device name comes from its serial number or UDID, so a device keeps its folder across sessions and two devices with the same name get separate folders. The .json beside each capture records the device's id, name, platform, kind and iOS version, the capture's size in pixels, when it was taken, and a recording's length. The list is rebuilt from these files, so captures from earlier sessions stay listed, and one deleted in Finder drops off.

The folder defaults to ~/.jetwhale/plugin-data/com.kitakkun.jetwhale.mirror/captures; Change folder… in the Captures panel picks another, and the host remembers it.

Captures in the toolbar opens the panel beside the live view:

  • This device / All devices and All kinds / Screenshot / Recording filter the list, and the date tags narrow it to one day. Captures are grouped by day, newest first.
  • Selecting a capture shows its details. Open opens it in its default app, Reveal shows it in Finder or Explorer, and Delete… removes it after you confirm. Copy image puts a screenshot on the clipboard as an image and as its file; Copy file puts a recording there as its file, which Finder, chat apps and upload fields accept. Copy path copies its absolute path.
  • Open folder opens the device's folder, or the captures folder when all devices are shown.

The Captures panel beside the live view: the filters, the day's captures, and the selected screenshot's details and actionsThe Captures panel beside the live view: the filters, the day's captures, and the selected screenshot's details and actions

MCP tools ​

With the MCP server running, an AI agent can do the same through these tools. They live in the host session, which jetwhale.listSessions always lists first, so pass host as their sessionId. The deviceId can be left out to use the device selected in the mirror.

ToolWhat it does
com.kitakkun.jetwhale.mirror.listDevicesThe devices, with their ids, platform, kind, and what they accept: input, buttons, recording. A device that takes no input says why in inputUnavailableReason. An iOS device or simulator also reports osVersion, and runnerKeptAliveUntil while a lease holds its runner; an Android device reports screenOn and locked
com.kitakkun.jetwhale.mirror.captureScreenshotSaves a screenshot among the device's captures; returns its path and size in pixels
com.kitakkun.jetwhale.mirror.tapTaps at a point, in the pixels of a screenshot
com.kitakkun.jetwhale.mirror.swipeSwipes between two points over a duration
com.kitakkun.jetwhale.mirror.pressButtonPresses a hardware button the device has
com.kitakkun.jetwhale.mirror.inputTextTypes text into the focused field
com.kitakkun.jetwhale.mirror.keepRunnerAliveKeeps an iOS device's XCTest runner from stopping after five idle minutes, for the given minutes, at most 120; returns at once, with the time the lease ends. Call it again to renew the lease, or with 0 to end it. A lease nobody renews ends on its own
com.kitakkun.jetwhale.mirror.setScreenTurns an Android device's screen on (on: true, also lifting a lock screen without a credential) or off; returns screenOn and locked afterwards, where locked: true means the device still needs unlocking
com.kitakkun.jetwhale.mirror.startRecordingStarts recording one device's screen, several devices' (deviceIds), or every device that can record (all: true)
com.kitakkun.jetwhale.mirror.stopRecordingStops one recording, several (deviceIds), or all of them (all: true); returns each video's path and length
com.kitakkun.jetwhale.mirror.listCapturesSaved captures, newest first, optionally only one deviceId, one kind (Screenshot or Recording), or those taken at or after since (epoch milliseconds)

An agent can read a returned path to look at the capture.

While an agent runs a tool, the single view rings the control that does the same: the device picker for listDevices, the live view for a tap or a swipe, the hardware buttons for a press, the text field for typed text, the screen button for setScreen, the record and screenshot buttons for captures, and the Captures panel's list for listCaptures. The ring marks the control, not the device: a tool that names another device rings the same control.

Limits ​

  • Input on a physical iPhone is experimental and needs a development team, iproxy and the iPhone settings in Input on a physical iPhone. Without them it is view-only, and MCP refuses input with the reason. Screenshots and recordings, taken from the capture of its screen, work either way, and need ffmpeg.
  • If an iPhone stays black, unlock it and keep its screen on, and check that it is connected by USB and trusts this Mac; the mirror lists the same hints. A missing Camera permission is reported as such: see iPhones over USB.
  • A simulator's volume buttons are shown disabled: XCTest cannot press them on a simulator.
  • Android stops a recording on its own after 180 seconds.
  • On Android, text with a line break is refused; type each line separately.

Released under the Apache License 2.0.