feat(mcp): simplify MCP tool architecture and unify platform tools #1507

quanru · 2025-11-25T12:25:17Z

Summary

This PR refactors the MCP packages to use a simplified, consistent tool architecture across all platforms (Web, Android, iOS).

Key Changes

🏗️ Shared Infrastructure (`@midscene/shared/mcp`)

Created BaseMCPServer and BaseMidsceneTools base classes for code reuse
Implemented dynamic tool generation from agent.getActionSpace()
Simplified common tools to only take_screenshot and wait_for (removed assert)
Added unified tool generation pattern in tool-generator.ts

📦 Platform-Specific Packages

@midscene/web-mcp (Renamed from @midscene/mcp): Web automation via Puppeteer & Bridge mode
@midscene/android-mcp (New): Android-specific MCP server with android_connect tool
@midscene/ios-mcp (New): iOS-specific MCP server with ios_connect tool

🎯 Simplified Tool Architecture

Each platform now provides exactly 3 types of tools:

ActionSpace tools (dynamic): Generated from agent.getActionSpace()
- Platform-specific interaction capabilities (Tap, Input, Scroll, etc.)
Common tools (2 tools):
- take_screenshot: Capture current page/screen
- wait_for: Wait until condition becomes true
Platform connection (1 tool):
- web_connect: Connect to web page (Puppeteer) or browser (Bridge mode)
- android_connect: Connect to Android device (optional deviceId and uri)
- ios_connect: Connect to iOS device (optional uri)

🔧 Recent Improvements

Bridge Mode Enhancement (d56be2d):

Improved bridge mode device initialization in base-tools.ts
Enhanced wait_for error handling with try-catch blocks
Better connection lifecycle management

Build & Type Fixes (0d4a373):

Fixed invalid planType parameter in tool-generator.ts
Added missing isError field to wait_for tool response
Removed unused @midscene/android dependency from web-mcp
Added externals configuration to android-mcp and ios-mcp rslib configs
Unified version handling using __VERSION__ constant across all MCP packages

🧪 Testing & Quality

All MCP package tests passing (web-mcp: 22/22, android-mcp, ios-mcp)
Build and lint checks passing
Fixed test organization (moved from @midscene/mcp to @midscene/web-mcp)

Migration Notes

Tools have been consolidated:

~~open_new~~ → web_connect
~~android_list_devices~~ → Removed (use android_connect without deviceId)
~~android_launch~~ → Merged into android_connect (use uri parameter)
~~ios_check_environment~~ → Removed (just use ios_connect)
~~assert~~ → Removed from common tools

Package rename:

@midscene/mcp → @midscene/web-mcp

Related Issues

Part of MCP architecture refactoring to support multiple platforms with consistent patterns.
Addresses GitHub Copilot review comments for improved code quality and type safety.

🤖 Generated with Claude Code

netlify · 2025-11-25T12:25:23Z

✅ Deploy Preview for midscene ready!

Name	Link
🔨 Latest commit	`69b9d84`
🔍 Latest deploy log	https://app.netlify.com/projects/midscene/deploys/6932ca4ac5d64e0008542477
😎 Deploy Preview	https://deploy-preview-1507--midscene.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot

Pull request overview

This PR refactors the MCP (Model Context Protocol) server architecture to use a unified, dynamic tool generation pattern across all platforms (Web, Android, iOS), significantly reducing code duplication and improving maintainability.

Key Changes

Introduced shared base classes (BaseMCPServer and BaseMidsceneTools) in @midscene/shared/mcp to provide common infrastructure for all platform MCP servers
Implemented dynamic tool generation from agent.getActionSpace() instead of hardcoded tool definitions, making the architecture more flexible
Created platform-specific MCP packages: @midscene/android-mcp for Android automation and @midscene/ios-mcp for iOS automation, plus @midscene/web-mcp as an alias for the existing @midscene/mcp

Reviewed changes

Copilot reviewed 38 out of 39 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
packages/shared/src/mcp/base-server.ts	New base MCP server class providing common launch() API and lifecycle management
packages/shared/src/mcp/base-tools.ts	New base tools manager with dynamic tool initialization from action space
packages/shared/src/mcp/tool-generator.ts	Core tool generation logic converting action space to MCP tool definitions
packages/shared/src/mcp/types.ts	Type definitions for tool architecture
packages/shared/src/mcp/index.ts	Export aggregation for shared MCP infrastructure
packages/shared/package.json	Added MCP module export path and SDK dependency
packages/mcp/src/server.ts	Refactored Web MCP server to extend BaseMCPServer
packages/mcp/src/web-tools.ts	New Web-specific tools manager extending BaseMidsceneTools
packages/mcp/src/index.ts	Simplified CLI entry to use new server architecture
packages/mcp/src/tools.ts	Removed (replaced by dynamic tool generation)
packages/mcp/src/midscene.ts	Removed (logic moved to base classes and web-tools)
packages/mcp/package.json	Added exports field for server module
packages/mcp/tests/index.test.ts	Removed obsolete tool snapshot tests
packages/android-mcp/*	New Android MCP server package with android_connect tool
packages/ios-mcp/*	New iOS MCP server package with ios_connect tool
packages/web-mcp/*	New alias package re-exporting @midscene/mcp for naming consistency
pnpm-lock.yaml	Updated dependencies for new packages
package.json	Added new packages to test script

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (4)

packages/android-mcp/rslib.config.ts:1

Unused import path.

import path from 'node:path';

packages/ios-mcp/rslib.config.ts:1

Unused import path.

import path from 'node:path';

packages/ios-mcp/src/ios-tools.ts:5

Unused import z.

import { z } from 'zod';

packages/shared/src/mcp/tool-generator.ts:29

Unused variable result.

      const result = await agent.action(`Use the action "${action.name}"`, {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/ios-mcp/src/ios-tools.ts

packages/android-mcp/README.md

packages/shared/src/mcp/tool-generator.ts

packages/web-mcp/src/index.ts

packages/android-mcp/rslib.config.ts

packages/shared/src/mcp/tool-generator.ts

packages/ios-mcp/README.md

packages/android-mcp/README.md

packages/shared/src/mcp/base-tools.ts

Copilot

Pull request overview

Copilot reviewed 39 out of 40 changed files in this pull request and generated 7 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/shared/src/mcp/base-tools.ts

packages/shared/src/mcp/tool-generator.ts

packages/android-mcp/src/android-tools.ts

packages/mcp/src/web-tools.ts

packages/android-mcp/src/android-tools.ts

packages/ios-mcp/rslib.config.ts

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 41 out of 45 changed files in this pull request and generated 6 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (1)

packages/mcp/tests/index.test.ts:7

The test file imports deepMerge, getChromePathFromEnv, and getSystemChromePath from ../src/utils, but this file no longer exists in the packages/mcp package. These utilities have been moved to packages/web-mcp/src/utils.ts as part of the refactoring.

Since @midscene/mcp is now just a wrapper for @midscene/web-mcp, these tests should either be:

Moved to the packages/web-mcp package where the actual implementation now lives, or
Updated to import from @midscene/web-mcp if testing the wrapper functionality

The current import path will cause test failures.

import {
  deepMerge,
  getChromePathFromEnv,
  getSystemChromePath,
} from '../src/utils';

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/web-mcp/src/web-tools.ts

packages/ios-mcp/src/server.ts

packages/ios-mcp/rslib.config.ts

packages/web-mcp/package.json

packages/android-mcp/src/server.ts

packages/android-mcp/rslib.config.ts

- Fix invalid planType parameter in tool-generator.ts - Add missing isError field to wait_for tool - Remove unused @midscene/android dependency from web-mcp - Add externals configuration to android-mcp and ios-mcp - Unify version handling using __VERSION__ constant - Improve error handling in base-tools.ts wait_for Resolves 8 review comments from GitHub Copilot 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

…Web tools; enhance action space retrieval

… and Web tools; streamline temporary device creation

…ers; enhance command line argument parsing

…gic for Android, iOS, and Web MCP servers

… MCP server and tools

… for unified HTTP support and argument parsing

- Improve error handling in base-tools.ts for bridge mode initialization - Add specific detection for bridge mode URL requirement errors - Provide friendly logging messages instead of generic failure messages - Clarify that deferred initialization is expected behavior - Improve comments in web-tools.ts - Remove misleading "deferred until first tool use" comment - Add clear explanations for bridge vs puppeteer mode behavior - Clarify that bridge mode requires URL while puppeteer can auto-start - Move tests from @midscene/mcp to @midscene/web-mcp - Relocate utils and puppeteer tests to correct package location - Update mcp package.json to remove obsolete test scripts - All 22 tests passing in new location 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

- Fix invalid planType parameter in tool-generator.ts - Add missing isError field to wait_for tool - Remove unused @midscene/android dependency from web-mcp - Add externals configuration to android-mcp and ios-mcp - Unify version handling using __VERSION__ constant - Improve error handling in base-tools.ts wait_for Resolves 8 review comments from GitHub Copilot 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Type Safety & Documentation: - Add aiAction and aiWaitFor optional methods to BaseAgent interface - Add comprehensive JSDoc for ensureAgent abstract method - Remove complex type assertions in favor of optional method calls Code Organization: - Extract magic numbers to named constants (defaultAppLoadingTimeoutMs, defaultAppLoadingCheckIntervalMs) - Add buildScreenshotContent helper method to reduce code duplication - Unify method naming: initAgentByBridgeMode → initBridgeModeAgent Error Handling & Logging: - Add debug logging to all catch blocks instead of silent failures - Replace error variable 'e' with descriptive 'error' Code Style: - Use optional chaining (?.) for cleaner agent.destroy calls - Simplify verbose comments (require usage explanations) - Improve comment clarity for temporary device creation All changes follow conventional commits and maintain backward compatibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

…kage Extract Constants: - Move defaultAppLoadingTimeoutMs (10000) to @midscene/shared/mcp/types - Move defaultAppLoadingCheckIntervalMs (2000) to @midscene/shared/mcp/types - Remove duplicate constant definitions from android-mcp and ios-mcp Type Safety: - Add BaseAgent import to android-tools.ts - Use double type assertion (as unknown as) for agent type conversion - Fix spelling: temp-for-actionspace → temp-for-action-space This follows DRY principle by maintaining constants in a single location and improves type safety with proper imports and assertions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

…rver and tools

…ainability

…ages

…and retry logic

Copilot

Pull request overview

Copilot reviewed 60 out of 63 changed files in this pull request and generated 5 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-04T11:38:05Z

package.json

-    "nx": "21.1.2",
-    "prettier": "^3.7.4",
+    "nx": "22.1.3",
+    "prettier": "^3.6.2",


The prettier version specifier in package.json has been downgraded from ^3.7.4 to ^3.6.2, but the lock file shows version 3.7.4 is still being used. This mismatch could cause confusion. Either update the specifier to match the actual version used (^3.7.4) or ensure the lock file reflects the intended version (^3.6.2).

Suggested change

"prettier": "^3.6.2",

"prettier": "^3.7.4",

Copilot · 2025-12-04T11:38:06Z

packages/web-bridge-mcp/src/web-tools.ts

+    if (this.agent && openNewTabWithUrl) {
+      try {
+        await this.agent?.destroy?.();
+      } catch (error) {
+        console.debug('Failed to destroy agent during re-init:', error);
+      }


The error handling in ensureAgent uses try-catch to suppress errors during agent destruction, but the error is only logged via debug(). Consider using console.error() or a more visible logging mechanism for destruction failures, as these could indicate resource leaks or other serious issues that should not be silently ignored.

Copilot · 2025-12-04T11:38:06Z

packages/web-bridge-mcp/tests/http-server.test.ts

+    // Close the HTTP server
+    if (httpServer) {
+      await new Promise<void>((resolve) => {
+        httpServer!.close(() => resolve());


The base expression of this property access is always null or undefined.

Copilot · 2025-12-04T11:38:06Z

packages/web-bridge-mcp/tests/http-server.test.ts

+
+    try {
+      // Start server in background
+      const serverPromise = server.launchHttp({


Unused variable serverPromise.

Copilot · 2025-12-04T11:38:06Z

packages/web-bridge-mcp/tests/http-server.test.ts

+      const conflictServer = new WebMCPServer();
+
+      // This should fail because the port is already in use
+      const launchPromise = conflictServer.launchHttp({


Unused variable launchPromise.

…ent creation

…ckages

…tion across Android, iOS, and Web packages

…MCP tools

…lexibility

github-actions bot added the change: feat label Nov 25, 2025

quanru requested a review from Copilot November 25, 2025 12:36

Copilot started reviewing on behalf of quanru November 25, 2025 12:36 View session

Copilot finished reviewing on behalf of quanru November 25, 2025 12:40

Copilot AI reviewed Nov 25, 2025

View reviewed changes

quanru force-pushed the feat/mcp-simplify-tools branch from 9535f2f to 4613d05 Compare November 25, 2025 12:53

quanru requested a review from Copilot November 27, 2025 02:35

Copilot started reviewing on behalf of quanru November 27, 2025 02:35 View session

Copilot finished reviewing on behalf of quanru November 27, 2025 02:38

Copilot AI reviewed Nov 27, 2025

View reviewed changes

quanru force-pushed the feat/mcp-simplify-tools branch from ff57aac to 9cfd87e Compare November 27, 2025 02:53

quanru requested a review from Copilot November 28, 2025 07:37

Copilot started reviewing on behalf of quanru November 28, 2025 07:37 View session

Copilot finished reviewing on behalf of quanru November 28, 2025 07:37

quanru force-pushed the feat/mcp-simplify-tools branch from bec2957 to 8f47ac4 Compare November 28, 2025 08:14

Copilot AI reviewed Nov 28, 2025

View reviewed changes

quanru requested a review from Copilot November 28, 2025 11:04

Copilot started reviewing on behalf of quanru November 28, 2025 11:04 View session

Copilot finished reviewing on behalf of quanru November 28, 2025 11:08

Copilot AI reviewed Nov 28, 2025

View reviewed changes

quanru force-pushed the feat/mcp-simplify-tools branch from 0d4a373 to 25fb2ee Compare December 1, 2025 02:47

quanru force-pushed the feat/mcp-simplify-tools branch from d380d75 to 870fe07 Compare December 2, 2025 12:23

quanru force-pushed the feat/mcp-simplify-tools branch from 870fe07 to 2fb6bab Compare December 3, 2025 05:59

quanru force-pushed the feat/mcp-simplify-tools branch from 2fb6bab to 1c23d4e Compare December 3, 2025 07:19

quanru and others added 14 commits December 4, 2025 16:07

feat(mcp): implement temporary device creation for Android, iOS, and …

bbf54e8

…Web tools; enhance action space retrieval

refactor(mcp): remove default action space methods from Android, iOS,…

60fe955

… and Web tools; streamline temporary device creation

feat(mcp): add HTTP launch support for Android, iOS, and Web MCP serv…

c95765a

…ers; enhance command line argument parsing

refactor(mcp): streamline command line argument parsing and launch lo…

c4e67ae

…gic for Android, iOS, and Web MCP servers

refactor(mcp): enhance type definitions and improve error handling in…

cb99f62

… MCP server and tools

refactor(mcp): delegate web MCP server launch to the main MCP package…

8569fde

… for unified HTTP support and argument parsing

feat(mcp): refactor MCP server structure and tools

cfa03e0

feat(web-bridge-mcp): add initial implementation of web bridge MCP se…

3fb98b4

…rver and tools

refactor(mcp): remove web-mcp package and update dependencies

0ee904f

feat(mcp): refactor code structure for improved readability and maint…

88698d3

…ainability

quanru force-pushed the feat/mcp-simplify-tools branch from 4280717 to 88698d3 Compare December 4, 2025 08:11

quanru added 2 commits December 4, 2025 16:19

chore(mcp): update @rslib/core to version 0.18.3 across multiple pack…

848fcd1

…ages

refactor(tests): enhance server startup handling with error catching …

5c2fddb

…and retry logic

quanru force-pushed the feat/mcp-simplify-tools branch from bd1d89d to 5c2fddb Compare December 4, 2025 11:09

quanru requested a review from Copilot December 4, 2025 11:35

Copilot started reviewing on behalf of quanru December 4, 2025 11:36 View session

Copilot finished reviewing on behalf of quanru December 4, 2025 11:37

Copilot AI reviewed Dec 4, 2025

View reviewed changes

quanru added 3 commits December 5, 2025 08:08

feat(android-tools, ios-tools): add autoDismissKeyboard option for ag…

6fe50da

…ent creation

feat(mcp): add rspack configuration with BannerPlugin for multiple pa…

ccd72e7

…ckages

feat(mcp): update package.json and rslib.config.ts for rspack integra…

34df6ec

…tion across Android, iOS, and Web packages

quanru force-pushed the feat/mcp-simplify-tools branch from 55e3380 to 34df6ec Compare December 5, 2025 03:40

feat(shared): enhance error handling and make parameters optional in …

eabb1f0

…MCP tools

quanru force-pushed the feat/mcp-simplify-tools branch from 82bffa2 to eabb1f0 Compare December 5, 2025 11:13

quanru added 2 commits December 5, 2025 19:47

feat(core): replace console.log with debug logging for locateParam

3ac6a22

feat(mcp): make 'prompt' field optional in locate schema for better f…

69b9d84

…lexibility

feat(mcp): simplify MCP tool architecture and unify platform tools #1507

Are you sure you want to change the base?

feat(mcp): simplify MCP tool architecture and unify platform tools #1507

Uh oh!

Conversation

quanru commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Changes

🏗️ Shared Infrastructure (@midscene/shared/mcp)

📦 Platform-Specific Packages

🎯 Simplified Tool Architecture

🔧 Recent Improvements

🧪 Testing & Quality

Migration Notes

Related Issues

Uh oh!

netlify bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for midscene ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

quanru commented Nov 25, 2025 •

edited

Loading

🏗️ Shared Infrastructure (`@midscene/shared/mcp`)

netlify bot commented Nov 25, 2025 •

edited

Loading