Swift MCP GUI Server: Precision Automation & Surgical Control

Key Features of Swift MCP GUI Server: Precision Automation & Surgical Control?

• Mouse Teleportation: The moveMouse tool sends your cursor to exact coordinates—perfect for bypassing that annoying "are you sure?" dialog.
• Button Whack-A-Mole: mouseClick handles left/right clicks so reliably, your cat could operate your computer (but please don’t).
• Keyboard Symphony: sendKeys plays your keys like a pianist—try ["command", "space"] to summon Spotlight without lifting a finger.
• Scrolling Sorcery: scroll lets you breeze through pages like a wizard flipping through spellbooks.

Use Cases of Swift MCP GUI Server: Precision Automation & Surgical Control?

Think of this as your Swiss Army knife for macOS automation:

• QA Engineers: Automate UI tests that click every button faster than you can say "edge case."
• Productivity Gurus: Script repetitive workflows so your computer does the heavy lifting while you plot world domination.
• Game Testers: Simulate 100-hour playthroughs to find that one pixel that breaks everything.

FAQ: Because We Know What You’re Thinking

Is this safe? Absolutely—if you treat it like a loaded keyboard. Only connect trusted clients, and never let it near your banking apps.

Does it work on Big Sur? Only if "Big Sur" refers to your caffeine consumption. Requires macOS 10.15+ and a sense of adventure.

Will it replace me? No, but it might start. Start small with this tutorial to keep control.

Swift MCP GUI Server

A Model Context Protocol (MCP) server that allows controlling Mac OS X through SwiftAutoGUI. This server provides tools for programmatically controlling the mouse and keyboard through MCP clients.

Requirements

• macOS 15.0 or later
• Swift 6.0 or later
• Xcode 16.0 or later

Installation

1. Clone this repository:

git clone https://github.com/NakaokaRei/swift-mcp-gui.git
cd swift-mcp-gui

2. Install

swift package experimental-install

3. Add command to your MCP client.

{
  "mcpServers" : {
    "swift-mcp-gui" : {
      "command" : "/Users/USERNAME/.swiftpm/bin/swift-mcp-gui"
    }
  }
}

Available Tools

The server provides the following tools for controlling macOS:

1. Mouse Movement

• Tool name: moveMouse
• Input:
  • x: double (x-coordinate)
  • y: double (y-coordinate)
• Moves the mouse cursor to the specified coordinates

2. Mouse Clicks

• Tool name: mouseClick
• Input:
  • button: String ("left" or "right")
• Performs a mouse click at the current cursor position

3. Keyboard Input

• Tool name: sendKeys
• Input:
  • keys: Array of strings (key names)
• Sends keyboard shortcuts or key combinations
• Example keys: "command", "control", "option", "shift", "return", "space", "a", "1", etc.

4. Scrolling

• Tool name: scroll
• Input:
  • direction: String ("up", "down", "left", "right")
  • clicks: Integer (number of scroll clicks)
• Performs scrolling in the specified direction

Security Considerations

This server has full control over your mouse and keyboard. Be careful when running it and only connect trusted MCP clients.

License

MIT License