Aerial Intelligence Meets Agentic AI

Introducing TestSamGeo — an MCP server that brings SAM 3-powered satellite imagery segmentation directly into Claude Cowork, OpenClaw, or Hermes AI Agents.

Finding a swimming pool in a satellite image used to mean hiring a GIS analyst, building a labeled training dataset, and waiting weeks for a custom detection model to converge. As of late 2025, it means typing a sentence.

Meta's Segment Anything Model 3 — SAM 3 — introduced Promptable Concept Segmentation: describe what you want to find in plain language, and the model locates, outlines, and scores every matching instance it detects. No training. No fine-tuning. No labeling pipeline.

We've wired that capability directly into Claude Cowork via an experimental TestSamGeo MCP connector, backed by our MapSavvy WMTS. Any Cowork session can now query a street address, describe a feature, and receive georeferenced GeoJSON polygons — ready for GIS, database, or a live Leaflet map overlay — without writing a single line of code.

What Is SAM 3?

SAM 3 builds on Meta's Segment Anything lineage by introducing Promptable Concept Segmentation (PCS). Where SAM 2 required you to click a point or draw a bounding box to isolate an object, SAM 3 accepts natural-language descriptions — "yellow school bus," "swimming pool," "rooftop solar array" — and finds every matching instance across the image.

The SamGeo library (segment-geospatial, by Dr. Qiusheng Wu at opengeos) wraps SAM 3 in a geospatial-native API: it handles tile downloading, CRS reprojection, memory-efficient tiled processing, and export to GeoTIFF, GeoPackage, Shapefile, or GeoJSON. Our EC2 endpoint runs SamGeo against MapSavvy aerial imagery and returns results over MCP.

MapSavvy WMTS serves satellite imagery tiles from multiple vendors, merged into a processing-ready grid before being handed off to SamGeo for segmentation. After features are discovered and vectorized, a Leafmap HTML overlay is generated — displaying polygon outlines atop the aerial base layer with per-feature confidence popups.

The MCP Interface

The TestSamGeo connector exposes exactly two tools inside Claude Cowork:

• segment_features — accepts a feature_type (text description), address, grid size, imagery tiles, and confidence threshold; returns GeoJSON polygons and a Leafmap HTML overlay.
• health — confirms the EC2 endpoint is live and ready.

Only feature_type is required. Every other parameter has a sensible default, so the simplest possible Cowork prompt is a single natural-language sentence pointed at an address.

Live Demo: 71 Pools in Scottsdale

To stress-test the connector, we pointed it at McCormick Ranch in Scottsdale, Arizona — one of the most pool-dense residential neighborhoods in the United States.

find swimming pools at McCormick Ranch, Scottsdale, AZ use grid 9 and confidence >= 0.3

SAM 3 processed the tile mosaic and returned 71 pool polygons with per-polygon confidence scores, a GeoJSON file, and a live Leafmap HTML overlay — all in a single round trip.

77% of detections scored above 0.70 — exceptionally high for a zero-shot, zero-training segmentation run. The nine low-confidence results below 0.30 warrant human review: they likely represent hot tubs, decorative ponds, or reflective surfaces that produced false positives.

"77% of detections scored above 0.70 confidence — no training, no labeling, no code."

The output GeoJSON is immediately usable: each feature carries a confidence property, coordinates in EPSG:4326, and polygon geometry ready for any GIS tool. A second prompt against a residential parcel in Paradise Valley demonstrated building detection:

find buildings at 6201 E Huntress Dr, Paradise Valley, AZ 85253 use grid 9 and confidence >= 0.5

The Leafmap HTML overlay renders polygons directly atop the satellite tile and is shareable as a standalone link — no GIS software required on the receiving end. Hermes Agent, running on an Nvidia Nano board, can invoke the same TestSamGeo MCP for edge-deployed queries, with results accessible via Tailscale-connected laptops through the Hermes WebUI.

TestSAMGeo — Swimming Pools · 8362 E Vía De Encanto, Scottsdale AZ

find swimming pools at 8362 E Vía De Encanto, Scottsdale, AZ 85258, grid 9, google satellite, zoom 20, confidence >= 0.3

Real-World Applications

Insurance & Property Assessment

Underwriters and assessors can query any address and immediately surface pools, outbuildings, trampolines, or other risk-relevant features — features that may not appear in tax records or MLS listings. SAM 3's text prompts eliminate the need for a pre-labeled training set for each feature type: just describe it.

Municipal & Tax Audit

Pools built after the last permit survey, unpermitted additions, and improvements that haven't triggered reassessment are all detectable at city scale. A single Cowork session can queue an entire subdivision for follow-up review.

Infrastructure Monitoring

Solar panel deployment, HVAC unit density, roof condition, and parking utilization — any visually distinctive feature can become a queryable dataset. Combine with RouteSavvy route optimization to dispatch field inspectors only to flagged parcels.

Field Dispatch Acceleration

Pair SAM 3 detections with the RouteSavvy MCP optimizer: identify target parcels via imagery, geocode the polygon centroids, and feed them directly into a route-optimized dispatch plan — all inside a single Cowork conversation.

What's Next

The TestSamGeo connector is live and connected in Claude Cowork today. Near-term work includes:

• Rooftop segmentation with area/perimeter export for insurance square-footage validation
• Solar panel detection and azimuth estimation for energy assessors
• Change detection between imagery vintages using SAM 3's video tracking primitives
• Batch processing across address lists with Cowork-native result visualization

The underlying infrastructure — an AWS g6.xlarge instance, MapSavvy for licensed aerial imagery, and the segment-geospatial library — is already handling experimental queries. The MCP abstraction means the capability is available to any agentic AI platform: Claude Cowork, OpenClaw, and Hermes alike. No Python environment, no GPU, no GDAL install required on the client side.

The future of aerial feature detection is a text prompt away.