✦
Create Lab
Compose scenes from forklifts, pedestrians, robots and shelves. Click paths, drag waypoints, set speeds — then analyse your own world.
Meta Global AI Developer Hackathon 2026
Indie Track · Push the Frontier
Understand what happened.
Verify what could happen next.
FrontierSight connects Meta AI — Muse Spark reasoning and SAM 3.1 segmentation — to an
independent deterministic physics engine. Models propose and explain. Geometry decides.
Every number carries its provenance: MODEL_DERIVED, DETERMINISTIC, FIXTURE, or ASSUMPTION.
Loading diagnostics…
TestedCPU engine & controls
7analytical validation cases
5curved-path scenarios
10MCP tools exposed
4Meta models integrated
I Built an AI That Tests Near-Misses with Meta SAM 3.1 + Physics | FrontierSight AI
Quantura AI: Market Forecasts · 3-minute demo · Watch on YouTube ↗
Powered byMuse Spark 1.3 — reasoning & hypotheses
SAM 3.1 — segmentation & trackingMuse Image — illustrative visuals
+ a deterministic engine that answers to physics, not prompts
Get started in three moves
○ Run the warehouse near-miss demo (Studio → Run analysis)
○ Build or generate a scene in Create Lab and analyse it
○ Ask Muse Spark to design a scene from a sentence
Progress is kept in this browser.
How it works
⌖
1 · Observe
SAM 3.1 segments and tracks people, forklifts, robots in video. Muse Spark proposes what to look for — it never invents a detection.
✓
2 · Verify
Closest-approach, time-to-collision, continuity and constraint checks run in a seeded, reproducible engine. Uncalibrated video stays in pixel space — no fake metres.
⇄
3 · Counterfact
“What if the forklift had slowed 1s earlier?” A deterministic braking simulation reruns the world, Monte-Carlo intervals included, and an independent verifier checks the result.
Try it in ten seconds
Load a ready-made world straight into the Studio — no setup, no keys.
Everything in the workbench
✎
Prompt-to-scene
Describe a scene in a sentence; Muse Spark designs it as schema-validated JSON. You edit every waypoint before anything runs.
▶
Video Studio
Frame-synchronised playback with heading-oriented entities, event seeking, occlusion markers and an all-pairs inspector.
⇄
Counterfactual simulator
Pick scene, agent and target; brake earlier or harder. Trajectory map, separation chart, Monte-Carlo intervals, per-actor results.
▤
Evidence & trace
Nine auditable agent stages with artifact hashes, then a human-approved, hash-bound evidence packet you can export and replay.
◉
Live Meta models
Muse Spark hypotheses, SAM 3.1 segmentation and Muse Image — verified with real API calls, honestly labelled when offline.
🎙
Voice dictation
Speak your scene instead of typing: Muse Voice Transcribe turns speech into an editable prompt (browser-decoded to studio WAV first), then Muse Spark builds it.
✦
Custom objects by Muse
Describe any object — Muse Spark designs a buildable version, Muse Image renders it, and it becomes a real component in 2D and 3D, saved to your profile.
⌘
MCP server
The deterministic engine is an MCP server (HTTP + stdio): ten tools any agent can call for measured analysis and counterfactuals — no credits spent, no approvals bypassed.
Architecture
Text version (for screen readers & copying)
Video / fixture ─► SAM 3.1 segmentation ─► Scene extraction ─► Muse Spark hypothesis
│
Evidence packet ◄─ Human review ◄─ Independent verification ◄─ Deterministic counterfactual simulation
(hash-bound) (approval gate) (engine/geometry) (engine/counterfactual)
Limitations, stated plainly: synthetic fixtures validate the pipeline on synthetic data only; no safety certification is claimed; metric claims require declared calibration. See docs/ in the repo.
What we will not claim
Synthetic fixtures validate the pipeline on synthetic data only. SAM video-mode tracking on real industrial footage is integrated but not yet benchmarked. Counterfactuals are conditional predictions under declared assumptions. No physical safety certification is claimed — and every evidence packet says so, next to the numbers.
Build a world. Then prove it.
Start with a scene. See what you can learn. No card required.
Experiment workspace
My workspaces
Saved Create Lab scenes and SAM frame sets — yours, searchable, reopenable, updatable. Stored in this browser; sign in to keep them on the server too.
Nothing saved yet — build a scene in Create Lab, or segment frames in the Studio, then save.
Analytical validation cases
- loading…
Video intelligence studio
Fixture replayView:
Same tracks, same clock — 3D is a visualisation; analysis stays planar.
Track legend
Entities are drawn to scale and oriented to their heading: forklift pedestrian robot vehicle cyclist shelf · trails = observed paths · red dashed ring = declared occlusion window. Click an event to seek.
SAM 3.1 Vision — segment an image or video frame
Load an image or video (stays in your browser), then segment the current frame with SAM 3.1. For video: segment at two or more moments, then build tracks from the boxes (greedy-IoU association) and run the full analysis on them. A still image can instead become a 2D cutout scene and an illustrative 3D mask extrusion. Detections are image-space — results stay in pixels unless you declare a scale in Create Lab terms.
Photo: Bernd Dittrich on Unsplash.
Captured frames: 0 · overlays show SAM boxes, stable ids and confidences. The gallery contains attributed Unsplash photographs. Choose another example or upload your own image or video. Inference runs only when you press Segment.
Motion & geometry inspector
Run analysis in the Studio, then inspect measured tracks, coordinate frames, and uncertainty here.
Scene from your segmented image
The 2D scene uses actual image pixels and decoded SAM masks. The 3D view extrudes those silhouettes with assumed thickness and layer positions; it is not a metric reconstruction or a physical safety model.
2D image space
3D silhouette approximations
Create Lab — build a scene, set the paths, run the proof
Create components, give each one a path and a speed, watch them move, then run the same deterministic analysis and counterfactual pipeline used for the built-in scenarios. The stage is a 16 × 14 metre floor plan (editable above); positions are metres, so metric claims are valid by construction. Nothing here is pre-baked: tracks are sampled from your paths at 10 Hz when you press Run analysis.
Untitled scene
unsavedView:
3D visualises the same 2D scene model — heights are illustrative, physics stays planar. Use Move objects to reposition a component and its path. Use Edit paths to change individual waypoints.
Floor size & scenery
The stage is 16 × 14 m by default — set your own floor size; paths, previews and analysis use it.
Drag an object to move it with its path, or select it and click a destination. Arrow keys move the selected object when the canvas is focused.
t = 0.00s
Closest pair: —
Counterfactual simulator
Compare original and braking trajectories. Changes rerun the deterministic simulation.
Presets:
Trajectory map
t = 0.00s
Trajectory legend
Dashed slate = agent's original path · teal = counterfactual · thin lines = other actors · ◇ = closest approach (counterfactual). Ghost marker = where the original agent would have been.
Separation over time
Focus pair distance: original (slate) vs counterfactual (teal); red band = contact distance (sum of radii).
Evidence & trace
Agent reasoning & verification
Structured stage evidence — status, attribution, artifact hashes, retries. No private chain-of-thought is exposed; decision summaries and tool evidence only.
- Run analysis to populate the trace.
Evidence report
Hash-bound report: experiment identifier, source digest, calibration, ground truth, run stages, validation cases, and honest labels
(real_video_evaluation_completed: false, hardware_validation: unavailable, no safety certification claimed).
Human approval gate
Export is locked until a named human approves the current run. Approval is recorded in the exported packet.
Run analysis first for a run-bound report.
Build. Test. Share with confidence.
Choose your next step.
Start exploring physical-world scenarios for free. Upgrade for deeper simulation, more saved work and a workflow built around your research.
Billed monthly
Save $38 with annual
Checking plan availability…
Explorer
$0
Explore at your own pace · no card required
- ✓ All 5 built-in scenarios
- ✓ Create Lab + prompt-to-scene
- ✓ Deterministic analysis & counterfactuals
- ✓ Evidence export with recorded human approval
- ✓ 10 server-saved scenes · Monte-Carlo ≤ 200
Researcher
$19/mo
For researchers turning experiments into evidence
- ✓ Everything in Explorer
- ✓ Your own Meta API key (encrypted, server-side)
- ✓ SAM 3.1 video-mode tracking
- ✓ Unlimited server-saved scenes
- ✓ Monte-Carlo up to 2,000 runs
- ✓ Priority support requests
Manage or cancel from your billing portal when subscribed.
Team / Enterprise
Custom
A tailored workspace for your team
- ✓ Everything in Researcher
- ✓ Shared org workspace & member roles
- ✓ Org audit log (exports, approvals, provisioning)
- ✓ Custom simulation budgets
- ✓ Onboarding session for your safety team
Scoped in the custom contract: SSO/SAML via your identity provider, private VPC deployment, SLA terms.
Compare plans
| Capability | Explorer | Researcher | Team |
|---|---|---|---|
| Built-in scenarios & Create Lab | ✓ | ✓ | ✓ |
| Deterministic analysis & counterfactuals | ✓ | ✓ | ✓ |
| Meta calls | Host key, fair use | Your own key (BYOK) | Your own key (BYOK) |
| SAM 3.1 video-mode tracking | — | ✓ | ✓ |
| Server-saved scenes | 10 | Unlimited | Unlimited + shared org library |
| Monte-Carlo runs / analysis | 200 | 2,000 | Scoped with your team |
| Evidence export with approval record | ✓ | ✓ | ✓ + org audit log |
| Support | Standard queue | Priority queue | SLA (contract) |
| SSO / SAML · VPC · SLA terms | — | — | Custom contract |
Start with included model access, subject to fair-use limits. Researcher supports your own Meta key for a dedicated workflow. Meta usage with your key is billed separately by Meta.
A few things worth knowing
Can I try FrontierSight before upgrading?
Yes. Explorer includes Create Lab, the built-in examples and deterministic analysis. No card is required.
What does annual billing save?
Researcher is $190 per year, compared with $228 over 12 monthly payments. That’s $38 saved. Annual plans are paid upfront.
Are model API costs included?
Shared model access has fair-use limits. When you connect your own Meta API key, Meta bills that usage separately.
How do I get a team workspace?
Request a quote to discuss shared workspaces, member roles, onboarding and your simulation needs. SSO, private deployment and service terms are scoped individually.
Your workspace, ready when you areWelcome to FrontierSight.
Save your scenes. Revisit your experiments. Build your next insight.
Loading secure sign-in…
Plan, billing and your own Meta API key live in Settings. Scenes and runs on this device are scoped to your account.
Need a hand? Contact our team.
Terms of Service
Last updated: October 28, 2026 · FrontierSight AI by Quantura Studio ("FrontierSight", "we").
1. What this service is
FrontierSight AI is a research workbench that connects Meta AI models (Muse Spark, SAM 3.1, Muse Image) to a deterministic physics/geometry engine for analysing recorded and user-created physical scenes. Available features and enrollment status are described on the Pricing page. Custom team services require a separate agreement.
2. No safety certification
Outputs are conditional analyses under declared assumptions. They are not safety certifications, risk assessments for operational use, or guarantees about real-world behaviour. You must not rely on FrontierSight output as the sole basis for deploying robots, vehicles, or industrial equipment. Synthetic fixtures validate the software pipeline on synthetic data only.
3. Your content
You retain all rights to scenes, paths, and media you create or load. Scenes you create are stored only in your browser (and encoded in share links you choose to copy). When you sign in and choose to save or sync, scene data and account records may also be processed on the server. You grant us no licence to your content beyond operating the features you invoke. You are responsible for having the right to use any footage or data you analyse, including the privacy rights of people appearing in it.
4. Acceptable use
- Do not use the service to operate or control physical equipment.
- Do not attempt to extract API keys, bypass rate limits, or disrupt the service.
- Do not upload unlawful content or content you have no right to process.
5. AI model outputs
Model-derived content (hypotheses, narration, generated scenes, segmentation) may be inaccurate and is labelled by provenance in-product. Deterministic calculations are reproducible from the artefacts exported in an evidence packet; model outputs are not represented as measurements.
6. Availability & liability
The service is provided "as is" without warranties. To the maximum extent permitted by law, Quantura Studio is not liable for indirect or consequential damages arising from use of the service. Paid access, when available, is billed at the amount and recurring interval displayed at checkout; custom contracts may provide additional terms.
7. Changes
We may update these terms; material changes will be dated here. Questions: hello@quantura.studio.
Privacy Policy
Last updated: October 28, 2026 · FrontierSight AI by Quantura Studio.
Account and workspace information
Clerk processes sign-in information and authentication cookies when you open sign-in or use an account feature. Our server verifies your session and may retain your verified account identity, subscription status and organization membership. We do not receive your Clerk password.
Guest scenes, preferences and run history stay in your browser. Choosing Save or Sync while signed in sends the relevant workspace, object definitions or captured frames to the configured server store. Contact, support and access-request forms send the information you submit to our server. Model keys you explicitly save use server-side encryption.
Model processing
When you invoke generation, transcription, hypotheses or segmentation, the necessary prompt, image, audio or frame is sent from our server to the Meta Model API. Provider processing is governed by Meta’s terms. Loading a local file alone does not send it for inference. Deterministic scene preparation, resizing and simulation do not call a model.
Billing and infrastructure
When checkout is enabled, Stripe processes payment details and provides billing status to us. Card details do not pass through FrontierSight’s server. The hosting provider may retain ordinary request logs such as IP addresses, URLs and timestamps. Clerk and model client libraries may load from their service/CDN domains. The YouTube player loads after you press Play.
Storage and retention
Browser records remain until removed in Workspace, Settings or your browser. Server retention depends on the configured store; the current account store does not guarantee retention across redeployments. Export work you need to retain. Paid checkout on the Vercel deployment remains unavailable until durable billing persistence is implemented and verified.
Sharing and deletion
A copied scene link contains the scene data: share it only with intended recipients. Signing out ends access on this device but does not itself delete saved records. Use the workspace deletion controls for your saved scenes, or contact us about account/server data. Clearing site data removes browser copies, not copies held by recipients or providers.
Sensitive data and contact
This workbench is intended for professional and research use, not children. Do not submit confidential footage or sensitive personal information without appropriate rights and consent. Privacy requests: hello@quantura.studio.
Help & guides
Everything FrontierSight does, how to drive it, and what to do when something looks wrong. Search this page, or jump to a topic.
Getting started (2 minutes)
- Watch or run the demo. Overview → Run the warehouse near-miss demo, or open the Video Studio and press Run analysis. Events appear on the right; click one and the player seeks to it.
- Make your own world. Create Lab loads with a near-miss template. Drag waypoints, change speeds, then Run analysis. Or describe a scene in words and let Muse Spark design it.
- Ask “what if?”. In Counterfactual, pick the agent that should have braked, drag the intervention sliders or a preset, and read the per-actor results — including when braking helps one actor and hurts another (that happens, and the tool shows it rather than hiding it).
No account or key is needed for any of this. Live Meta features light up automatically when the deployment has a key configured (see the badge in the top bar).
Create Lab & path editing
- Add a component (forklift, pedestrian, robot, vehicle, cyclist, static shelf) — it appears on the stage selected. Use Move objects to drag its body and complete path, or switch to Edit paths to lay out waypoints. Choose straight lines or smooth curve per component (curves are Catmull-Rom splines driven at constant, physically-limited speed — objects slow for tight turns and brake to a stop, just like the analysis assumes).
- In Edit paths, click any path to select its component. Then: click near the line to insert a point exactly there, click empty stage to append, drag points to reshape, Shift-drag a path to move it whole, right-click or double-click a point to delete it, Delete/Backspace removes the last point. Hovering a path shows a pointer — paths are meant to be grabbed.
- Path modes: stop (ends and holds), loop (patrol), ping-pong (back and forth). Amber warnings flag turns tighter than the vehicle can physically make, paths leaving the stage, or routes through a shelf — fix them by widening the curve.
- 2D/3D: both views edit and preview the same scene on the same clock; 3D adds orbit and object dragging and destination placement in Move objects mode. Edit individual waypoints in 2D. Your per-section view choice is remembered.
- Generate with Muse: describe the scene (up to 4,000 characters). Muse returns components, curved paths and speeds that are schema-validated and then scored by the engine (interaction quality, shelf collisions, turn warnings) — a poor first draft is retried once with the exact issues fed back. The scene is auto-labelled; press Save workspace to keep it. It also appears in the Video Studio dropdown under My workspaces.
- Workspaces: save labs and SAM frame sets, then search, open, update, rename or delete them from Workspace (⋯ menu on each card). Guests keep them in this browser; signed-in users sync to the server.
Analysis explained (honestly)
- Measured numbers are sampled along the actual (curved) trajectories at 50 ms — closest distance, contact, when it happened. Predicted CPA/TTC is a constant-velocity extrapolation shown next to it. An LLM never produces these numbers.
- Calibration rule: without a declared scale, results stay in pixels and the tool refuses metric claims rather than inventing metres. Create Lab scenes are in metres by construction.
- Counterfactuals replay paths with a braking agent (path-following simulation + Monte-Carlo intervals). They are conditional predictions under listed assumptions — not safety certifications, and every evidence packet says so.
- Evidence export requires a named human's approval of that exact run (Evidence view). The approval is recorded inside the exported packet; the hash chain makes tampering detectable.
Accounts, plans & billing
- Sign in with Clerk. Guests can use everything in Explorer; signing in adds server-saved workspaces.
- Explorer (free): all built-in scenarios, Create Lab, prompt-to-scene, deterministic analysis, evidence export, 10 server-saved scenes, Monte-Carlo up to 200 runs.
- Researcher ($19/mo via Clerk/Stripe when enabled): your own Meta API key (BYOK, encrypted), SAM 3.1 video-mode tracking, unlimited server scenes, Monte-Carlo up to 2,000, priority support.
- Team (custom): shared org workspace and roles, org audit log, onboarding; SSO/SAML, VPC and SLA are scoped in the contract (Pricing → Request a quote).
- Seeing 402? You hit a plan gate — the message names the limit and the plan that removes it. Seeing BLOCKED_CREDENTIALS? The deployment has no Meta key configured; fixture/demo mode still works fully.
API & MCP
The deterministic engine is exposed two ways:
- REST: the same endpoints the UI uses (see Settings for the live
/api/me, scenarios, analysis, counterfactual, SAM and billing surfaces). - MCP server:
POST /api/mcp— connect Claude Code in one line:claude mcp add --transport http frontiersight https://frontiersight-ai.vercel.app/api/mcp, or run it offline from a clone:python3 scripts/mcp_stdio.py. Eight deterministic tools (scenarios, measured analysis, path-following counterfactuals, curve sampling, scene validation). By design, no MCP tool spends Meta credits or bypasses the human approval gate. Full guide:docs/MCP.mdin the repository.
Troubleshooting
- Analysis seems stuck: live runs include a Muse call and can take 5–25 s (the button shows an elapsed timer). Past ~95 s the UI aborts with a named error — press Run again.
- “SAM returned zero matches”: a valid result, shown as-is. Try a clearer concept (one short noun phrase, e.g. “forklift”), a sharper frame, or the default example frame.
- Export is locked: evidence downloads unlock only after a named human approves that run in the Evidence view — by design.
- 3D won’t open: your browser/GPU declined WebGL; the app returns to 2D with your scene untouched. Nothing depends on 3D.
- My workspace is gone: guest workspaces live in this browser (clearing site data deletes them). Sign in to keep them on the server.
- The timer looks wrong: playback is true delta-time (0.5×/1×/2× are exact); if a scene is longer than the scrub range, open it via Workspace so the range rescales.
Contact & support
Signed-in users can open a support request in Settings → Support (paid plans go to the priority queue). Otherwise: email hello@quantura.studio, file an issue on GitHub, or watch the 3-minute demo again — most “how do I…” questions are answered in it.
System settings
Your plan & account
Loading account…
Meta Model API connection (deployment)
loading…
The API key lives server-side only (MODEL_API_KEY) and is never sent to the browser. Model ids are configurable via META_MODEL / META_SAM_MODEL / META_IMAGE_MODEL. Retention: uploaded media is processed in-memory in this demo deployment; fixtures are synthetic and openly redistributable.
API & MCP server
MCP endpoint: /api/mcp · also runnable offline over stdio:
python3 scripts/mcp_stdio.py · docs: docs/MCP.md
loading tool list…
Claude Code: claude mcp add --transport http frontiersight https://frontiersight-ai.vercel.app/api/mcp
Cursor / VS Code (mcp.json): {"mcpServers": {"frontiersight": {"url": "https://frontiersight-ai.vercel.app/api/mcp"}}}
Deployment & limits
App version — · ingest bound 12 MB per request · rate limit 120 POST/min per client · retention: this demo stores scenes/runs only in your browser; the server keeps no uploaded media.
Organisation (Team)
Loading organisation…
Support
Local data
Profiles, scenes, runs, theme and preferences live in this browser only.