Hayabusa Operator Manual

6) Networking

Networking is where overlay, tunnel, and tenant isolation are monitored and controlled — the transport and internet layers from the mental model (§2), made operable in the UI.

When diagnosing incidents, prioritize: identity scope → tunnel status → route/selector correctness → host reachability and latency. That order prevents false conclusions from stale or out-of-scope sessions.

Subsections in this category

• 6.1 WireGuard + Headscale — mesh overlay.
• 6.2 IPsec — classic/interop tunnels.
• 6.3 Tenant VPC / Subnet Allocation — private CIDR ownership.

6.1) WireGuard + Headscale

WireGuard provides encrypted transport. Headscale provides self-hosted coordination and identity for mesh networks. Together they support operator-managed overlay connectivity between Core, regional hubs/controllers, and managed devices.

Interpret overlay status with underlay evidence. A tunnel can be “up” while service quality degrades due to jitter or loss beneath it — use map overlay health coloring and link telemetry together before changing peer config.

Operator checklist

1. Check server/client (or Headscale) status before troubleshooting endpoints.
2. Confirm tenant / namespace assignment matches the intended organization or user scope.
3. Verify expected peers appear in host/peer lists.
4. If quality is bad but the tunnel is up, switch to Underlay (Workstations) and Path Trace before rotating keys.

6.2) IPsec

IPsec supports tunnel scenarios that need selector-based or interoperability-oriented connectivity (site-to-site, partner networks). It often complements mesh overlay workflows rather than replacing them.

Validate three dimensions independently: IKE/session establishment, selector correctness, and application reachability. Successful IKE does not guarantee selector accuracy; selector accuracy does not guarantee firewall allowance.

When to prefer IPsec vs WireGuard mesh

• IPsec — peer requires classic IPsec/IKEv2, strict selectors, or existing enterprise VPN standards.
• WireGuard + Headscale — operator-managed mesh, modern clients, simpler key/identity coordination inside Hayabusa.

6.3) Tenant VPC / Subnet Allocation

Hayabusa can allocate tenant private address realms from a configured pool and persist assignments. That creates deterministic ownership of private CIDR space per user or organization so overlays and tunnels do not accidentally collide.

How to operate safely

• Validate proposed CIDRs before assignment (size, RFC1918 fit, no overlap).
• Avoid overlap with existing tenant realms and with underlay LAN ranges you still need to reach.
• Document ownership of each assigned block (org/user, purpose, date).
• After change, verify routing/overlay peers still reach expected hosts.

7) Zero Touch Provisioning (ZTP)

ZTP automates first contact so blank or factory devices enter service with a consistent baseline — DHCP/PXE on the site LAN, config fetch from the controller, then claim and follow-on OpenTofu/Ansible from Core.

When DHCP controls appear in the UI, treat enablement as a controlled network event. An unintended DHCP server can conflict with existing address assignment on a shared segment. Use a change window and confirm you own the broadcast domain first.

Recommended flow

1. Prepare baseline templates (and Image Nest media if you need Windows/Linux install artifacts).
2. Review pending/discovered devices in Fleet Incoming before claim.
3. Apply provisioning and validate post-state.
4. Record deviations and update templates.

7.1) Image Nest — Windows & Linux install media

Image Nest is a Controller feature (open it from the connected controller UI at /image-nest). You bring a legal Windows or Linux ISO; Hayabusa never redistributes Microsoft media. Nest banks dated artifacts and publishes hydrate placeholders such as <<IMAGE_NEST:latest-win11-stripped>> for OpenTofu/Ansible on LAN Approve.

Deploy targets on the Image Nest page

Works for Windows and Linux ISOs. Pick a target first — the banked file type changes with the target:

Buttons and actions

• Strip & bank / Full bank — bake for the selected deploy target (optional package strip).
• Publish ISO as-is — bank the uploaded medium for bare metal without a strip bake.
• Guest initial config — hostname, admin user, password, disk size for Windows Autounattend when baking Bootable / Hyper-V / VMware disks.
• → VHDX / → VMDK on a bank row — convert an existing banked disk with qemu-img.
• Hayabusa hand-off — write tfvars for Core; the controller never runs OpenTofu.

How this ties to OpenTofu / ZTP

1. Upload ISO on the controller Image Nest page.
2. Choose target → Strip/Full bank (or Publish ISO as-is).
3. In OpenTofu/Ansible commit placeholders only, e.g. source_image = "<<IMAGE_NEST:latest-win11-bootable>>".
4. Queue the job from Hayabusa; on LAN Approve the controller hydrates nest refs (same pattern as secrets).
5. Core applies OpenTofu/Ansible with resolved paths.

8) Security and Pentesting

Security workflows in Hayabusa support authorized assessment and risk discovery — not unscoped scanning of networks you do not own. Pair every scan or SECops playbook with scope control, approval, and a remediation path.

What lives where

• Fleet / DevOps workspace — SECops Ansible trees and related playbooks under your tenant workspace (often with packaged defaults that GitOps does not overwrite).
• Controller — site-local secrets and LAN Approve for jobs that need vault material or Image Nest media.
• Observability / alerts — evidence after changes; optional AI suggest-from-alert paths when enabled.

Responsible workflow

1. Define scope and written approval (systems, CIDRs, time window).
2. Run low-impact reconnaissance first.
3. Escalate to active checks only where authorized.
4. Convert findings to tracked remediation tasks; re-test after fixes.

9) Integrations

Integrations connect Hayabusa to outside services and workflows — chat login providers, Git (GitOps), observability backends, Incoming, vendor ZTP, and optional AI providers. Every integration should have an owner, expected behavior, and a rollback plan.

Common integration families

• Identity — Google, Slack, Discord, Microsoft sign-in (Core).

10) AI and User Keys

Hayabusa supports user-scoped AI keys for assistant functionality. Store keys in your controller and use the SECRET NAMES on Hayabusa core. Never put secrets on Hayabusa core.

• Use least-privilege keys where possible.
• Avoid sending secrets in prompts.
• Validate AI outputs before operational use.

What the embedded AI helper now understands

The floating Hayabusa AI Assistant is aware of the main operating surfaces and should guide users to the correct page, tab, and safety boundary. It can explain Workstations map views, the Ansible/OpenTofu console and Graphical View, tenant-scoped openPLC/ directories, Observability, SCADA/PLC workflows, ROS2 robotics tooling, ZTP/controller roles, and Image Nest deploy targets.

Recommended prompts

• "Show me how to switch from my personal observability view to my organization Grafana view."
• "How do I upload an OpenPLC project folder into the Graphical View?"
• "Which PLC protocols can I read or write from the PLC Viewer, and what safety checks apply?"
• "Where do I find the Robotics Viewer and what ROS2 tools are installed?"

11) Grafana, Prometheus, and InfluxDB

Hayabusa exposes the observability stack through authenticated subpath proxies: /grafana/, /prometheus/, /influxdb/, and the Hayabusa dashboard at /observability. The goal is to keep Grafana, Prometheus, and InfluxDB usable without asking operators to leave Hayabusa or bypass tenant controls.

How the stack is separated

• User scope: personal metrics, dashboards, datasources, and retention settings are associated with the signed-in user context.
• Organization scope: organization metrics and dashboards are associated with the selected organization context and require the user to be a member of that organization.
• Grafana organization separation: Hayabusa provisions or selects Grafana organizations and passes the appropriate auth proxy headers so users land in the expected Grafana org.
• Prometheus query separation: Hayabusa rewrites tenant-aware PromQL queries where appropriate so metric results are filtered to the selected user or organization scope.
• InfluxDB bucket separation: InfluxDB bucket access and retention are tied to the tenant bucket model used by Hayabusa's observability bootstrap logic.

Using the Observability page

1. Open Observability from the top navigation or use the Observability map view on Workstations.
2. Confirm whether the page is in personal or organization context. If you are a member of organizations, use the scope selector to switch deliberately.
3. Use the Grafana panel for dashboards, Prometheus for metric queries and targets, and InfluxDB for time-series bucket inspection where enabled.
4. Check the status cards before troubleshooting a dashboard. A dashboard issue may be caused by a downstream service, proxy path, datasource, or tenant scope mismatch.

Grafana

Grafana is used for dashboarding, alert visualization, datasource exploration, and organization-scoped views. Hayabusa proxies Grafana under /grafana/ and adjusts subpath behavior so assets, API calls, avatars, and dashboard links resolve under Hayabusa instead of the Grafana container root.

• Use Grafana dashboards for high-level operational views.
• Use Explore for ad hoc metric questions.
• Use the Hayabusa scope selector before opening Grafana when you need personal vs organization separation.
• If a panel looks empty, confirm datasource scope, time range, and tenant labels before assuming data loss.

Prometheus

Prometheus stores and queries operational metrics. Hayabusa proxies Prometheus under /prometheus/ and supports tenant-aware query behavior for user and organization separation. Prometheus is best for instant/range queries, target health, rule evaluation, and low-level metric debugging.

• Use the Prometheus query UI when you need raw metric evidence.
• Use target health to distinguish scrape failures from dashboard problems.
• Use scalar queries such as time() normally; Hayabusa avoids turning scalar expressions into invalid tenant-filtered expressions.

InfluxDB

InfluxDB is used for time-series buckets and data that fits the Influx model. Hayabusa proxies InfluxDB under /influxdb/, rewrites root-relative UI paths for static assets, and exposes health/setup calls through the Hayabusa auth boundary.

• Use InfluxDB for bucket-level inspection and time-series workflows configured by operators.
• Use tenant bucket names and retention settings to understand where data lives.
• If the Influx UI appears to onboard unexpectedly, verify the configured admin token and bucket/bootstrap state.

Retention and administration

Retention can be controlled globally by administrators and per allowed tenant context. Admin retention updates are exposed on the Administration page and through /api/admin/observability/retention. User/org observability settings are exposed through /api/observability/retention.

Troubleshooting observability

• Static asset 404s: verify requests include the Hayabusa subpath such as /grafana/, /prometheus/, or /influxdb/.
• Empty Grafana dashboards: confirm tenant scope, datasource provisioning, and the dashboard time range.
• Prometheus query errors: check the raw PromQL and whether the query is tenant-label compatible.
• Influx setup/onboarding screens: confirm PEREGRINE_INFLUX_ADMIN_TOKEN, bucket existence, and service health.
• Live websocket warnings: confirm whether the upstream feature is enabled and whether the browser request is being made through the correct Hayabusa path.

12) SCADA, PLC, ROS2, and NVIDIA Isaac

Hayabusa now includes industrial-control and robotics surfaces for lab, emulator, and authorized real-system workflows. These tools are powerful and must be operated with explicit authorization and correct target scope.

SCADA viewer

The SCADA Workstations view embeds FUXA, an open-source browser-based SCADA/HMI screen builder. Use it when the operator task is about controller status, process visibility, protocol targeting, or SCADA-style context rather than general host infrastructure.

• Start in SCADA when working with controllers, plants, cells, lines, or emulator labs.
• Use FUXA for HMI screens, alarms, process visualization, and industrial-protocol presentation.
• Use it together with PLC Viewer for protocol-specific reads/writes.
• Use Observability alongside SCADA to correlate controller actions with metrics and host telemetry.

FUXA is installed inside the Hayabusa application container and started by /peregrine/start-server.sh when /opt/fuxa/server/main.js is present. It listens internally on FUXA_PORT (default 1881) and stores runtime data under PEREGRINE_FUXA_DATA_DIR (default /var/lib/peregrine/fuxa). Hayabusa exposes the embedded browser UI through the same-origin /fuxa/ proxy and exposes /api/scada/status so the SCADA view can show whether FUXA is reachable and where the browser-facing service is located. Protect the FUXA URL with network controls and FUXA/Hayabusa access policy; do not expose unrestricted write-capable SCADA screens to untrusted networks.

PLC Viewer

The PLC Viewer is available from the tabbed console surfaces on Fleet and Workstations. It supports unified PLC reads and controlled writes through Hayabusa APIs. Current protocol families include Modbus TCP, OPC UA, Siemens S7, Allen-Bradley CIP, Beckhoff ADS, Mitsubishi MC, and Omron FINS UDP, depending on installed driver availability.

Reading PLC data

1. Open Fleet or Workstations and select the relevant host/controller context.
2. Open the console window and select PLC Viewer.
3. Select the protocol and enter the target host, port, address, register/tag/symbol, unit/device ID, and count/type fields required by that protocol.
4. Click Read PLC and review the JSON result.

Writing PLC data

PLC writes are supported but guarded. The server requires the global plc_writes_enabled operator safety flag, and the user must explicitly confirm authorization in the UI for each write request. Write attempts are audited whether they succeed or fail.

1. Confirm you own or are approved to control the target controller or emulator.
2. Confirm admin policy allows PLC writes. In development images this can be controlled with PEREGRINE_PLC_WRITES_ENABLED and persisted safety policy.
3. Enter the write value(s), function code/tag/symbol, and protocol-specific addressing.
4. Check I am authorized to write to this PLC.
5. Click Write PLC and verify the result out of band before continuing.

Programming and OpenPLC

OpenPLC Editor and MatIEC support controller-program development workflows. Keep source files under the tenant-scoped openPLC/ workspace directory, edit them in Graphical View or console, and use OpenPLC tooling from the container when you need project editing or code generation.

Robotics Viewer and ROS2

The Robotics Viewer Workstations view is the robotics-oriented operating surface. ROS2 Humble tooling is installed in the Hayabusa container, including ros2, rosdep, colcon, and base ROS packages. Drone workflows use MAVLink as the vehicle protocol, MAVSDK for application-level drone APIs, and MAVROS when ROS2 topic/service bridging is needed.

• Use ros2 node list to identify active nodes when attached to a ROS2-enabled environment.
• Use ros2 topic list and ros2 topic echo for topic discovery and message inspection.
• Use colcon for ROS2 workspace builds when source packages are available in the container or mounted workspace.
• Use PX4 SITL or ArduPilot SITL for first tests before attaching to real aircraft.
• Keep robot credentials, network keys, and vendor tokens out of prompts and source control.

/api/robotics/status reports local availability of ROS2, MAVLink/PyMAVLink, MAVSDK, MAVROS, and simulator tooling so operators can tell the difference between a missing tool and a target connectivity issue.

ArduPilot SITL and PX4 SITL simulation

The Robotics Viewer includes a simulator control panel and flight visualizer. The current image bundles ArduPilot Copter SITL through dronekit-sitl; PX4 SITL and PX4 vehicles are supported as external MAVLink endpoints such as udp://:14540. Both simulator and real-vehicle telemetry use the same MAVLink helper, so the visualizer, heartbeat, position, battery, attitude, GPS, and guarded-command behavior are consistent.

1. Open Workstations, expand Views, and select Robotics Viewer.
2. In the simulator panel, leave ArduPilot Copter SITL selected, optionally set a home latitude/longitude, and click Start SITL.
3. Click Use SITL endpoint. Hayabusa fills the MAVLink endpoint, starts telemetry, and updates the attitude/position visualizer.
4. For PX4, start PX4 SITL outside Hayabusa and connect to its MAVLink endpoint, commonly udp://:14540. PX4 is not bundled in this image.
5. Use MAVSDK or MAVROS from ROS2 workflows when you need higher-level application APIs or ROS2 topic/service bridging.

MAVLink/MAVSDK connection workflow

The Robotics Viewer includes a MAVLink endpoint form for real telemetry connections. Hayabusa stores the endpoint per user or organization scope, starts a tenant-scoped MAVLink helper process, and reads live heartbeat, position, attitude, GPS, battery, status text, command acknowledgements, parameters, mission state, and richer flight data through PyMAVLink. MAVSDK remains installed for higher-level drone application workflows, while the UI uses the MAVLink helper for reliable cross-worker telemetry.

1. Start a simulator or approved vehicle gateway. Common examples: PX4 SITL udp://:14540, ArduPilot TCP tcp://127.0.0.1:5760, UDP outbound udpout://vehicle-gateway:14550, or serial radio /dev/ttyUSB0:57600.
2. Open Workstations, expand Views, and select Robotics Viewer.
3. Enter the MAVLink endpoint and an optional vehicle label.
4. Click Test heartbeat for a one-shot connectivity check, or Connect telemetry to start the background helper.
5. Watch the live telemetry panel for heartbeat, position, battery, attitude, GPS, status text, and command messages.
6. Use guarded commands only for owned/authorized simulators or vehicles. Commands require an authorization checkbox and exact text confirmation.

Hayabusa Native GCS, ADS-B, and Remote ID

The Robotics Viewer is Hayabusa's native ground-control station for licensed-safe drone operations without bundling Qt/QGroundControl or GPL Mission Planner. It includes a drone world map for ownship MAVLink position, ADS-B aircraft decoded by dump1090, and Remote ID drone tracks ingested from an OpenDroneID/DroneScanner-style bridge. It is designed for tenant-scoped telemetry, operator safety gates, audit logs, and browser-native mission/control workflows.

• Native flight deck: Hayabusa supports flight mode changes, parameter fetch/edit, command ACK display, mission download, mission clear, guarded goto, and guarded waypoint mission upload through the active MAVLink helper.
• Optional QGroundControl / Mission Planner: Hayabusa can still store tenant-scoped launcher URLs for external or Incoming-hosted ground-station desktops. These are optional compatibility tools, not bundled dependencies. QGroundControl requires Qt licensing care; Mission Planner is GPLv3 and should remain external/separate.
• dump1090 / RTL-SDR: the container includes rtl-sdr and dump1090-mutability where installed. Hayabusa can start/stop a tenant-scoped dump1090 background receiver bound to loopback, reads its aircraft JSON, and plots decoded ADS-B aircraft.
• OpenDroneID / DroneScanner / OpenSpyglass-style Remote ID: Hayabusa exposes a tenant-scoped ingest endpoint for decoded Remote ID tracks. A local bridge process can listen on an Alfa Wi-Fi card or Bluetooth receiver, decode Remote ID packets, and POST normalized track JSON into Hayabusa.
• Safety controls: writes such as mode changes, parameter edits, mission clear, goto, arm/takeoff/land/RTL, and mission upload require explicit authorization and confirmation text.
• Multi-drone fleet: save multiple vehicles (id, label, MAVLink endpoint, camera URL) per tenant with /api/robotics/fleet. Each vehicle gets its own MAVLink helper session. Pass vehicle_id on connect, commands, missions, and native actions, or choose the active vehicle in the Robotics Viewer fleet panel.
• Gamepad / joystick: the browser Gamepad API sends MAVLink MANUAL_CONTROL to the active vehicle after guarded confirmation text enable joystick. Use only on authorized aircraft in a suitable manual flight mode with a visual observer and emergency recovery plan.
• Cameras: configure per-vehicle camera_url values (HTTP/HTTPS MJPEG, RTSP, HLS, or embed). Hayabusa proxies MJPEG/RTSP through /api/robotics/camera/<vehicle_id>/stream for same-origin viewing. Do not embed credentials in camera URLs.

NVIDIA Isaac remote integration

The NVIDIA ISAAC Workstations view connects Hayabusa to a remote NVIDIA Isaac server that you provide and license separately. This is designed for environments where Hayabusa handles fleet, ROS2, MAVLink, SCADA, observability, and policy context while NVIDIA Isaac / Isaac Sim / Isaac ROS handles GPU-heavy robotics workloads such as simulation, perception pipelines, synthetic data, model evaluation, and Omniverse-based visualization.

How to connect a remote Isaac server

1. Deploy and license NVIDIA Isaac, Isaac Sim, Isaac ROS, Omniverse Streaming, CUDA, GPU drivers, and any NVIDIA container/runtime pieces on your GPU host according to NVIDIA's terms.
2. Expose an authenticated HTTPS Isaac or streaming UI endpoint to the Hayabusa operator network. Do not put usernames, passwords, API keys, or tokens in URLs.
3. Open Workstations, expand Views, and select NVIDIA ISAAC.
4. Enter the Isaac server URL, optional streaming/UI URL, optional ROS bridge URL such as ws://isaac-host:9090, ROS domain ID, and operator notes.
5. Click Save and test. Hayabusa stores the connection in the current user or organization scope and checks remote reachability.
6. Use the embedded frame or Open remote link to access the remote Isaac UI, then bridge telemetry/control through your authorized ROS2/MAVLink network.

The backend endpoint /api/isaac/status supports GET and POST. It stores URLs per tenant scope, rejects URLs with embedded credentials, rejects loopback/link-local/metadata targets, and reports reachability without storing secrets.

Open-source and commercial-use posture

The recommended stack is open source and can be used commercially, but some pieces carry copyleft obligations if you modify and redistribute them.

• FUXA: MIT licensed core project; commercial use is permitted with notice retention.
• ROS2: mostly Apache-2.0/BSD/MIT by package; verify package-specific notices for any extra ROS packages you add.
• MAVLink: generator is LGPLv3-family, while generated protocol code includes a permissive MIT exception.
• MAVSDK: BSD-3-Clause; commercial use is permitted with license/disclaimer retention.
• MAVROS: triple-licensed BSD/GPLv3/LGPLv3; use the license path that matches your distribution model.
• ArduPilot SITL / DroneKit SITL: ArduPilot is GPL-3.0-or-later. Hayabusa treats it as a separate unmodified simulator process controlled over MAVLink. Redistributing the image requires GPL notices/source availability for ArduPilot/DroneKit SITL, but this separation does not require publishing Hayabusa source code.
• PX4: BSD-3-Clause and commercially usable with notice retention. PX4 is supported as an external MAVLink endpoint and is not bundled in the current image.
• QGroundControl: dual Apache-2.0/GPLv3. The Apache-2.0 path is compatible with proprietary deployments but requires appropriate Qt licensing if you build/distribute it that way; GPL builds should remain separate/external if you do not want copyleft obligations on a combined work.
• Mission Planner: GPLv3. Hayabusa treats it as an external/separate ground station and does not link it into Hayabusa.
• dump1090 / rtl-sdr: useful for passive ADS-B receiving. Common dump1090 packages are GPL lineage; Hayabusa consumes JSON output over a process/data boundary.
• OpenDroneID: Apache-2.0 reference receivers and core libraries are suitable for commercial integration with notice retention. Verify the license of any specific DroneScanner/OpenSpyglass desktop build before bundling it.
• OpenPLC Editor / MatIEC / Beremiz lineage: commercial use is permitted, but GPL/copyleft terms can require source availability for modified redistributed tooling. PLC programs you author are not automatically forced open by using the editor.
• NVIDIA Isaac: not bundled by Hayabusa. Commercial use depends on NVIDIA's current license terms for the specific Isaac/Omniverse/CUDA/GPU components you deploy remotely.

Relevant APIs

• /fuxa/: same-origin embedded FUXA UI proxy used by the SCADA view.
• /api/scada/status: FUXA service URL, reachability, and license summary for the SCADA view.
• /api/robotics/status: ROS2 and drone tooling availability for Robotics Viewer.
• /api/robotics/mavlink/status: current tenant-scoped MAVLink endpoint, helper-process status, and latest telemetry snapshot.
• /api/robotics/mavlink/test: one-shot heartbeat test for a MAVLink endpoint.
• /api/robotics/mavlink/connect: save the endpoint and start the MAVLink telemetry helper.
• /api/robotics/mavlink/disconnect: stop the active MAVLink telemetry helper.
• /api/robotics/mavlink/command: guarded arm/disarm/takeoff/land/return-to-launch/goto command endpoint.
• /api/robotics/mavlink/native: native GCS endpoint for parameter fetch/edit, flight mode change, mission download, and mission clear.
• /api/robotics/mavlink/mission: guarded waypoint mission upload endpoint.
• /api/robotics/ground-control/status: tenant-scoped QGroundControl, Mission Planner, dump1090, and Remote ID bridge launcher/status endpoint.
• /api/robotics/airspace/status: combined ownship, ADS-B, Remote ID, and mission map data endpoint.
• /api/robotics/remote-id/ingest: decoded Remote ID track ingest endpoint for OpenDroneID/DroneScanner bridge clients.
• /api/robotics/adsb/status, /api/robotics/adsb/start, and /api/robotics/adsb/stop: tenant-scoped dump1090 receiver control and status endpoints.
• /api/robotics/sitl/status: current tenant-scoped ArduPilot SITL process status, endpoint, and compliance summary.
• /api/robotics/sitl/start: start tenant-scoped ArduPilot Copter SITL with a validated home latitude/longitude.
• /api/robotics/sitl/stop: stop the active tenant-scoped ArduPilot SITL process.
• /api/isaac/status: tenant-scoped remote NVIDIA Isaac connection save/test/status endpoint.
• /api/plc/drivers/status: installed driver and write-support status.
• /api/plc/read: unified multi-vendor read endpoint.
• /api/plc/write: guarded multi-vendor write endpoint.
• /api/plc/modbus/read: compatibility endpoint for Modbus TCP read workflows.

13) Profile

Profile is where you verify identity, provider linkage, personal settings, AI keys, and usage/cost context. If actions seem unexpected — wrong org data, missing keys, denied APIs, or unexpected token spend — check Profile first.

How the page is organized

Profile uses categories and tabs so related controls stay grouped (identity, API/AI keys, notifications/preferences, and usage/billing). Use the tabs rather than scrolling one long list.

Typical tasks

• Confirm logged-in identity and linked providers (Google, Slack, Discord, Microsoft).
• Manage personal AI keys used by the assistant and alert-suggest flows.
• Review usage and costs for your account (included tokens, spend, overage signals).
• Adjust preferences that affect notifications or observability scope.

14) Organizations

Organizations provide scope boundaries and team ownership. Structure orgs to mirror real operational responsibility (site, product line, customer) — not arbitrary folders.

What org scope affects

• Fleet/Workstations inventory visibility and claim rights.
• Ansible/OpenTofu and openPLC/ workspace trees (physically separate from personal).
• Observability Grafana org / Prometheus tenant filters / Influx buckets.
• Robotics, Isaac, and other tenant-scoped connections.

Operator guidance

• Grant least privilege; separate read-only members from writers.
• Document who owns each org and which controllers/sites attach to it.
• Never run destructive playbooks until the org selector matches the intended blast radius.

15) Administration

Administration controls global platform behavior — safety toggles, identity posture, and operational visibility. Treat admin actions as reversible control-plane operations with a defined owner and verification signal.

Before changing global toggles, define: expected blast radius, rollback condition, and how you will know it worked (login success rate, tunnel health, host freshness, or scrape health).

Subsections in this category

• 15.1 Safety Controls — sign-in, Headscale, bans/allowlists, throttling.
• 15.2 Manual Identity Controls — local/fallback auth paths.
• 15.3 Logs and Operational Visibility — audit and accountability.

15.1) Safety Controls

Safety controls exist to protect platform integrity during incidents, abuse events, and maintenance windows.

What each control does

• Sign-in toggle: temporarily blocks new sign-ins (preserves emergency paths configured by policy).
• Headscale toggle: disables/reenables overlay control behavior during network investigations.
• Bandwidth throttling controls: operator-level constraints for controlled behavior.
• Bans: prevent specific identities from accessing the platform.
• Allowlists: define explicit identities permitted under tighter posture.

How to operate safety controls

1. Define incident objective first (containment, recovery, investigation).
2. Apply minimum required control change.
3. Document who changed what and why.
4. Monitor service behavior.
5. Revert controls when safe and record closure notes.

15.2) Manual Identity Controls

Manual identity tooling supports controlled local auth workflows and admin fallback when federated providers (Google, Slack, Discord, Microsoft) are unavailable or policy requires an emergency path.

When to use it

• Provider outage or misconfiguration blocks normal sign-in.
• Policy requires a temporary local account for a controlled maintenance window.
• Incident response needs a known-good admin identity independent of IdP health.

Recommended usage

• Enable only when required by policy or incident response.
• Use strong credentials; rotate after high-risk windows.
• Disable fallback paths when no longer needed; record who enabled them and why.
• Prefer federated login for day-to-day users to reduce password custody on Core.

15.3) Logs and Operational Visibility

Admin-level visibility includes audit history, safety actions, and network-related event logs. Use logs for accountability and root-cause analysis — not only for “when something breaks.”

Minimum logging discipline

• Record actor, action, target, and reason for safety and identity changes.
• Correlate changes with incident timelines (and with Grafana/Prometheus when metrics moved).
• Store enough context for post-incident review; avoid logging secrets or raw credentials.
• Review PLC write audits and robotics command audits after any guarded write session.

16) Architecture and Runtime

Hayabusa Core runs as a containerized application (Flask/gunicorn in typical deploys) composed of templates, static assets, and route handlers. Controllers are separate site appliances (API, DHCP/PXE edge, secrets, Image Nest, LAN Approve). Operational behavior is always code + persisted state.

When debugging, verify both: “Is the code path deployed?” and “Is runtime state what we think it is?” Map telemetry is sampled asynchronously and rendered as periodic snapshots — freshness and sampler heartbeat distinguish a quiet network from a stale pipeline.

Core vs Controller (one diagram in words)

• Core — public/brain: identity, Fleet/Workstations, GitOps agent, OpenTofu/Ansible orchestration, observability proxies.
• Controller — site edge: dnsmasq DHCP/PXE, /ztp/fetch, secrets vault, Image Nest, job hydrate on LAN Approve.
• Git — optional Advanced source of truth; not the LAN boot pipe.

17) Persistence and State

Hayabusa persists key operational state to disk volumes: identity linkage, tenant network allocations, safety policy, user-scoped encrypted materials, workspaces, and controller data (leases, nest bank, vault).

Know which controls survive restarts and which are session-derived. Some diagnostics (immediate baseline compare, transient path highlights) are intentionally runtime-scoped for speed. If compliance needs durable history, add explicit storage policy rather than assuming every visualization is persisted.

Examples

• Persists — org memberships, tenant CIDRs, Image Nest bank on the controller volume, Grafana orgs/datasources (as provisioned).
• Session / ephemeral — browser session cookies, some map overlays, in-memory baselines.

18) Common Runbooks

Runbook: New operator onboarding

1. Verify identity and role assignment (Profile + org selector).
2. Walk Workstations → Fleet → Networking in read-only mode (§3–§6).
3. Execute one approved low-risk action (refresh, inspect, AI draft review).

Runbook: Contain suspicious account activity

1. Apply account ban/allowlist controls (Admin safety).
2. If needed, disable new sign-ins temporarily — keep break-glass admin.
3. Review audit and network logs.
4. Document and restore normal posture once resolved.

Runbook: Tenant subnet issue

1. Review tenant allocation records (§6.3).
2. Validate overlap constraints and expected ownership.
3. Apply corrected assignment with validation.
4. Confirm connectivity and overlay/IPsec behavior.

Runbook: Overlay tunnel slow but not down

1. Open Workstations Overlay and inspect link health color/state.
2. Open Underlay and confirm whether adjacent links show elevated jitter/loss.
3. Run Path Trace between affected source/destination IPs.
4. Use telemetry drawer top talkers to identify likely traffic contributors.
5. Apply smallest corrective action first (policy, route, selector, or rate control), then re-verify health trend.

Runbook: Bank Windows/Linux media then create a VM

1. On the controller, open Image Nest (/image-nest) and upload a legal ISO (§7.1).
2. Choose deploy target (bare metal / hypervisor media / bootable / Hyper-V / VMware) and Strip or Full bank.
3. Commit OpenTofu with <<IMAGE_NEST:…>> placeholders only.
4. Queue from Hayabusa; LAN Approve on the controller; apply on Core.
5. Verify guest boot; remember empty disk stubs are not a runnable OS.

19) Troubleshooting

Symptom: Page looks outdated or broken

• Hard refresh browser assets (Ctrl/Cmd+Shift+R).
• Verify auth session and route existence.
• Check whether Core was restarted after template/image changes.

Symptom: Host list or network status inconsistent

• Refresh discovery and overlay states.
• Validate safety toggles and Headscale/WireGuard posture.
• Inspect recent admin actions in logs.
• Confirm organization selector matches the inventory you expect.

Symptom: Access unexpectedly denied

• Check role/scope context (Profile + org).
• Verify active bans/allowlist policy.
• Confirm admin identity matching policy inputs.

Symptom: Link health appears severe but throughput is high

• Check whether loss or jitter thresholds are tripping critical-state logic.
• Confirm telemetry freshness and sampler heartbeat before acting.
• Correlate with top-talker behavior and path trace results.
• Treat this as quality degradation, not capacity success.

Symptom: OpenTofu/Ansible cannot find an image or secret

• Confirm placeholders are still in git (not real paths/passwords).
• Confirm the job was LAN Approved on the controller that holds the vault/Nest bank.
• Check Image Nest registry aliases (§7.1) and secret names on the controller.

Symptom: /docs section missing after an update

• Hard refresh; open the section hash directly (e.g. /docs#image-nest).
• Confirm Core image/template includes the latest docs.html (DEV_SYNC or rebuild).

20) Best Practices

• Separate hosts (using Opentofu) and their data (Ansible) to facilitate easy recreation, replication, and to protect against user error.
• Test before deploying Consider testing scripts before deploying. You can always put a controller in a test environment to test on low-risk hosts first.
• Make scope explicit before every change (user vs org, site vs lab).
• Prefer staged rollouts over broad immediate actions; use OpenTofu plans and Ansible check modes where available.
• Least privilege for accounts, org roles, AI keys, and all credentials.
• Placeholders only in git — prefer {{ hayabusa_secret:KEY }} or <<SECRET:KEY>> (plus <<IMAGE_NEST:…>>); hydrate on LAN Approve.
• Auditability by default — who changed what, when, and why.
• Environment-specific runbooks — this manual is general; your CIDRs, change windows, and owners are local.
• Use AI to draft, humans to approve — especially for OpenTofu, Ansible, PLC, and MAVLink commands.
• Empty Nest stubs are not OS images — know ISO vs WIM vs bootable disk vs VHDX/VMDK before attaching media.

21) FAQ

Does this page include API route listings?

No. This page is operational and conceptual. Use /api for route lists and endpoint details.

Can new users become productive from this guide?

Yes — start with §3 onboarding, then Ansible/OpenTofu (§4.2) and Fleet (§5). Teams should still keep local runbooks for environment-specific standards.

What is the fastest way to avoid mistakes?

Verify identity scope and target scope (org, host, CIDR, PLC/drone) before executing changes. Prefer dry-runs and LAN Approve for secrets/media.

Do I need a Hayabusa Controller?

Core alone supports sign-in, maps, and crafting. A controller on the site LAN is required for DHCP/PXE ZTP, Image Nest banking, and LAN Approve hydration of secrets and nest refs.

OpenTofu vs Ansible — which do I use?

OpenTofu when the resource must converge to a declared state (often VMs/cloud/objects). Ansible when you need procedural configuration with softer failure semantics. Many stacks use both.

Where do Windows/Linux ISOs get prepared?

On the controller Image Nest page (/image-nest). See §7.1 for deploy targets (bare metal, hypervisor media, bootable disk, Hyper-V, VMware).

Why is my Grafana dashboard empty?

Confirm personal vs organization scope, datasource provisioning, time range, and tenant labels — then check Prometheus targets. See §11.

How do I contact Tracedroute support?

Two options: open an in-app support ticket on Service Orders, or email support@tracedroute.net.

22) Derivatives

In Hayabusa, a derivative is any reusable operational unit that originates from core capabilities without becoming a separate product. Derivatives reuse Hayabusa’s identity, network, governance, and operational model.

Examples

• Workspace templates and runbook variants (Ansible/OpenTofu packs).
• Policy overlays (safety, allowlists, tenant conventions).
• Integration adapters (GitOps layouts, vendor ZTP packs).
• Tenant-specific workflow packs and deployment conventions.
• Image Nest bake recipes and Autounattend profiles reused across sites.

How to manage derivatives well

• Keep each derivative versioned, scoped (user/org/site), and owned.
• Document where it diverges from base platform expectations.
• Prefer placeholders for secrets and Image Nest refs so packs stay portable.
• Before adoption, verify security assumptions, identity scope, network boundaries, and rollback strategy.

23) About

Hayabusa is an infrastructure command center built around open-source building blocks — Ansible, OpenTofu, WireGuard/Headscale, Grafana/Prometheus/Influx, OpenPLC, ROS2/MAVLink tooling, and more — with a unified identity, network, and governance model for individuals and organizations.

The platform is designed so you may never use every surface. Start from onboarding (§3), learn Ansible/OpenTofu (§4.2), then deepen into the categories that match your role (networking, ZTP/Nest, observability, SCADA/robotics, admin).

For reusable packs built on top of Hayabusa, see §22 Derivatives. For bundled component licenses, see §24 Open Source notices.

24) Open Source & Third-Party Notices

Hayabusa ships with and invokes many open-source components across SECops, DevOps, ZTP, observability, remote access, and industrial/robotics tooling. This section surfaces license and trademark disclosures so operators can redistribute or audit compliance correctly.

Download the full notices file, or read the embedded summary below. Component-specific boundaries (for example ArduPilot SITL as a separate process) are also called out in domain sections such as SCADA / PLC / ROS2.

Standalone page: /open-source-notices