diff --git a/A-Team-JARVIS_AI-v2.2.7z b/A-Team-JARVIS_AI-v2.2.7z new file mode 100644 index 0000000..d732d39 Binary files /dev/null and b/A-Team-JARVIS_AI-v2.2.7z differ diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100755 index 0000000..9d0031a --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,6 @@ +## v2.2 +- Initial GTK3 GUI App Code Refactor +- Public Release v2.2 + + + diff --git a/README.md b/README.md index b63b1bf..18e71f4 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,755 @@ -# A-Team-Jarvis_AI +A-Team JARVIS AI -Give your Local AI a full stack: memory, voice, face, and hands. This is the "I want an AI agent" shortcut. It sets up the entire A-Team Local AI stack for you with an installation wizard. Select which pieces you want or do it all! \ No newline at end of file +Local-first JARVIS desktop agent using Ollama/Qwen, optional local voice, optional AI visuals, Barehands hand tracking, persistent memory, and optional web research. + +This README documents the current integrated v2.2 package, including the GTK3 desktop shell, embedded visualizer/Barehands panels, local voice pipeline, diagnostic logging, desktop launcher, service recovery, and the current Barehands tracking integration. + +1. What This Package Is + +A-Team JARVIS AI is a local desktop agent built around a Python runtime and local Ollama model service. + +The package combines: + +JARVIS agent runtime — Python + Ollama/Qwen + +GTK3 desktop GUI — PyGObject + +Local voice — Whisper/faster-whisper input + Kokoro output + +AI Visualizer — configurable animated JARVIS faces + +Barehands — webcam-based hand-tracked interface + +Persistent Memory Vault — local Markdown-based memory + +Research tools — local file/archive tools plus optional web research + +Durable diagnostics — rotating file logs and crash logging + +Desktop launcher creation — .desktop files for the Linux desktop and application menu + +Release/update checking — Updates/version.json on the configured Releases branch + +The design is local-first. Ollama, voice processing, visualizer state, Barehands state, and the main agent runtime operate on the local machine. Internet access is optional and is controlled separately for research and for the external assets used by Barehands. + +2. Current v2.2 Desktop GUI + +The GTK3 shell is implemented in: + +scripts/jarvis_gui.py + +Launch it with: + +./start_gui.sh + +or: + +./venv/bin/python scripts/jarvis_gui.py + +Main GUI layout + +Left side: Agent Options and JARVIS chat + +Agent Options: + +Voice + +Visuals + +Barehands + +Internet + +Diagnostics + +Outline/theme selection + +Desktop launcher creation + +Setup + +Start/Stop JARVIS + +Right side: optional Face and Barehands panes + +Face selector: visible when Visuals is selected + +Voice selector: visible when Voice is selected + +Face pane: embedded with WebKitGTK when available + +Barehands pane: embedded with WebKitGTK when available + +Browser fallback: used when an embedded WebKit view is unavailable + +Status indicator: shows Stopped, Starting, Running, Working, or Stopping states + +Session startup behavior + +The GUI intentionally starts each new GUI session with the optional runtime features turned off: + +Voice + +Visuals + +Barehands + +Internet + +Diagnostics + +This prevents stale feature selections from unexpectedly launching optional services when the GUI is opened. + +Persistent settings such as the selected face, voice, and GUI theme are still retained. + +When the user presses Start JARVIS, the currently selected feature toggles are written to config/jarvis.json and used for that runtime session. + +3. JARVIS Runtime + +The main agent is: + +scripts/jarvis.py + +The normal launcher is: + +./scripts/start_jarvis.sh + +The launcher: + +Validates the project virtual environment. + +Activates the virtual environment. + +Installs the selected core/optional requirements. + +Reads runtime feature selections from config/jarvis.json. + +Initializes the visualizer and Barehands state files. + +Checks the local Ollama service. + +Replaces a stale user-owned Ollama process when its working directory has become invalid. + +Starts Ollama locally when necessary. + +Starts the AI Visualizer when enabled. + +Starts Barehands when enabled. + +Starts the JARVIS Python agent. + +Service recovery + +The launcher contains checks for stale services from previous extracted/copied installations. + +For the AI Visualizer, it does not blindly trust a successful /state response. It also verifies the configured face URL and the visualizer health response before accepting an existing server. + +This prevents a stale visualizer process from another copy of the project from serving the wrong document root and producing a literal 404/not found face page. + +Barehands similarly validates the stage endpoint and replaces stale Barehands server instances when necessary. + +4. GUI Status States + +The GUI reports the runtime state at the top of the Agent Options area. + +Typical states are: + +JARVIS Stopped +JARVIS Starting.. +JARVIS Running +JARVIS Working… +JARVIS Stopping… + +The GUI changes to Working while JARVIS is listening/processing and returns to Running after a completed JARVIS: response. + +Stopping JARVIS removes the live Face/Barehands panes while retaining the user's checkbox selections for the next run. + +5. Voice + +Voice is optional and is controlled by: + +"voice": { + "enabled": true +} + +The voice pipeline uses local components: + +Whisper/faster-whisper for speech recognition + +Kokoro for local text-to-speech + +sounddevice/soundfile and related local audio packages + +The current voice requirements deliberately pin the versions associated with the working Whisper/AV integration: + +faster-whisper==1.1.1 +av==12.3.0 + +This avoids the metadata_errors incompatibility that occurred with the affected newer AV/faster-whisper combination. + +Thinking sound + +The thinking sound is now coordinated with the JARVIS backend. + +JARVIS writes: + +ai-visualizer/.voice_loading_pid + +while the backend thinking sound is active. + +The visualizer respects that marker so the browser-side thinking sound does not play simultaneously with the backend sound. + +The thinking sound asset is: + +ai-visualizer/assets/thinking.wav + +6. AI Visualizer + +The visualizer lives under: + +ai-visualizer/ + +and runs on: + +http://127.0.0.1:8790/ + +The visualizer is an embedded upstream component by Jared Rhodenizer (@jaredrhod). + +Upstream project: + +https://github.com/jaredrhod/ai-visualizer + +The package retains the upstream copyright/license notices in the visualizer source. + +Included faces + +The current package contains: + +rain +a-team +a-team-android +a-team-android-1 +a-team-command +a-team-moto-pad +a-team-ops +board +neural +radial + +The configured face is selected through: + +"visuals": { + "face": "board" +} + +Visualizer signal bus + +The face reads the local state bus: + +ai-visualizer/.voice_state +ai-visualizer/.voice_waveform +ai-visualizer/.voice_loading_pid + +States include: + +idle +listening +thinking +speaking + +Current visualizer logging + +Routine /state polling is intentionally not written as individual log lines. + +The visualizer logs useful events instead: + +startup + +configured face + +WebView/face boot + +configuration loading + +state transitions + +duration of the previous state + +periodic heartbeat + +JavaScript errors + +unhandled promise rejections + +state-poll failures + +audio/microphone problems + +thinking-sound initialization/playback problems + +HTTP failures + +The visualizer log is: + +logs/faces.log + +This keeps the log useful instead of filling it with hundreds of normal polling requests. + +7. Barehands + +Barehands lives under: + +barehands/ + +and runs on: + +http://127.0.0.1:8794/stage.html + +Barehands is an upstream component by Jared Rhodenizer (@jaredrhod). + +Upstream project: + +https://github.com/jaredrhod/barehands + +The package retains the upstream copyright and AGPL license notices in the Barehands source. + +Barehands uses the webcam to track hand position and provides the interactive board/ring interface. + +Embedded camera support + +When WebKitGTK is available, the GTK GUI embeds Barehands directly. + +The GUI explicitly handles WebKitGTK's user-media permission request and allows the Barehands video request for the dedicated Barehands WebView. + +This is required because WebKitGTK otherwise denies getUserMedia() requests by default. + +The embedded settings enable the WebGL/media functionality required by the stage. + +If WebKitGTK is unavailable, the GUI provides the browser fallback. + +Current hand-tracking implementation + +The current integrated package uses the legacy browser MediaPipe Hands runtime rather than the Tasks Vision HandLandmarker path that was previously tested. + +The package currently loads: + +@mediapipe/hands@0.4.1675469240 + +and configures tracking for up to two hands. + +The tracker is asynchronous and throttled so it does not attempt to process every camera frame at full camera rate. + +The current implementation records detailed tracker information specifically so two-hand tracking can be diagnosed without filling the log with routine HTTP traffic. + +Barehands logging + +The Barehands log is: + +logs/barehands.log + +Routine /state and /orb polling is intentionally quiet. + +Useful tracker information includes: + +camera initialization + +capture resolution + +tracker initialization + +tracker errors + +0 → 1 → 2 hand transitions + +left/right hand identification + +confidence values + +bounding boxes + +landmark positions + +wrist/index positions + +pinch gap + +tracker latency + +tracker FPS + +periodic heartbeat + +JavaScript errors + +HTTP failures + +A two-hand test should produce compact records similar to: + +TRACK hands=2 Left:0.91,Right:0.87 +TRACK hands=2 Left:0.90,Right:0.84 +TRACK hands=1 Left:0.92 +TRACK hands=0 + +This makes it possible to determine whether a second hand is being missed because of detection confidence, tracking loss, hand size/distance, camera conditions, or another runtime problem. + +8. Barehands State Bus + +Barehands uses: + +barehands/state/state +barehands/state/mood.json +barehands/state/wave.json + +The basic assistant states are: + +idle +listening +thinking +speaking + +The state timeout prevents a dead session from leaving the assistant ring permanently stuck in a non-idle state. + +The Barehands board also supports the bundled command helpers: + +barehands/bin/board.sh +barehands/bin/board-state.sh + +The bundled upstream documentation explains the board protocol, gestures, media airlock, and assistant integration. + +9. Persistent Memory + +Persistent memory lives under: + +Memory_Vault/ + +The project uses local Markdown files rather than a hosted memory service. + +Relevant project files include: + +Memory_Vault/VAULT-INDEX.md +Memory_Vault/Active Priorities.md +Memory_Vault/01 - Daily Notes/ +Memory_Vault/02 - Profile/ +Memory_Vault/04 - Knowledge/ + +The memory implementation is: + +scripts/memory.py + +The project documentation notes that this architecture is inspired by Jared Rhodenizer's AI Memory Vault. + +10. Research and Tools + +The project includes: + +scripts/agent_tools.py +scripts/research_tools.py + +The runtime can work with: + +local files + +project files + +archives + +local research sources + +optional web research + +Internet research is controlled separately through: + +"research": { + "internet": true +} + +Turning Internet off does not disable durable local logging or the local agent. + +11. Diagnostics and Logging + +JARVIS has two different concepts: + +Console diagnostics + +Controlled by: + +"diagnostics": { + "enabled": true +} + +When enabled, diagnostic records are also visible in the terminal/GUI output. + +Durable file logging + +Durable logging remains active regardless of the Diagnostics toggle. + +The centralized logger is: + +scripts/logging_config.py + +Primary logs include: + +logs/jarvis.log +logs/crash.log +logs/faces.log +logs/barehands.log +logs/ollama.log + +The purpose of the Diagnostics toggle is therefore visibility, not disabling background logging. + +Crash handling also captures uncaught main/thread exceptions and enables Python fault diagnostics. + +12. GUI Themes + +The GTK GUI uses: + +assets/ateam_header.png + +and the supplied banner artwork for the selectable outline theme. + +Available themes: + +Blue + +Red + +Green + +Purple + +Orange + +Cyan + +Pink + +The selected theme is saved under: + +"gui": { + "outline_color": "Cyan" +} + +The outline is applied to the Agent Options, Agent Chat, and active right-side panels. + +13. Face and Voice Selectors + +Face selection is stored under: + +"visuals": { + "face": "board" +} + +Voice selection is stored in the GUI's persistent configuration. + +The selector controls are conditional: + +Face selector appears when Visuals is enabled. + +Voice selector appears when Voice is enabled. + +The actual Face and Barehands panes do not start merely because their checkboxes are selected. They become live after Start JARVIS launches the backend. + +14. Face/Barehands Pane Controls + +Both right-side panels have small maximize controls in their upper-right corner. + +Maximizing one optional panel gives it the available right-side workspace while hiding the other optional panel. + +Restoring the normal view returns the layout to approximately the original right-pane allocation. + +The controls are intentionally compact so they do not cover the visualizer/Barehands content. + +15. Desktop Launcher + +The GUI provides: + +Create Desktop Launcher + +It creates: + +~/Desktop/A-Team-JARVIS-AI.desktop +~/.local/share/applications/A-Team-JARVIS-AI.desktop + +The launcher: + +uses assets/desktop_icon_jarvis.png when available + +falls back to assets/desktop_icon.png + +starts the root start_gui.sh + +is marked executable + +The generated launcher does not replace the running installation. + +16. Configuration + +The main runtime configuration is: + +config/jarvis.json + +Important settings include: + +voice.enabled +visuals.enabled +visuals.barehands_enabled +visuals.face +research.internet +diagnostics.enabled +gui.outline_color + +The GUI is the normal interface for changing these values. + +Feature toggles are treated as session controls: a new GUI launch starts with optional features off, and the current selections are committed when JARVIS is started. + +17. Setup + +Run: + +./scripts/setup_jarvis.sh + +The setup process creates or reuses: + +venv/ + +and installs dependencies based on the selected features. + +Core requirements: + +requirements/requirements-core.txt + +Voice requirements: + +requirements/requirements-voice.txt + +Web/research requirements: + +requirements/requirements-web.txt + +The setup process can also check Ollama and optionally pull the configured model. + +18. Project Layout + +A-Team-JARVIS_AI/ +├── start_gui.sh +├── scripts/ +│ ├── jarvis.py +│ ├── jarvis_gui.py +│ ├── agent_tools.py +│ ├── research_tools.py +│ ├── memory.py +│ ├── local_voice.py +│ ├── logging_config.py +│ ├── start_gui.sh +│ ├── start_jarvis.sh +│ ├── setup_jarvis.sh +│ ├── select_jarvis_face.sh +│ └── select_jarvis_voice.sh +├── config/ +│ ├── jarvis.json +│ ├── system_prompt.md +│ └── .aider.conf.yml +├── ai-visualizer/ +├── barehands/ +├── Memory_Vault/ +├── Project_Folder/ +├── requirements/ +├── logs/ +├── Updates/ +├── assets/ +├── docs/ +└── VERSION + +The root keeps the simple GUI launcher. Project Python modules are under scripts/, user-facing Bash utilities/selectors are under scripts/, and component servers remain in their component directories. + +19. Updates + +The GUI checks the dedicated Releases branch: + +https://ateam.gotadell.com/git/PizzaG/A-Team-Jarvis_AI/raw/branch/Releases/Updates/version.json + +Release archives are expected on that same branch. + +The GUI checks the archive at the branch root first and then under Updates/. + +Current metadata format: + +{ + "version": "1", + "display": "2.2", + "file": "A-Team-Jarvis_AI-v2.2.7z", + "changelog": "Initial GTK3 GUI Code Refactor, Public Release v2.2" +} + +Downloaded release archives are stored under: + +Updates/ + +The GUI does not automatically overwrite the running installation. + +20. Current Release + +Version: 2.2 +Primary runtime: Python + Ollama +GUI: GTK3 / PyGObject +Embedded browser: WebKitGTK 4.x +AI Visualizer: 127.0.0.1:8790 +Barehands: 127.0.0.1:8794 + +Version source: + +VERSION + +Release metadata: + +Updates/version.json + +Upstream Attribution + +AI Visualizer + +The AI Visualizer integrated into this package originates from: + +Jared Rhodenizer (@jaredrhod) + +Upstream repository: + +https://github.com/jaredrhod/ai-visualizer + +The upstream visualizer source files retain their copyright and AGPL-3.0-or-later notices. + +Barehands + +The Barehands integrated into this package originates from: + +Jared Rhodenizer (@jaredrhod) + +Upstream repository: + +https://github.com/jaredrhod/barehands + +The upstream Barehands source files retain their copyright and AGPL-3.0-or-later notices. + +Upstream Author + +Jared Rhodenizer's GitHub: + +https://github.com/jaredrhod + +The upstream projects and their source files retain their applicable copyright and license notices. + +Other Upstream Components + +Barehands uses Google MediaPipe for hand tracking and three.js for browser-side 3D rendering. + +The AI Visualizer includes the VT323 typeface. Its applicable font license is preserved at: + +ai-visualizer/assets/VT323-OFL.txt + +The upstream component licenses and notices remain applicable to their respective source files. This project-level README does not relicense those upstream components. + +License + +GNU General Public License v3.0 (GPL-3.0). + +2019-Present — A-Team Digital Solutions diff --git a/Updates/version.json b/Updates/version.json new file mode 100644 index 0000000..718752f --- /dev/null +++ b/Updates/version.json @@ -0,0 +1,6 @@ +{ + "version": "1", + "display": "2.2", + "file": "A-Team-Jarvis_AI-v2.2.7z", + "changelog": "Initial GTK3 GUI App Code Refactor, Public Release v2.2" +}