feat: add hold-to-talk listener and restore Handy-managed preferences
This commit is contained in:
1 parent
0992f3b863
commit
e61f230978
7 files changed
+135
-47
No files matched your search
+16
-12
@@ -119,15 +119,17 @@ reports are not a fresh comparison with this baseline.
|
|||||||
## Open choices and next discussion
|
## Open choices and next discussion
|
||||||
|
|
||||||
Dictation: plain local transcription is selected; rambling cleanup is deferred.
|
Dictation: plain local transcription is selected; rambling cleanup is deferred.
|
||||||
Handy preferences are Nix-owned in `modules/capabilities/handy.nix`: Nemotron
|
Handy preferences are application-owned again: the Nix settings generator and
|
||||||
Streaming 3.5, automatic language detection, default microphone, Live overlay at
|
startup reset are removed. The existing settings file is left untouched.
|
||||||
the bottom, Ctrl+Shift+V paste through wtype, preserved clipboard, manual submission
|
Nix installs Handy/wtype and runs Handy as a graphical-session user service.
|
||||||
and no cloud processing. Other preferences use pinned app defaults. A user service
|
Super+D launches a short-lived Python/evdev listener: hold to record, release D
|
||||||
copies the baseline into Handy's writable store on each start, allowing migrations
|
or Super to finish. Both keys must be on the same keyboard. Niri owns the shortcut
|
||||||
while resetting GUI edits. Models/recordings stay outside Nix; the selected model
|
and disables repeat; no persistent keyboard logger or exclusive input grab runs.
|
||||||
is already downloaded on Zenith. Other hosts still need model provisioning.
|
A udev uaccess rule grants the active local user access to keyboard event devices.
|
||||||
Super+Shift+D toggles; Super+Ctrl+D cancels. Super+D push-to-talk awaits the user's
|
Super+Shift+D toggles; Super+Ctrl+D cancels. Both stop any active hold listener
|
||||||
decision about a keyboard listener: Niri 26.04 has no key-release bindings.
|
before issuing their Handy command, preventing a later release from restarting
|
||||||
|
recording. Begin push-to-talk from idle: Handy exposes toggles, not idempotent
|
||||||
|
start/stop commands. Niri 26.04 itself has no key-release bindings.
|
||||||
Noctalia suppresses microphone activity popups/indicators; camera and screen-sharing
|
Noctalia suppresses microphone activity popups/indicators; camera and screen-sharing
|
||||||
indications remain. Automatic paste and live capture need interactive checks.
|
indications remain. Automatic paste and live capture need interactive checks.
|
||||||
|
|
||||||
@@ -139,9 +141,11 @@ and Vulkan batch mode (156 ms inference, 420 ms load). This checks model inferen
|
|||||||
not live capture/stop/cancel. No new audio was recorded. Switching to CPU is not a
|
not live capture/stop/cancel. No new audio was recorded. Switching to CPU is not a
|
||||||
confirmed fix. After activation, retry live recording and stopping, and inspect
|
confirmed fix. After activation, retry live recording and stopping, and inspect
|
||||||
the service logs if it hangs.
|
the service logs if it hangs.
|
||||||
Verification: formatting, flake evaluation, both workstation builds and whitespace
|
Synthetic listener checks cover release during startup, repeat, modifier-first
|
||||||
checks passed. Handy loaded the generated settings in an isolated data directory
|
release, device disconnect and cancellation, without live input/audio access.
|
||||||
without settings parse errors. System activation remains user-owned and pending.
|
System activation and an interactive hold/release trial remain user-owned.
|
||||||
|
Formatting, flake evaluation, both workstation builds, synthetic listener checks
|
||||||
|
and whitespace checking passed. No live input listener or microphone was started.
|
||||||
|
|
||||||
The focus is the broader environment, not further API/database polish. Candidate
|
The focus is the broader environment, not further API/database polish. Candidate
|
||||||
areas remain places/ownership, navigation/interaction, human/agent collaboration,
|
areas remain places/ownership, navigation/interaction, human/agent collaboration,
|
||||||
|
|||||||
@@ -29,8 +29,8 @@ use a flake to declare its tools without Nix owning the surrounding workspace.
|
|||||||
interaction with useful connections to the rest of the environment.
|
interaction with useful connections to the rest of the environment.
|
||||||
- Use Handy for local plain dictation, mainly English with occasional Dutch.
|
- Use Handy for local plain dictation, mainly English with occasional Dutch.
|
||||||
Use native Wayland automatic paste and optional live text with a streaming model.
|
Use native Wayland automatic paste and optional live text with a streaming model.
|
||||||
Nix owns Handy preferences and desktop integration; the user service restores
|
Handy owns its preferences, models and recordings. Nix installs and starts it
|
||||||
settings on each start. Model downloads and recordings remain application data. Cleanup
|
and supplies desktop shortcuts, including a short-lived push-to-talk listener. Cleanup
|
||||||
through Codex, a cloud API or a local language model is deferred.
|
through Codex, a cloud API or a local language model is deferred.
|
||||||
- Outlook is acceptable as an application, but its interaction and integration
|
- Outlook is acceptable as an application, but its interaction and integration
|
||||||
do not fit as naturally. Accessible mail for tools and agents matters separately
|
do not fit as naturally. Accessible mail for tools and agents matters separately
|
||||||
|
|||||||
@@ -1,30 +1,16 @@
|
|||||||
{
|
{
|
||||||
|
flake.modules.nixos.handy = { pkgs, ... }: {
|
||||||
|
services.udev.packages = [
|
||||||
|
(pkgs.writeTextDir "lib/udev/rules.d/70-handy-keyboard.rules" ''
|
||||||
|
SUBSYSTEM=="input", KERNEL=="event*", ENV{ID_INPUT_KEYBOARD}=="1", TAG+="uaccess"
|
||||||
|
'')
|
||||||
|
];
|
||||||
|
};
|
||||||
|
|
||||||
flake.modules.homeManager.handy =
|
flake.modules.homeManager.handy =
|
||||||
{ config, pkgs, ... }:
|
{ pkgs, ... }:
|
||||||
let
|
let
|
||||||
settings = (pkgs.formats.json { }).generate "handy-settings.json" {
|
python = pkgs.python3.withPackages (ps: [ ps.evdev ]);
|
||||||
settings = {
|
|
||||||
settings_schema_version = 1;
|
|
||||||
selected_model = "handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf/nemotron-3.5-asr-streaming-0.6b-Q8_0.gguf";
|
|
||||||
selected_language = "auto";
|
|
||||||
translate_to_english = false;
|
|
||||||
selected_microphone = null;
|
|
||||||
always_on_microphone = false;
|
|
||||||
push_to_talk = true;
|
|
||||||
paste_method = "ctrl_shift_v";
|
|
||||||
typing_tool = "wtype";
|
|
||||||
clipboard_handling = "dont_modify";
|
|
||||||
auto_submit = false;
|
|
||||||
overlay_style = "live";
|
|
||||||
overlay_position = "bottom";
|
|
||||||
post_process_enabled = false;
|
|
||||||
start_hidden = true;
|
|
||||||
autostart_enabled = false;
|
|
||||||
update_checks_enabled = false;
|
|
||||||
show_whats_new_on_update = false;
|
|
||||||
onboarding_completed = true;
|
|
||||||
};
|
|
||||||
};
|
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
home.packages = [ pkgs.handy ];
|
home.packages = [ pkgs.handy ];
|
||||||
@@ -37,10 +23,18 @@
|
|||||||
};
|
};
|
||||||
Install.WantedBy = [ "graphical-session.target" ];
|
Install.WantedBy = [ "graphical-session.target" ];
|
||||||
Service = {
|
Service = {
|
||||||
# Handy writes migrations and runtime preferences: restore the Nix baseline on start.
|
|
||||||
ExecStartPre = "${pkgs.coreutils}/bin/install -Dm600 ${settings} ${config.xdg.dataHome}/com.pais.handy/settings_store.json";
|
|
||||||
ExecStart = "${pkgs.handy}/bin/handy --start-hidden";
|
ExecStart = "${pkgs.handy}/bin/handy --start-hidden";
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
|
systemd.user.services.handy-ptt = {
|
||||||
|
Unit = {
|
||||||
|
Description = "Handy push-to-talk key release listener";
|
||||||
|
PartOf = [ "graphical-session.target" ];
|
||||||
|
Requisite = [ "handy.service" ];
|
||||||
|
After = [ "handy.service" ];
|
||||||
|
};
|
||||||
|
Service.ExecStart = "${python}/bin/python ${./handy/_push_to_talk.py} ${pkgs.handy}/bin/handy";
|
||||||
|
};
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
"""Wait for release of the physical Super+D chord that launched this service."""
|
||||||
|
|
||||||
|
import signal
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from contextlib import closing
|
||||||
|
|
||||||
|
from evdev import InputDevice, ecodes as key, list_devices
|
||||||
|
|
||||||
|
|
||||||
|
def held(keys):
|
||||||
|
return key.KEY_D in keys and bool({key.KEY_LEFTMETA, key.KEY_RIGHTMETA} & set(keys))
|
||||||
|
|
||||||
|
|
||||||
|
def interrupted(signum, frame):
|
||||||
|
raise KeyboardInterrupt
|
||||||
|
|
||||||
|
|
||||||
|
def main(handy):
|
||||||
|
signal.signal(signal.SIGTERM, interrupted)
|
||||||
|
for path in list_devices():
|
||||||
|
try:
|
||||||
|
device = InputDevice(path)
|
||||||
|
except OSError:
|
||||||
|
continue # Devices can disappear between enumeration and opening.
|
||||||
|
with closing(device):
|
||||||
|
if not held(device.active_keys()):
|
||||||
|
continue
|
||||||
|
action = "--cancel"
|
||||||
|
try:
|
||||||
|
subprocess.run([handy, "--toggle-transcription"], check=True)
|
||||||
|
# A quick release may already be queued while the CLI starts.
|
||||||
|
if held(device.active_keys()):
|
||||||
|
for event in device.read_loop():
|
||||||
|
if not held(device.active_keys()):
|
||||||
|
break
|
||||||
|
action = "--toggle-transcription"
|
||||||
|
finally:
|
||||||
|
# Disconnects, cancellation and service shutdown discard the recording.
|
||||||
|
subprocess.run([handy, action], check=True)
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
try:
|
||||||
|
main(sys.argv[1])
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
pass
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
"""Synthetic key-state checks; never reads a keyboard or records audio."""
|
||||||
|
|
||||||
|
from unittest.mock import Mock, patch
|
||||||
|
|
||||||
|
import _push_to_talk as ptt
|
||||||
|
|
||||||
|
down = [ptt.key.KEY_LEFTMETA, ptt.key.KEY_D]
|
||||||
|
|
||||||
|
|
||||||
|
def check(states, expected, error=None):
|
||||||
|
device = Mock()
|
||||||
|
device.active_keys.side_effect = states
|
||||||
|
device.read_loop.return_value = iter([object(), object()])
|
||||||
|
if error:
|
||||||
|
device.read_loop.side_effect = error
|
||||||
|
with (
|
||||||
|
patch.object(ptt, "list_devices", return_value=["keyboard"]),
|
||||||
|
patch.object(ptt, "InputDevice", return_value=device),
|
||||||
|
patch.object(ptt.signal, "signal"),
|
||||||
|
patch.object(ptt.subprocess, "run") as run,
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
ptt.main("handy")
|
||||||
|
except (OSError, KeyboardInterrupt):
|
||||||
|
assert error is not None
|
||||||
|
assert [call.args[0][1] for call in run.call_args_list] == expected
|
||||||
|
device.close.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
check([[]], []) # Released before the listener starts: do not record.
|
||||||
|
check([down, []], ["--toggle-transcription", "--toggle-transcription"])
|
||||||
|
check([down, down, down, [ptt.key.KEY_D]], ["--toggle-transcription"] * 2)
|
||||||
|
check([down, down], ["--toggle-transcription", "--cancel"], OSError("unplugged"))
|
||||||
|
check([down, down], ["--toggle-transcription", "--cancel"], KeyboardInterrupt())
|
||||||
|
assert ptt.held([ptt.key.KEY_RIGHTMETA, ptt.key.KEY_D])
|
||||||
|
assert not ptt.held([ptt.key.KEY_LEFTMETA])
|
||||||
|
print("Push-to-talk release, repeat, disconnect and cancellation checks passed")
|
||||||
@@ -36,19 +36,23 @@ in
|
|||||||
};
|
};
|
||||||
|
|
||||||
# Dictation
|
# Dictation
|
||||||
"Mod+Shift+D" = {
|
"Mod+D" = {
|
||||||
spawn = [
|
spawn = [
|
||||||
handy
|
"systemctl"
|
||||||
"--toggle-transcription"
|
"--user"
|
||||||
|
"start"
|
||||||
|
"handy-ptt.service"
|
||||||
];
|
];
|
||||||
_props.repeat = false;
|
_props.repeat = false;
|
||||||
|
_props.hotkey-overlay-title = "Hold to Dictate";
|
||||||
|
};
|
||||||
|
"Mod+Shift+D" = {
|
||||||
|
spawn-sh = "systemctl --user stop handy-ptt.service && ${handy} --toggle-transcription";
|
||||||
|
_props.repeat = false;
|
||||||
_props.hotkey-overlay-title = "Toggle Dictation";
|
_props.hotkey-overlay-title = "Toggle Dictation";
|
||||||
};
|
};
|
||||||
"Mod+Ctrl+D" = {
|
"Mod+Ctrl+D" = {
|
||||||
spawn = [
|
spawn-sh = "systemctl --user stop handy-ptt.service && ${handy} --cancel";
|
||||||
handy
|
|
||||||
"--cancel"
|
|
||||||
];
|
|
||||||
_props.repeat = false;
|
_props.repeat = false;
|
||||||
_props.hotkey-overlay-title = "Cancel Dictation";
|
_props.hotkey-overlay-title = "Cancel Dictation";
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ in
|
|||||||
nixosModules.network-manager
|
nixosModules.network-manager
|
||||||
nixosModules.nftables-firewall
|
nixosModules.nftables-firewall
|
||||||
nixosModules.niri
|
nixosModules.niri
|
||||||
|
nixosModules.handy
|
||||||
nixosModules.notes
|
nixosModules.notes
|
||||||
nixosModules.plymouth
|
nixosModules.plymouth
|
||||||
nixosModules.printing
|
nixosModules.printing
|
||||||
|
|||||||
Reference in new issue
Block a user