feat: add hold-to-talk listener and restore Handy-managed preferences

This commit is contained in:
kiri committed 2026-09-09 20:44:59 +02:00
1 parent 0992f3b863
commit e61f230978
7 files changed
+135 -47

No files matched your search

+16 -12
View File
@@ -119,15 +119,17 @@ reports are not a fresh comparison with this baseline.
## Open choices and next discussion ## Open choices and next discussion
Dictation: plain local transcription is selected; rambling cleanup is deferred. Dictation: plain local transcription is selected; rambling cleanup is deferred.
Handy preferences are Nix-owned in `modules/capabilities/handy.nix`: Nemotron Handy preferences are application-owned again: the Nix settings generator and
Streaming 3.5, automatic language detection, default microphone, Live overlay at startup reset are removed. The existing settings file is left untouched.
the bottom, Ctrl+Shift+V paste through wtype, preserved clipboard, manual submission Nix installs Handy/wtype and runs Handy as a graphical-session user service.
and no cloud processing. Other preferences use pinned app defaults. A user service Super+D launches a short-lived Python/evdev listener: hold to record, release D
copies the baseline into Handy's writable store on each start, allowing migrations or Super to finish. Both keys must be on the same keyboard. Niri owns the shortcut
while resetting GUI edits. Models/recordings stay outside Nix; the selected model and disables repeat; no persistent keyboard logger or exclusive input grab runs.
is already downloaded on Zenith. Other hosts still need model provisioning. A udev uaccess rule grants the active local user access to keyboard event devices.
Super+Shift+D toggles; Super+Ctrl+D cancels. Super+D push-to-talk awaits the user's Super+Shift+D toggles; Super+Ctrl+D cancels. Both stop any active hold listener
decision about a keyboard listener: Niri 26.04 has no key-release bindings. before issuing their Handy command, preventing a later release from restarting
recording. Begin push-to-talk from idle: Handy exposes toggles, not idempotent
start/stop commands. Niri 26.04 itself has no key-release bindings.
Noctalia suppresses microphone activity popups/indicators; camera and screen-sharing Noctalia suppresses microphone activity popups/indicators; camera and screen-sharing
indications remain. Automatic paste and live capture need interactive checks. indications remain. Automatic paste and live capture need interactive checks.
@@ -139,9 +141,11 @@ and Vulkan batch mode (156 ms inference, 420 ms load). This checks model inferen
not live capture/stop/cancel. No new audio was recorded. Switching to CPU is not a not live capture/stop/cancel. No new audio was recorded. Switching to CPU is not a
confirmed fix. After activation, retry live recording and stopping, and inspect confirmed fix. After activation, retry live recording and stopping, and inspect
the service logs if it hangs. the service logs if it hangs.
Verification: formatting, flake evaluation, both workstation builds and whitespace Synthetic listener checks cover release during startup, repeat, modifier-first
checks passed. Handy loaded the generated settings in an isolated data directory release, device disconnect and cancellation, without live input/audio access.
without settings parse errors. System activation remains user-owned and pending. System activation and an interactive hold/release trial remain user-owned.
Formatting, flake evaluation, both workstation builds, synthetic listener checks
and whitespace checking passed. No live input listener or microphone was started.
The focus is the broader environment, not further API/database polish. Candidate The focus is the broader environment, not further API/database polish. Candidate
areas remain places/ownership, navigation/interaction, human/agent collaboration, areas remain places/ownership, navigation/interaction, human/agent collaboration,
+2 -2
View File
@@ -29,8 +29,8 @@ use a flake to declare its tools without Nix owning the surrounding workspace.
interaction with useful connections to the rest of the environment. interaction with useful connections to the rest of the environment.
- Use Handy for local plain dictation, mainly English with occasional Dutch. - Use Handy for local plain dictation, mainly English with occasional Dutch.
Use native Wayland automatic paste and optional live text with a streaming model. Use native Wayland automatic paste and optional live text with a streaming model.
Nix owns Handy preferences and desktop integration; the user service restores Handy owns its preferences, models and recordings. Nix installs and starts it
settings on each start. Model downloads and recordings remain application data. Cleanup and supplies desktop shortcuts, including a short-lived push-to-talk listener. Cleanup
through Codex, a cloud API or a local language model is deferred. through Codex, a cloud API or a local language model is deferred.
- Outlook is acceptable as an application, but its interaction and integration - Outlook is acceptable as an application, but its interaction and integration
do not fit as naturally. Accessible mail for tools and agents matters separately do not fit as naturally. Accessible mail for tools and agents matters separately
+20 -26
View File
@@ -1,30 +1,16 @@
{ {
flake.modules.nixos.handy = { pkgs, ... }: {
services.udev.packages = [
(pkgs.writeTextDir "lib/udev/rules.d/70-handy-keyboard.rules" ''
SUBSYSTEM=="input", KERNEL=="event*", ENV{ID_INPUT_KEYBOARD}=="1", TAG+="uaccess"
'')
];
};
flake.modules.homeManager.handy = flake.modules.homeManager.handy =
{ config, pkgs, ... }: { pkgs, ... }:
let let
settings = (pkgs.formats.json { }).generate "handy-settings.json" { python = pkgs.python3.withPackages (ps: [ ps.evdev ]);
settings = {
settings_schema_version = 1;
selected_model = "handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf/nemotron-3.5-asr-streaming-0.6b-Q8_0.gguf";
selected_language = "auto";
translate_to_english = false;
selected_microphone = null;
always_on_microphone = false;
push_to_talk = true;
paste_method = "ctrl_shift_v";
typing_tool = "wtype";
clipboard_handling = "dont_modify";
auto_submit = false;
overlay_style = "live";
overlay_position = "bottom";
post_process_enabled = false;
start_hidden = true;
autostart_enabled = false;
update_checks_enabled = false;
show_whats_new_on_update = false;
onboarding_completed = true;
};
};
in in
{ {
home.packages = [ pkgs.handy ]; home.packages = [ pkgs.handy ];
@@ -37,10 +23,18 @@
}; };
Install.WantedBy = [ "graphical-session.target" ]; Install.WantedBy = [ "graphical-session.target" ];
Service = { Service = {
# Handy writes migrations and runtime preferences: restore the Nix baseline on start.
ExecStartPre = "${pkgs.coreutils}/bin/install -Dm600 ${settings} ${config.xdg.dataHome}/com.pais.handy/settings_store.json";
ExecStart = "${pkgs.handy}/bin/handy --start-hidden"; ExecStart = "${pkgs.handy}/bin/handy --start-hidden";
}; };
}; };
systemd.user.services.handy-ptt = {
Unit = {
Description = "Handy push-to-talk key release listener";
PartOf = [ "graphical-session.target" ];
Requisite = [ "handy.service" ];
After = [ "handy.service" ];
};
Service.ExecStart = "${python}/bin/python ${./handy/_push_to_talk.py} ${pkgs.handy}/bin/handy";
};
}; };
} }
@@ -0,0 +1,48 @@
"""Wait for release of the physical Super+D chord that launched this service."""
import signal
import subprocess
import sys
from contextlib import closing
from evdev import InputDevice, ecodes as key, list_devices
def held(keys):
return key.KEY_D in keys and bool({key.KEY_LEFTMETA, key.KEY_RIGHTMETA} & set(keys))
def interrupted(signum, frame):
raise KeyboardInterrupt
def main(handy):
signal.signal(signal.SIGTERM, interrupted)
for path in list_devices():
try:
device = InputDevice(path)
except OSError:
continue # Devices can disappear between enumeration and opening.
with closing(device):
if not held(device.active_keys()):
continue
action = "--cancel"
try:
subprocess.run([handy, "--toggle-transcription"], check=True)
# A quick release may already be queued while the CLI starts.
if held(device.active_keys()):
for event in device.read_loop():
if not held(device.active_keys()):
break
action = "--toggle-transcription"
finally:
# Disconnects, cancellation and service shutdown discard the recording.
subprocess.run([handy, action], check=True)
return
if __name__ == "__main__":
try:
main(sys.argv[1])
except KeyboardInterrupt:
pass
@@ -0,0 +1,37 @@
"""Synthetic key-state checks; never reads a keyboard or records audio."""
from unittest.mock import Mock, patch
import _push_to_talk as ptt
down = [ptt.key.KEY_LEFTMETA, ptt.key.KEY_D]
def check(states, expected, error=None):
device = Mock()
device.active_keys.side_effect = states
device.read_loop.return_value = iter([object(), object()])
if error:
device.read_loop.side_effect = error
with (
patch.object(ptt, "list_devices", return_value=["keyboard"]),
patch.object(ptt, "InputDevice", return_value=device),
patch.object(ptt.signal, "signal"),
patch.object(ptt.subprocess, "run") as run,
):
try:
ptt.main("handy")
except (OSError, KeyboardInterrupt):
assert error is not None
assert [call.args[0][1] for call in run.call_args_list] == expected
device.close.assert_called_once()
check([[]], []) # Released before the listener starts: do not record.
check([down, []], ["--toggle-transcription", "--toggle-transcription"])
check([down, down, down, [ptt.key.KEY_D]], ["--toggle-transcription"] * 2)
check([down, down], ["--toggle-transcription", "--cancel"], OSError("unplugged"))
check([down, down], ["--toggle-transcription", "--cancel"], KeyboardInterrupt())
assert ptt.held([ptt.key.KEY_RIGHTMETA, ptt.key.KEY_D])
assert not ptt.held([ptt.key.KEY_LEFTMETA])
print("Push-to-talk release, repeat, disconnect and cancellation checks passed")
+11 -7
View File
@@ -36,19 +36,23 @@ in
}; };
# Dictation # Dictation
"Mod+Shift+D" = { "Mod+D" = {
spawn = [ spawn = [
handy "systemctl"
"--toggle-transcription" "--user"
"start"
"handy-ptt.service"
]; ];
_props.repeat = false; _props.repeat = false;
_props.hotkey-overlay-title = "Hold to Dictate";
};
"Mod+Shift+D" = {
spawn-sh = "systemctl --user stop handy-ptt.service && ${handy} --toggle-transcription";
_props.repeat = false;
_props.hotkey-overlay-title = "Toggle Dictation"; _props.hotkey-overlay-title = "Toggle Dictation";
}; };
"Mod+Ctrl+D" = { "Mod+Ctrl+D" = {
spawn = [ spawn-sh = "systemctl --user stop handy-ptt.service && ${handy} --cancel";
handy
"--cancel"
];
_props.repeat = false; _props.repeat = false;
_props.hotkey-overlay-title = "Cancel Dictation"; _props.hotkey-overlay-title = "Cancel Dictation";
}; };
+1
View File
@@ -17,6 +17,7 @@ in
nixosModules.network-manager nixosModules.network-manager
nixosModules.nftables-firewall nixosModules.nftables-firewall
nixosModules.niri nixosModules.niri
nixosModules.handy
nixosModules.notes nixosModules.notes
nixosModules.plymouth nixosModules.plymouth
nixosModules.printing nixosModules.printing