review: validate SSH host alias + rename LAST30DAYS_YT_SSH_HOST -> LAST30DAYS_YOUTUBE_SSH_HOST
Addresses two concerns surfaced during PR #376 review: 1. **SSH option-injection on the host value.** The original PR uses shlex.quote() on the remote command and added a `--` option terminator in front of the host, but neither one stops a hostile env var like `LAST30DAYS_YT_SSH_HOST=-oProxyCommand=...` from being read in the first place. Tighten `_ytdlp_ssh_host()` to validate the host against `^[a-zA-Z0-9._-]+$` (plain hostname/SSH-config-alias shape: letters, digits, dot, underscore, hyphen). Any value that doesn't match logs a warning to stderr and returns None, so the wrap function falls back to local execution. The `--` terminator stays as defense-in-depth for the case where a valid host happens to start with `-`, but the regex closes the door on the env var reaching ssh at all. 2. **Env var naming consistency.** Existing skill-internal config knobs spell out their domain: `LAST30DAYS_X_BACKEND`, `LAST30DAYS_X_MODEL`, `LAST30DAYS_PLANNER_MODEL`, `LAST30DAYS_RERANK_MODEL`, etc. The module is `youtube_yt.py`, the source key is `youtube`, the function family is `is_youtube_*()` — `YT` was the odd abbreviation out. Rename to `LAST30DAYS_YOUTUBE_SSH_HOST` so the variable matches the user mental model ("route YouTube fetches via residential IP") and the codebase's spelled-out convention. Adds three new tests: - test_host_alias_with_dash_prefix_is_rejected (validator rejects `-o...`) - test_host_alias_with_shell_metacharacters_is_rejected (rejects spaces, ;, $, `, &) - test_host_alias_validator_accepts_realistic_aliases (allows FQDNs, IPs, bare aliases) The existing test_wrap_cmd_uses_option_terminator is rewritten to use a valid host value (since an invalid one is now filtered upstream) and continues to assert the `--` terminator placement as defense-in-depth. 44/44 youtube_yt tests pass (40 prior + 4 net new validator tests).
This commit is contained in:
@@ -545,8 +545,8 @@ def main() -> int:
|
||||
# youtube_yt) can read it without taking a config dependency. This
|
||||
# routes yt-dlp through `ssh <host>` to bypass YouTube's bot-wall on
|
||||
# datacenter IPs (see lib/youtube_yt.py for details).
|
||||
if config.get("LAST30DAYS_YT_SSH_HOST") and "LAST30DAYS_YT_SSH_HOST" not in os.environ:
|
||||
os.environ["LAST30DAYS_YT_SSH_HOST"] = config["LAST30DAYS_YT_SSH_HOST"]
|
||||
if config.get("LAST30DAYS_YOUTUBE_SSH_HOST") and "LAST30DAYS_YOUTUBE_SSH_HOST" not in os.environ:
|
||||
os.environ["LAST30DAYS_YOUTUBE_SSH_HOST"] = config["LAST30DAYS_YOUTUBE_SSH_HOST"]
|
||||
|
||||
# Handle setup subcommand
|
||||
topic = " ".join(args.topic).strip()
|
||||
|
||||
@@ -100,7 +100,7 @@ def _log(msg: str):
|
||||
def is_ytdlp_installed() -> bool:
|
||||
"""Check if yt-dlp is available locally, or if SSH routing is configured.
|
||||
|
||||
When LAST30DAYS_YT_SSH_HOST is set, returns True without a local check —
|
||||
When LAST30DAYS_YOUTUBE_SSH_HOST is set, returns True without a local check —
|
||||
yt-dlp lives on the remote host. Failures surface naturally on first use.
|
||||
"""
|
||||
if _ytdlp_ssh_host():
|
||||
@@ -108,10 +108,16 @@ def is_ytdlp_installed() -> bool:
|
||||
return shutil.which("yt-dlp") is not None
|
||||
|
||||
|
||||
# Host aliases must be plain hostnames / SSH config aliases — no flags, no
|
||||
# shell metacharacters. Rejects any value that could be reinterpreted by ssh
|
||||
# (or the surrounding shell) as something other than a destination.
|
||||
_SSH_HOST_ALIAS_RE = re.compile(r"^[a-zA-Z0-9._-]+$")
|
||||
|
||||
|
||||
def _ytdlp_ssh_host() -> Optional[str]:
|
||||
"""Return SSH host alias if yt-dlp should be routed via SSH, else None.
|
||||
|
||||
Set LAST30DAYS_YT_SSH_HOST=<ssh-alias> (e.g. 'macmini') in the environment
|
||||
Set LAST30DAYS_YOUTUBE_SSH_HOST=<ssh-alias> (e.g. 'macmini') in the environment
|
||||
to route yt-dlp through SSH for residential IP egress. This bypasses
|
||||
YouTube's bot-wall on datacenter IPs (Hetzner, DigitalOcean, AWS, etc.)
|
||||
where ytsearch returns 0 results regardless of cookies.
|
||||
@@ -121,13 +127,30 @@ def _ytdlp_ssh_host() -> Optional[str]:
|
||||
add brew shellenv to ~/.zshenv (not just ~/.zprofile) so non-login SSH
|
||||
shells find yt-dlp on PATH.
|
||||
|
||||
Validation: host value must match ``[A-Za-z0-9._-]+``. Anything starting
|
||||
with ``-`` or containing shell/SSH metacharacters is rejected with a
|
||||
stderr warning and treated as unset, so a misconfigured or attacker-
|
||||
controlled value can't slip through as an SSH option flag or proxy command.
|
||||
The ``--`` option terminator in ``_wrap_ytdlp_cmd`` is a second line of
|
||||
defense; this regex closes the door on the env var ever reaching ssh
|
||||
in the first place.
|
||||
|
||||
To use a value from ~/.config/last30days/.env, export it into the
|
||||
environment before invoking the engine, e.g. in a wrapper:
|
||||
set -a; source ~/.config/last30days/.env; set +a
|
||||
python3 last30days.py "..."
|
||||
"""
|
||||
host = os.environ.get("LAST30DAYS_YT_SSH_HOST", "").strip()
|
||||
return host or None
|
||||
host = os.environ.get("LAST30DAYS_YOUTUBE_SSH_HOST", "").strip()
|
||||
if not host:
|
||||
return None
|
||||
if not _SSH_HOST_ALIAS_RE.match(host):
|
||||
sys.stderr.write(
|
||||
f"[youtube_yt] WARNING: LAST30DAYS_YOUTUBE_SSH_HOST={host!r} "
|
||||
"does not look like a plain hostname/alias; ignoring. "
|
||||
"Expected pattern: letters, digits, dot, underscore, hyphen.\n"
|
||||
)
|
||||
return None
|
||||
return host
|
||||
|
||||
|
||||
def _wrap_ytdlp_cmd(cmd: List[str]) -> List[str]:
|
||||
@@ -136,7 +159,7 @@ def _wrap_ytdlp_cmd(cmd: List[str]) -> List[str]:
|
||||
Args are shell-quoted to survive the remote shell. Uses BatchMode=yes so
|
||||
a misconfigured key fails fast instead of hanging on a password prompt.
|
||||
The `--` option terminator prevents an SSH option-injection if
|
||||
LAST30DAYS_YT_SSH_HOST were ever set to a value starting with `-`.
|
||||
LAST30DAYS_YOUTUBE_SSH_HOST were ever set to a value starting with `-`.
|
||||
"""
|
||||
host = _ytdlp_ssh_host()
|
||||
if not host:
|
||||
|
||||
Reference in New Issue
Block a user