It's been too long since I wrote my other Nagios posts. The server I had running died, and I didn't recover the nagios files from it. Fortunately, I had an AWS machine to compare to, so I didn't lose everything.
Setting up the check_by_ssh checks is, like all of nagios (it seems), a multi-config process. You'll need to modify
$NAGIOS_HOME/etc/objects/commands.cfg, $NAGIOS_HOME/etc/nagios.cfg, and $NAGIOS_HOME/etc/servers/<server_name>.