Opened 9 months ago

Closed 8 weeks ago

#13234 closed Developer Task (Fixed)

Support for Nvidia Shield Remote

Reported by: Peter Bennett Owned by: Peter Bennett
Priority: minor Milestone: 30.0
Component: Ports - Android Version: Master Head
Severity: medium Keywords:
Cc: mtdean@… Ticket locked: no

Description

The NVidia Shield has a very limited remote with only the following buttons:

  • Left
  • Right
  • Up
  • Down
  • Select
  • Back
  • Home (not accessible to applications)

Since the Shield is touted as a multimedia device, it should be possible to navigate the application with only the supplied remote.

Attachments (12)

20180313_1414_bookmark.patch (1.6 KB) - added by Peter Bennett 8 months ago.
New bookmark key mapping
20180313_1533_longpress.patch (7.7 KB) - added by Peter Bennett 8 months ago.
Latest Long Press patch
20180314_1955_menus_and_bookmark.patch (5.2 KB) - added by Peter Bennett 8 months ago.
Menu and bookmark changes
20180315_1936_menus_and_bookmark.patch (8.9 KB) - added by Peter Bennett 8 months ago.
Updated patch, see comment:19
20180904_1233_hidden_focus_bug.patch (1.6 KB) - added by Peter Bennett 7 months ago.
UI Hidden Focus Bug.
20180409_1234_reset_button_bug.patch (717 bytes) - added by Peter Bennett 7 months ago.
Fix Reset button bug
20180409_1244_rec_grp_arrows.patch (8.5 KB) - added by Peter Bennett 7 months ago.
Playback Support change recording group with arrow keys
20180409_1248_new_osd_nav.patch (119.8 KB) - added by Peter Bennett 7 months ago.
New OSD Navigation screen
20180424_1410_osdnav_livetv.patch (2.0 KB) - added by Peter Bennett 7 months ago.
Fix some buttons not working with LiveTV
20180425_1504_bkmark_and_pause.patch (3.0 KB) - added by Peter Bennett 7 months ago.
Bookmark and pause enhancement
20180725_1136_long_press_back.patch (475 bytes) - added by Peter Bennett 4 months ago.
Fix for long press of back button
20180725_1646_long_press_back_and_jumppoints.patch (1.9 KB) - added by Peter Bennett 4 months ago.
Fix long press back and jumppoints

Download all attachments as: .zip

Change History (55)

comment:1 Changed 9 months ago by Peter Bennett

Many of the themes have used the triggerevent tag to map left, right, up and down to various actions. Themes are inconsistent about this. With the Shield limited remote, we will have to be careful how the buttons are used so the the user is not locked out of getting things done.

We can create a shield-specific (android tv specific?) theme or themes or an overlay that gets loaded on top of other themes in the case of the shield or android tv.

I suggest using Select to bring up a menu and right to perform the default action when there is nothing to move to on the right. For example in the recordings screen right will bring you from the groups to the recordings and right on a recording will play. When on a recording, select will bring up the menu. This is the opposite of the recent change in Steppes, but it seems more logical to me. The Select button will universally bring up the menu rather than sometimes the right arrow bringing up the menu and sometimes select bringing it up.

I will work out further details and add them here, as well as a method to make sure users with a limited remote can do everything while traditional users do not find their remotes work differently from before.

comment:2 Changed 9 months ago by Craig Treleaven <ctreleaven@…>

Currently, I think the most limited remote supported by Myth is the Apple Remote. I believe it can do everything except text entry. These have only 6 buttons:

4 arrow keys Play/Pause? Menu

(There are overloaded functions available with a long press of each of the buttons.)

I wonder if the Shield "Select" and "Back" buttons should be mapped the same as the Apple Remote's Play/Pause? and Menu. This means the Shield remote would work with all themes out of the box. Not the most intuitive, perhaps, but no special theming support would be required.

comment:3 Changed 9 months ago by Peter Bennett

Please elaborate on how the apple remote is used. There is a wiki page on it which says that it is not much use since it has so few buttons.

https://www.mythtv.org/wiki/Apple_Remote

I wonder how you exit playback without a stop button or a back button, for example.

comment:4 Changed 9 months ago by Craig Treleaven <ctreleaven@…>

Yeah, lhe Apple Remote wiki page has the "Outdate" tag...and really deserves it! There is brief mention on the main Mac page:

https://www.mythtv.org/wiki/MythTV_on_Mac_OS_X#Apple_Remote

I don't know the implementation details, but I think the Apple remote Play/Pause? and Menu buttons are normally passed as the Enter and Esc keystrokes, respectively. The Play/Pause? button will start a recording playing. The Menu button exits playback.

A key difference however, is that a long press on the Apple Remote buttons sends different events. A long press on the Menu button is passed as the M keystroke. Long press on the Play/pause button passes a P keystroke. (It is Mac OS X code that interprets long v. short presses and passes different events; not MythTV code.)

In practise, the only long press I use is the Menu key. Perhaps lacking that differentiation is a killer for the Shield remote, however. Effectively, each of the Apple Remote's 6 buttons send 2 different key codes.

comment:5 Changed 9 months ago by Peter Bennett

I tested this on the Shield remote and a long press just generates the same code many times over.

If I fix it so that the Shield can work with its remote that should also enable other limited remotes, so I would prefer not to make it Shield-specific or Android-specific, and preferably not specific to certain themes.

comment:6 Changed 9 months ago by Peter Bennett

Status: newaccepted

I believe I can implement the "long press". I plan to make "long press" of the SELECT button invoke the MENU action.

Details:

  • Add a setting for keys (e.g. the SELECT key) that will support "long press". Those keys will NOT support repeat action. To set "long press" on the arrow keys, for example, would be very bad.
  • Normally the SELECT button takes effect immediately upon pressing. With this change it will only take effect upon release.
  • Use the KeyPress and KeyRelease events to time how long one of those keys is held down and generate a single action upon release. Discard all KeyPress events for that key that are generated in the interim.
  • If the button was held down for more than 1 second, generate a MetaModifier on the keystroke to support a long press. The MetaModifier is normally generated when holding the "Windows" key and pressing a button, I am hijacking it for another purpose.
  • The normal key mapping feature of !MythTV can assign an action to Meta-Enter or other Meta keys. In particular, I will create a default entry for Meta-Enter to generate the MENU action.
  • This is not android specific so it could be used with any remote that uses a KeyPress and KeyRelease event on any operating system.
  • I do not know if LIRC would support this, but any remote that connects as a keyboard would support it.

I also plan to add some entries to the menus so that all important actions are available via menu. For example:

  • Add "Info" to menus - this will bring up the info menu
  • Add "Pause", "fast forward", and "bookmark" to playback menus.
  • Possibly add "next menu" to menus so that you can cycle through menus.

comment:7 Changed 9 months ago by Mark Spieth

Be nice to use the voice functionality of the shield remote too. That would solve lots of usability issues.

comment:8 Changed 9 months ago by gigem

I believe Kodi 18 supports some integration with Google Assistant on the Shield. Probably mainly search.

comment:9 Changed 9 months ago by gigem

Peter, for Pause, FastForward? and other basic functions during playback, I think an OSD menu like in the Youtube TV interface, VLC, Kodi and others would be more intuitive than the traditonal, MythTV menu that comes up when pressing MENU. The traditional menu could be accessed by choosing the "hamburger" icon in the new menu.

comment:10 Changed 8 months ago by Peter Bennett

This is step 1 of the solution.

The attached patch adds support for long presses.

  • Long press will work for Shield remote and for keyboards, for selected keys only.
  • I added a new section to "edit keys" called "Long Press". You can assign up to 4 keys that will support log presses. Initially only "Enter" is in there.
  • I added a default setting for Menu in the global settings, assigned to a long press of Enter. This only takes effect for new frontends. To use it with an existing frontend, use "edit keys" to add a long press of enter to Global Menu.

Next steps

  • Solution for bookmarks. The "Enter" key adding a bookmark is IMHO not the best use of the one and only button.
  • Solution for an OSD as suggested by David.
  • Add missing entries to menus and/or OSD.

comment:11 Changed 8 months ago by gigem

Peter, the long press configuration seems confusing to me. Is there some reason it can't work like the following? When asked to press a key, press a key and release it quickly for a normal (short) press and press and hold for a second before releasing for a long press.

comment:12 Changed 8 months ago by Peter Bennett

Here is an update of the patch to allow for up to 16 keys that support long press.

If you previously installed patch 20180312_1140_longpress.patch, please run this SQL after installing the updated patch (20180312_1913_longpress.patch):

DELETE from keybindings
where action = 'LONGPRESS';

There will now be 4 actions LONGPRESS1, LONGPRESS2, LONGPRESS3, LONGPRESS4 instead of LONGPRESS.

comment:13 Changed 8 months ago by Peter Bennett

This next patch creates a new key mapping for setting and removing bookmarks. It adds Ctrl+K, space and return for setting a bookmark. I would like to remove space and return because I believe that space should pause and return should do something more relevant to most people, like show a menu or a navigation aid. "Enter" is left off since I want to use it for playback navigation on the Shield. Please pass on any comments on what to do about space, Return and Enter.

Changed 8 months ago by Peter Bennett

New bookmark key mapping

comment:14 Changed 8 months ago by Peter Bennett

Updated longpress patch checks response from TranslateKeyPress and fixes a problem in the previous version.

Changed 8 months ago by Peter Bennett

Latest Long Press patch

comment:15 Changed 8 months ago by Peter Bennett

Patch to allow bookmarks and a couple of other things to be used with Shield remote. This supersedes the bookmark patch added earlier

  • BOOKMARK key mapping defaulting to the existing select keys.
  • Program details screen that normally only switches between the two layouts with Info - switch with SELECT action (e.g. ENTER) (as if I was pressed)
  • "Set bookmark" and "jump to bookmark" added to Playback navigation menu
  • Video list page: switch from the group menu to the file menu (as if Info was pressed) when Menu is pressed again, like Recordings do.

Changed 8 months ago by Peter Bennett

Menu and bookmark changes

comment:16 Changed 8 months ago by Mark Spieth

Both LGTM
SHIPEM

comment:17 Changed 8 months ago by Mark Spieth

NOTE: I set bookmarks all the time if I'm walking away from a show in progress, just in case.

comment:18 Changed 8 months ago by Peter Bennett

I will be changing this a bit as recommended by Micheal Dean (see https://lists.gt.net/mythtv/dev/616275#616275) before committing

comment:19 Changed 8 months ago by Peter Bennett

Cc: mtdean@… added

A new patch for menus and bookmarks. This uses the approach recommended by Michael Dean, see ​https://lists.gt.net/mythtv/dev/616275#616275.

  • Added SETBOOKMARK and TOGGLEBOOKMARK actions in TV Playback context. On a new frontend they are not assigned to any key.
  • If an existing frontend gets upgraded to the new version, the keys assigned to SELECT in the Global context will be assigned to either SETBOOKMARK or TOGGLEBOOKMARK depending on whether the "alternate set and remove bookmark" setting was checked.
  • The "alternate set and remove bookmark" setting is removed from frontend setup and is no longer used.
  • On a new frontend, the space bar will be added as a key on the Pause action. This does not happen on an existing frontend being upgraded.
  • Set bookmark, Toggle bookmark and Jump to bookmark are added to the Playback navigation menu.
  • The Program details screen that normally only switches between the two layouts with INFO will also switch with SELECT action (e.g. ENTER)
  • Video list page: switch from the group menu to the file menu when MENU is pressed a second time. This is similar to what the Recordings list does.

Changed 8 months ago by Peter Bennett

Updated patch, see comment:19

comment:20 Changed 8 months ago by Peter Bennett <pbennett@…>

In 5c4dffb3e853ef819ef3f62ca0cf7283e92a6b9d/mythtv:

Frontend GUI: Add support for long presses of selected keys

Long press will work for Shield remote and for keyboards,
for selected keys only.

I added a new section to "edit keys" called "Long Press".
You can assign up to 16 keys that will support log presses.
Initially only "Enter" is in there.

I added a default setting for Menu in the global settings,
assigned to a long press of Enter. This only takes effect
for new frontends. To use it with an existing frontend,
use "edit keys" to add a long press of enter to Global Menu.

Adding a key to "Long Press" will disable autorepeat for the key.

Refs #13234

comment:21 Changed 8 months ago by Peter Bennett <pbennett@…>

In 5f9ddd6df32844633bf86f1221071cbdb0e9d812/mythtv:

Frontend GUI: Menu and Bookmark enhancements

Added SETBOOKMARK and TOGGLEBOOKMARK actions in TV Playback context.
On a new frontend they are not assigned to any key.

If an existing frontend gets upgraded to the new version, the keys
assigned to SELECT in the Global context will be assigned to either
SETBOOKMARK or TOGGLEBOOKMARK depending on whether the "alternate set
and remove bookmark" setting was checked.

The "alternate set and remove bookmark" setting is removed from frontend
setup and is no longer used.

On a new frontend, the space bar will be added as a key on the Pause
action. This does not happen on an existing frontend being upgraded.

Set bookmark, Toggle bookmark and Jump to bookmark are added to the
Playback navigation menu.

The Program details screen that normally only switches between the two
layouts with INFO will also switch with SELECT action (e.g. ENTER)

Video list page: switch from the group menu to the file menu when MENU
is pressed a second time. This is similar to what the Recordings list does.

Refs #13234

comment:22 Changed 8 months ago by Peter Bennett <pbennett@…>

In a96c04ac5be8b15a94af8894b9306a8f5b64770f/mythtv:

Frontend GUI: Remove default of Enter as Long Press

Keys assigned as Long Press may not work with lirc, so Enter may stop
working for some users. Remove the default that caused this.
Users will have to manually add keys they want to use with Long Press.

Refs #13234

Changed 7 months ago by Peter Bennett

UI Hidden Focus Bug.

comment:23 Changed 7 months ago by Peter Bennett

UI Hidden Focus Bug.

20180904_1233_hidden_focus_bug.patch

Fix bug where if a group containing buttons was hidden, focus tabbing did not work correctly owing to hidden buttons being selected for focus.

Changed 7 months ago by Peter Bennett

Fix Reset button bug

comment:24 Changed 7 months ago by Peter Bennett

UI Fix Reset button bug.

20180409_1234_reset_button_bug.patch

Reset action on a screen should restore everything to the startup state. Fix the bug that if there are buttons on the screen they would vanish since they were set to invalid state "default" instead of valid state "active".

Changed 7 months ago by Peter Bennett

Playback Support change recording group with arrow keys

comment:25 Changed 7 months ago by Peter Bennett

Playback Support change recording group with arrow keys

20180409_1244_rec_grp_arrows.patch

New feature to help with limited remotes such as on the NVidia Shield. On the playback screen, allow the arrow keys to navigate to the recording groups buttons and select a new recording group. This is disabled unless the user selects new option "Change Recording Group using the arrow keys" in setup. This requires some things to be done in themes: (recordings-ui.xml).

  1. add focusorder 2 to recgroups buttonlist
  2. add state selectedactive and active to recgroups buttonlist to highlight when the recgroups have focus.
  3. Remove triggerevent left from groups buttonlist to allow "left" to get to recgroups or set wrapstyle "none" to allow "up" to get to recgroups.

These have been done in the mythcenter and mythcenter wide themes.

Changed 7 months ago by Peter Bennett

New OSD Navigation screen

comment:26 Changed 7 months ago by Peter Bennett

New OSD Navigation screen

20180409_1248_new_osd_nav.patch Note this patch needs the two bug fix patches above to work correctly.

When invoked, OSD Navigation provides a row of icons along the bottom of the playback screen with items like play, pause, jump forward, jump back. There is a "more" button that shows other selections of icons. The navigation icons are displayed in conjunction with the osd_status (progress bar).

There is a new keyboard action available called OSDNAVIGATION. This needs to be assigned to a key in setup under TV Playback. I recommend assigning "Enter", so that the enter button can be used on the NVidia Shield.

There is a default OSD Navigation window included in the default and defaultwide themes, that will show on all themes unless overridden.

Theme designers can create as many buttons as they wish and assign them to as many groups as they wish. They must be set up so as not to overlap with the osd_status display. Actions that are available in TV Playback can be added to buttons as desired by the theme designer. There is no fixed list of actions, they can be decided by the theme designer.

I will document it fully in the wiki for theme designers.

Last edited 7 months ago by Peter Bennett (previous) (diff)

comment:27 Changed 7 months ago by Peter Bennett <pbennett@…>

In 8b49f8aee17030a2fa822e601f473117d73f7a15/mythtv:

GUI: Fix hidden focus bug

Fix bug where if a group containing buttons was hidden, focus
tabbing did not work correctly owing to hidden buttons being
selected for focus.

Refs #13234

comment:28 Changed 7 months ago by Peter Bennett <pbennett@…>

In a3ae3a8288beaafbac71acd9d4300792711b025d/mythtv:

GUI: Fix bug with buttons vanishing when Reset used

Reset action on a screen should restore everything to the startup
state. Fix the bug that if there are buttons on the screen they
would vanish since they were set to invalid state "default"
instead of valid state "active".

Refs #13234

comment:29 Changed 7 months ago by Peter Bennett <pbennett@…>

In f20ab149a873ba80c23908533e4461b672223d43/mythtv:

GUI: Enhance Playback to allow change of recording group with arrow keys

New feature to help with limited remotes such as on the
NVidia Shield. On the playback screen, allow the arrow keys
to navigate to the recording groups buttons and select a new
recording group. This is disabled unless the user selects
new option "Change Recording Group using the arrow keys" in
setup.
This requires some things to be done in themes:
(recordings-ui.xml).

  1. add focusorder 2 to recgroups buttonlist
  2. add state selectedactive and active to recgroups buttonlist to highlight when the recgroups have focus.
  3. Remove triggerevent left from groups buttonlist to allow "left" to get to recgroups or set wrapstyle "none" to allow "up" to get to recgroups.

These have been done in the mythcenter and mythcenter wide themes.

Refs #13234

comment:30 Changed 7 months ago by jpoet

The 'master' version of Steppes has been updated. I don't have it working on a Shield right now, so if you notice any issues with Steppes, please let me know.

comment:31 Changed 7 months ago by Peter Bennett <pbennett@…>

In eb947177c1268de5b23a5869993f5945c7ec0b4a/mythtv:

Playback OSD: New navigation screen

When invoked, OSD Navigation provides a row of icons along the bottom of the
playback screen with items like play, pause, jump forward, jump back.
There is a "more" button that shows other selections of icons.
The navigation icons are displayed in conjunction with the osd_status
(progress bar).

There is a new keyboard action available called OSDNAVIGATION.
This needs to be assigned to a key in setup under TV Playback.
I recommend assigning "Enter", so that the enter button can be used
on the NVidia Shield.

There is a default OSD Navigation window included in the default and defaultwide
themes, that will show on all themes unless overridden.

Theme designers can create as many buttons as they
wish and assign them to as many groups as they wish. They must be set
up so as not to overlap with the osd_status display. Actions
that are available in TV Playback can be added to buttons as desired
by the theme designer. There is no fixed list of actions, they can
be decided by the theme designer.

I will document it fully in the wiki for theme designers.

Refs #13234

Changed 7 months ago by Peter Bennett

Fix some buttons not working with LiveTV

comment:32 Changed 7 months ago by Peter Bennett

The play and info buttons are not working in LiveTV. Attached patch 20180424_1410_osdnav_livetv.patch addresses this.

comment:33 Changed 7 months ago by Peter Bennett

Owner: changed from Peter Bennett to Peter Bennett
Status: acceptedassigned

Changed 7 months ago by Peter Bennett

Bookmark and pause enhancement

comment:34 Changed 7 months ago by Peter Bennett

This patch fixes a couple of anomalies

  1. Enables bookmarks on LiveTV (both with assigned key (spacebar) and through the OSD Nav).
  2. The Play/Pause icon will show Play when in FF or REW mode, so that the user can resume play without first having to pause.

This patch is dependent on the prior patch that fixes LiveTV issues.

comment:35 Changed 7 months ago by Peter Bennett <pbennett@…>

In ca1ec3cdb4/mythtv:

Playback OSD Nav Screen: Fix some buttons not working with LiveTV

The play and info buttons were not working in LiveTV.

Refs #13234

comment:36 Changed 7 months ago by Peter Bennett <pbennett@…>

In f1054a3bb/mythtv:

Playback OSD: Navigation: Bookmark and pause enhancement

This change fixes a couple of anomalies

  1. Enables bookmarks on LiveTV (both with assigned key (spacebar) and through the OSD Nav).
  2. The Play/Pause? icon will show Play when in FF or REW mode, so that the user can resume play without first having to pause.

Refs #13234

Changed 4 months ago by Peter Bennett

Fix for long press of back button

comment:37 Changed 4 months ago by Peter Bennett

Patch 20180725_1136_long_press_back.patch :

Allows long press to be used with back.

Once you assign Back to Long Press, the QT default whereby Back causes a "close" operation will be disabled. Before assigning back to long press, you must assign back to Global Escape and Main Menu exit. Otherwise back will stop working and you will be unable to complete the assignment of long press.

Some back actions may work differently after this, since they will be going through keyboard translation instead of the default QT close action.

Changed 4 months ago by Peter Bennett

Fix long press back and jumppoints

comment:38 Changed 4 months ago by Peter Bennett

Patch20180725_1646_long_press_back_and_jumppoints.patch replaces prior patch

Fixes above issue with long press of Back. Also fixes long press with a jump point, which was not working correctly.

The rules about assigning long press to back button in comment 37 still apply.

comment:39 Changed 4 months ago by Peter Bennett <pbennett@…>

In ca1ec3cdb4/mythtv:

Playback OSD Nav Screen: Fix some buttons not working with LiveTV

The play and info buttons were not working in LiveTV.

Refs #13234

comment:40 Changed 4 months ago by Peter Bennett <pbennett@…>

In f1054a3bb/mythtv:

Playback OSD: Navigation: Bookmark and pause enhancement

This change fixes a couple of anomalies

  1. Enables bookmarks on LiveTV (both with assigned key (spacebar) and through the OSD Nav).
  2. The Play/Pause? icon will show Play when in FF or REW mode, so that the user can resume play without first having to pause.

Refs #13234

comment:41 Changed 4 months ago by Peter Bennett <pbennett@…>

In 3b0dacda1/mythtv:

Frontend GUI: long press: Fix for back button and juppoints

Once you assign Back to Long Press, the QT default whereby Back
causes a "close" operation will be disabled. Before assigning back
to long press, you must assign back to Global Escape and Main Menu
exit. Otherwise back will stop working and you will be unable to
complete the assignment of long press.

Some back actions may work differently after this, since they will
be going through keyboard translation instead of the default QT
close action.

Also fixes long press with a jump point, which was not working correctly.

Refs #13234

comment:42 Changed 4 months ago by Peter Bennett <pbennett@…>

In 33cdfd97a/mythtv:

Frontend GUI: long press: Fix long press in guide invoked from playback

Correct the long press handling in playback. calls were out of sequence
when keys were pressed during a GUI invoked from playback such as
the guide.

Refs #13234

comment:43 Changed 8 weeks ago by Peter Bennett

Resolution: Fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.