Absolute mouse coordinates

[kitrinx]

In most operating systems, there is two types of mouse input: relative, like (-1, 0) which is the change in position, and absolute, like (279, 1000) which is the absolute pixel position on the screen. Is there a way we could provide absolute coordinates to cores in the form of 0:255 x and 0:255 y? I'm honestly not very sure how this works on an OS level, and I always thought that absolute coordinates were just an abstract construct of the operating system, but some devices such as the Aimtrak light gun and Dolphinbar rely on them to work.

[Sorgelig]

Mouse generates only amount of movement (EV_REL event). It goes directly from mouse HW and pass without changes through driver, OS, MiSTer binary to the core. So it's up to core how it will use. It's not a problem to convert it to absolute positions by just using some accumulalor register and clip it by required window.

Lightguns probably generate absolute positions and act as an analog stick of gamepad (EV_ABS event). I don't have such device so cannot confirm.

[Sorgelig]

Dolphinbar from the name looks like Wiimote's optical tracking adaptation for general use?

[Locutus73]

Mouse generates only amount of movement (EV_REL event).

Nope, there are some rare pointing-devices/mice, like touchscreens (which are emulated by a Dolphinbar and modern PC lightguns) which exposes themselves as HID mice (see Dolphinbar lsusb attributes here viewtopic.php?f=117&t=32736&p=368102#p368102 to confirm it’s a composite device with a HID mouse descriptor, so bInterfaceClass=3 and bInterfaceProtocol=2), but they send the absolute position (EV_ABS) events (again see the same post). They are rare, but they exist ad they are seen by operating systems as a mouse with no special driver. Think about it, how could a touchscreen be a mouse and send you the precise position of a single tap on the screen?

P.S.: Dolphinbar has several operational modes and in one it acts as a K+M combo, while in another it acts as a mouse+joypad combo, no driver needed. Obviously in both modes, as soon as you move it, MiSTer gets confused by its EV_ABS events and fires its joypad wizard, but with Rysha/Kitrinx latest implementations it shouldn’t matter since the EV_ABS events are translated into absolute screen positions for lightguns games on NES catching two birds with a stone: supporting both joypads and lightgun. The hard part is passing the joypad wizard pointing the screen with a lightgun and doing precise movements. I’ll report back as soon as I have some time.

Regards.
Locutus73

[kitrinx]

Dolphinbar from the name looks like Wiimote's optical tracking adaptation for general use?

Yes, it lets you use wiimote as a mouse, basically.

[Sorgelig]

Mouse generates only amount of movement (EV_REL event).

Nope, there are some rare pointing-devices/mice, like touchscreens (which are emulated by a Dolphinbar and modern PC lightguns) which exposes themselves as HID mice (see Dolphinbar lsusb attributes here viewtopic.php?f=117&t=32736&p=368102#p368102 to confirm it’s a composite device with a HID mouse descriptor, so bInterfaceClass=3 and bInterfaceProtocol=2), but they send the absolute position (EV_ABS) events (again see the same post). They are rare, but they exist ad they are seen by operating systems as a mouse with no special driver. Think about it, how could a touchscreen be a mouse and send you the precise position of a single tap on the screen?

P.S.: Dolphinbar has several operational modes and in one it acts as a K+M combo, while in another it acts as a mouse+joypad combo, no driver needed. Obviously in both modes, as soon as you move it, MiSTer gets confused by its EV_ABS events and fires its joypad wizard, but with Rysha/Kitrinx latest implementations it shouldn’t matter since the EV_ABS events are translated into absolute screen positions for lightguns games on NES catching two birds with a stone: supporting both joypads and lightgun. The hard part is passing the joypad wizard pointing the screen with a lightgun and doing precise movements. I’ll report back as soon as I have some time.

Regards.
Locutus73

You are not correct about mice device. /dev/input/mice is common for all devices pretending to be a mouse. It always returns relative movements regardless if original device provides absolute or relative positions. This is how DualShock 4 touchpad is handled currently (it also returns EV_ABS in inputX). What you are talking is /dev/inputX devices - they send raw events from HW. So touchpads there may return EV_ABS (my Logitech touchpad returns EV_REL). So inputX events are fully depend on device implementation. Mice (true mice) return EV_REL even from inputX. Pseudo-mouse may return whatever it wants through inputX.
I have Wiimote (as i have the Wii) but never tried it outside the Wii. When i will have time and enough info about it, i will try. Most likely nothing needs to be changed in core. Probably some changes in MiSTer binary required for easier setup. Probably some hardcoded VID/PID will be enough. I think there is no other way to distinguish Dolphinbar (or some Lightguns) from generic gamepad.

P.S.: My previous post was only about mouse(trackball).

[Sorgelig]

MiSTer gets confused by its EV_ABS events and fires its joypad wizard

it's not true already several releases of MiSTer.

[Locutus73]

You are not correct about mice device. /dev/input/mice is common for all devices pretending to be a mouse..
[...]
P.S.: My previous post was only about mouse(trackball).

Whatever we want to call them (pseudo mice, pointing-devices, computer devices, pick the name you prefer) and whatever device Linux uses to map them, the gist of my post was that there are computer usb devices which identifies themselves as Human Interface Device (bInterfaceClass=3) and mouse (bInterfaceProtocol=2) and send absolute events; when modern operating systems see this combination they map these absolute events to a screen absolute position and they move the mouse cursor to that position.

Most likely nothing needs to be changed in core.

If the core was written (as Rysha/Kitrinx NES zapper implementation) with modern lightguns in mind and translates the absolute event to an absolute screen position for the aiming cursor, no change should be needed. If the core was written with a real analog joypad in mind and mapped the absolute event (stick tilt) to the aiming cursor speed vector, so the implementation should be reverted to Rysha/Kitrinx one. Ideally cores could have an option for “Analog as screen position (lightgun)” or “Analog as speed (joypad)”, but IMHO Rysha/Kitrinx approach is the best one if we have to pick one.

Probably some changes in MiSTer binary required for easier setup. Probably some hardcoded VID/PID will be enough. I think there is no other way to distinguish Dolphinbar (or some Lightguns) from generic gamepad.

Yup. Probably there aren’t so much lightguns out there to bother to implement something different than hardcoded VID/PID, but a generic approach could be based on checking and storing the bInterfaceClass and bInterfaceProtocol of the device at the beginning, if they are 2 and 3 and we see EV_ABS then we are dealing with pointing devices (lightguns, touchscreens, whatever) which send absolute events and treat them accordingly (i.e. not launching the wizard, or maybe just skipping the analog values letting mapping just buttons, I don’t know, but continuing to send the absolute value to the core as MiSTer does with joypads).

MiSTer gets confused by its EV_ABS events and fires its joypad wizard

it's not true already several releases of MiSTer.

And that’s interesting: last time I tried (at the end of February, when I posted this viewtopic.php?f=117&t=32736&p=368102#p368102 ) the joypad wizard was fired, but several main MiSTer updates has been released in the meantime and I didn’t test the Dolphinbar again. As soon as I have some time (today I have relatives visiting us for Easter), I’ll report back for sure.
Is there any specific test you want me to do?
Probably if we manage to use the Dolphinbar, we will be able to use all modern lightguns.

Thank you as always.
Regards.

Locutus73

[Sorgelig]

Your logic of detecting lightguns by EV_ABS is absolutely wrong. EV_ABS is the main event for analog joystick and analog sticks of gamepads.
And of course all input devices (including gamepads and joysticks) are HID devices. Many devices also advertise mouse protocol as they may be a complex multifunctional devices. Nothing from mentioned is identified lightguns as exact lightguns. They are bare pointing devices like touchscreen or touchpad. And if call more generic - they are bare analog sticks

So, currently i see the only way is to detect them by VID/PID and do a hardcoded setting to pass their X/Y as an analog joystick.
So, i need VID/PID of such devices and Axes numbers used for X and Y. Current implementation scales any range of analog sticks to -127..+127 range. This will be more than enough for lightguns. So change here is not required.

[Locutus73]

Your logic of detecting lightguns by EV_ABS is absolutely wrong. EV_ABS is the main event for analog joystick and analog sticks of gamepads.
And of course all input devices (including gamepads and joysticks) are HID devices. Many devices also advertise mouse protocol as they may be a complex multifunctional devices. Nothing from mentioned is identified lightguns as exact lightguns. They are bare pointing devices like touchscreen or touchpad. And if call more generic - they are bare analog sticks

Call these pointing devices whatever you like, call them sticks if you prefer, call them John if you like, not everything analog (I.e. a wheel, a gas pedal, a trigger) is a stick, it’s just something reporting a value with a min and max range; you’re fighting a semantic war I find pointless and that I don’t mean to fight. What I’m trying to do is reporting some common features/behavior of these devices, not winning an internet arguing fight.

So, currently i see the only way is to detect them by VID/PID and do a hardcoded setting to pass their X/Y as an analog joystick.
So, i need VID/PID of such devices and Axes numbers used for X and Y. Current implementation scales any range of analog sticks to -127..+127 range. This will be more than enough for lightguns. So change here is not required.

Is there any test/debug we can do to find some common features? There’s something common that let modern operating systems identify them as pointing devices (not putting their axis in any joystick control panel) so they use them for moving mouse pointer without special drivers and/or custom VID/PID lists (I believe it’s the bInterfaceProtocol, but I might be wrong). I have a Dolphinbar and on the Discord channel there is a guy with an Aimteack gun.

Thank you in advance, regards.

Locutus73

[Sorgelig]

I didn't mean to fight, i've just pointing out that your assumption of EV_ABS/HID/Mouse flags doesn't identify lightguns specifically.

Most likely lightguns are the same class as touchscreens (note: not touchpads - they are EV_REL).
The only thing from my point of view can identify them are axes numbers. I remember DS4 generates axes 8/9 (IIRC) for its touchpad which acts as touchscreen as well.
So if lightguns generate the same axes and these axes are different from 0-5 then it can be used as identification.

As usual all input events can be seen in USB console while running Menu core.

[Locutus73]

Most likely lightguns are the same class as touchscreens (note: not touchpads - they are EV_REL).

Exactly! I strongly suspect touchscreens are characterized by HID/mouse/EV_ABS, but, again, I might be superwrong.
Have you (or anyone) a touchscreen (a RPi one should be ok) to test?

The only thing from my point of view can identify them are axes numbers. I remember DS4 generates axes 8/9 (IIRC) for its touchpad which acts as touchscreen as well.
So if lightguns generate the same axes and these axes are different from 0-5 then it can be used as identification.

As usual all input events can be seen in USB console while running Menu core.

As you can see in my post here viewtopic.php?f=117&t=32736&p=368102#p368102 when I use Dolphinbar in mode 2 (Keyboard and Mouse Game Mode), I get these events while wiggling the wiimote

Code: Select all

Input event: type=EV_ABS, Axis=0, Offset:=485, jnum=2, ID:0079:1802. ABS_INFO: min = 0 max = 512
Input event: type=EV_ABS, Axis=1, Offset:=400, jnum=2, ID:0079:1802. ABS_INFO: min = 0 max = 400
Input event: type=EV_ABS, Axis=0, Offset:=484, jnum=2, ID:0079:1802. ABS_INFO: min = 0 max = 512
Input event: type=EV_ABS, Axis=0, Offset:=483, jnum=2, ID:0079:1802. ABS_INFO: min = 0 max = 512


Other events are regular left or right mouse clicks or keyboard strokes.
I should try again the Dolphinbar with MiSTer: if the joypad configuration wizard is still fired and I mange to set the axes (and left mouse button for gunfire), I should be able to use the wiimote for shooting to ducks in NES DuckHunt with latest Rysha/Kitrinx NES implementations.

I’m here for any possible debug and log report. I just cannot assure I will report back quickly, but I will.

Regards.

Locutus73

[Sorgelig]

As i see it uses axes 0/1 for X/Y which is the same as analog stick on gamepad. So, alas axes numbers cannot show it's exactly lightgun.

[Sorgelig]

While i don't have light gun, i have DualShock4 whose touchpad acts as touchscreen.
So, i'm implementing light gun through on it.
And it works well (after some using it some time).

[kitrinx]

While i don't have light gun, i have DualShock4 whose touchpad acts as touchscreen.
So, i'm implementing light gun through on it.
And it works well (after some using it some time).

That's awesome, thank you!

[Locutus73]

While i don't have light gun, i have DualShock4 whose touchpad acts as touchscreen.
So, i'm implementing light gun through on it.
And it works well (after some using it some time).

Great!
Tomorrow I should have some spare time to test again the Dolphinbar and report back.

Thank you.
Ciao.

Locutus73

[Locutus73]

This morning I had the opportunity to update my MiSTer setup and test again the Mayflash Dolphinbar http://www.mayflash.com/Products/NINTENDOWiiU/W010.html with the latest NES core with Lightgun support by Rysha/Kitrinx. I got partial (I’d say 90%) success and this is my experience so far:

This is how the Dolphinbar maps the Wiimote

I use mode 2 (Keyboard and Mouse Game)
These are some logs I get from main MiSTer when I wiggle the Wiimote

Code: Select all

Input event: type=EV_ABS, Axis=1, Offset=262, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
PS2 MOUSE: 28 32 229
Input event: type=EV_ABS, Axis=0, Offset=214, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x28, dx=14, dy=-9, scroll=0
Input event: type=EV_ABS, Axis=1, Offset=267, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
Input event: type=EV_ABS, Axis=1, Offset=273, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
Combined mouse event: btn=0x28, dx=0, dy=-12, scroll=0
PS2 MOUSE: 28 14 235
Input event: type=EV_ABS, Axis=1, Offset=285, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
Combined mouse event: btn=0x28, dx=0, dy=-23, scroll=0
Input event: type=EV_ABS, Axis=0, Offset=211, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x38, dx=-6, dy=-21, scroll=0
Input event: type=EV_ABS, Axis=1, Offset=296, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
PS2 MOUSE: 38 250 212
Input event: type=EV_ABS, Axis=1, Offset=309, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
Combined mouse event: btn=0x28, dx=0, dy=-25, scroll=0
Combined mouse event: btn=0x38, dx=-6, dy=-21, scroll=0
Input event: type=EV_ABS, Axis=0, Offset=208, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Input event: type=EV_ABS, Axis=1, Offset=320, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
PS2 MOUSE: 38 250 210
Input event: type=EV_ABS, Axis=0, Offset=206, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x38, dx=-4, dy=-15, scroll=0
Input event: type=EV_ABS, Axis=1, Offset=328, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
Input event: type=EV_ABS, Axis=0, Offset=212, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x28, dx=12, dy=-12, scroll=0
Input event: type=EV_ABS, Axis=1, Offset=334, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
PS2 MOUSE: 28 8 229
Input event: type=EV_ABS, Axis=0, Offset=221, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x28, dx=18, dy=-13, scroll=0
Input event: type=EV_ABS, Axis=1, Offset=341, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
Input event: type=EV_ABS, Axis=0, Offset=227, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x28, dx=12, dy=-12, scroll=0
Input event: type=EV_ABS, Axis=1, Offset=347, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
PS2 MOUSE: 28 30 231
Input event: type=EV_ABS, Axis=0, Offset=398, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x28, dx=127, dy=-102, scroll=0
Input event: type=EV_ABS, Axis=1, Offset=400, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 400
Combined mouse event: btn=0x08, dx=127, dy=0, scroll=0
Combined mouse event: btn=0x08, dx=88, dy=0, scroll=0
Input event: type=EV_ABS, Axis=0, Offset=387, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x18, dx=-22, dy=0, scroll=0
PS2 MOUSE: 68 255 154
Input event: type=EV_ABS, Axis=0, Offset=233, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x18, dx=-127, dy=0, scroll=0
Combined mouse event: btn=0x18, dx=-127, dy=0, scroll=0
Combined mouse event: btn=0x18, dx=-54, dy=0, scroll=0
PS2 MOUSE: 58 1 0
Input event: type=EV_ABS, Axis=0, Offset=274, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x08, dx=82, dy=0, scroll=0
Input event: type=EV_ABS, Axis=0, Offset=147, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
Combined mouse event: btn=0x18, dx=-127, dy=0, scroll=0
Combined mouse event: btn=0x18, dx=-127, dy=0, scroll=0
PS2 MOUSE: 18 84 0
Combined mouse event: btn=0x08, dx=2, dy=0, scroll=0
Input event: type=EV_ABS, Axis=0, Offset=148, jnum=1, ID:0079:1802. ABS_INFO: min = 0 max = 512
PS2 MOUSE: 8 2 0


First of all MiSTer doesn’t automatically fires the joypad config wizard as soon as I wiggle the Wiimote ad it did before.

So I was able to try DuckHunt without performing any Joypad config (so no map file) and selecting Zapper(Mouse) as peripheral I got some movements when using the Wiimote. I didn’t expect that, because as we discussed before, the Dolphinbar acts as a sort of touchscreen sending EV_ABS events, which should be interpreted by MiSTer as a Joypad analog stick, but evidently main MiSTer changed in the meantime, I don’t know. The movements are erratic though, and the Wiimote is unusable as a lightgun in this way.

So I fiddled with the Joypad configuration wizard in the main menu: when I move the Wiimote as first action, MiSTer interprets the EV_ABS event as coming from Joypad 0079:1802 and let configure consequently only joypads inputs. When I click a button (i.e. 1,2,+,-, but not A and B which are mapped as mouse left click and right click) then MiSTer try to configure Keyboard 0079:1802 and ignores joypad events. If I click A or B, then a mouse click is sent and the Joypad configuration wizard ignores it. Now after several tries I managed to map only Wiimote movements as a Joypad, so aiming to the center of the screen corresponds to having the analog stick 1 centered, aiming to right corresponds to moving the analog stick to the right, aiming to the top corresponds to moving the stick up and so on.
This is the resulting file

input_0079_1802_v2.zip

With this configuration I selected Zapper(Joy1) as Peripheral in NES core while running Duck Hunt and BINGO!!! Aiming to the screen with the Wiimote moves the red dot accordingly as if we are using a real Lightgun!!! SUCCESS!!!

But now the problem: while the Wiimote aiming to the screen is seen as a Joypad analog stick, A and B button are seen as mouse buttons and other buttons are seen as keystroke. When Zapper(Joy1) is selected as Peripheral, the core expects a button mapped as Fire from the same joypad for triggering the virtual lightgun fire button, but when I click the trigger on the Wiimote MiSTer gets a mouse click. If I select Zapper(Mouse) I can successfully fire clicking the wiimote trigger, but I can’t aim.

A (simple? I don’t know) solution would be triggering virtual lightgun fire on left mouse click even when Zapper(Joy1) is selected. In this mode both Joy1 Fire button and mouse left click should trigger the virtual lightgun fire. This way we could successfully use the Dolphinbar/Wiimote combo.

Another marginal feature I’d like to get is using Wiimote buttons for navigating MiSTer menus. I partially achieved this, since the d-pad is mapped as arrow keyboard keys, + as RETURN and – as ESC. Problem is that home button is mapped as Windows key and I didn’t manage to map it for firing the MiSTer menu. I tried the joypad wizard, but it’s impossible to reach the menu definition. I tried to remap the keyboard, but again, I didn’t manage to get a result.

Thank you in advance.
Regards.

Locutus73

[Locutus73]

After watching again this picture with the mappings

I noticed that Mode 3 Game Controller, when a nunchuk is connected, maps all buttons, axes and the aiming sensor to a Joypad… so no mouse or keyboard.
So I switched to Mode 3, then I performed the Joypad wizard both in main menu and in NES core and BAZINGA!!!
These are the mappings for Mode 2 and 3 (they don't overlap since PID changes).

MiSTer-DolphinBar.zip

I can launch and navigate the menu with the Wiimote, I can use the Wiimote horizontally as a NES controller for classic games and… I can aim and fire as a Lightgun!!!

Regards.

Locutus73

[Sorgelig]

So, Dolphinbar has BT receiver for wiimote?

[Locutus73]

So, Dolphinbar has BT receiver for wiimote?

Yes, that's the gist of the Dolphinbar. It pairs throuh BT up to 4 wiimotes and translates them into mouse/keyboard/joypad combos according to the selected mode. Think to it as a sort of 8bitdo retroreceiver for wiimotes. It has a 4th special mode for Dolphin emulator (probably superseeded by Dolphin BT passthrough mode).

Regards.

Locutus73

[Sorgelig]

I think i need to buy it.

[kitrinx]

While i don't have light gun, i have DualShock4 whose touchpad acts as touchscreen.
So, i'm implementing light gun through on it.
And it works well (after some using it some time).

How are you able to send these coordinates to the cores? I'd like to add this to zapper emulation so Aimtrak works.

[Sorgelig]

I hope i will will release MiSTer update with lightgun on DS4 today. I use analog joystick mode.

[onaryc29]

have someone try to use a wiimote with just a bluetooth dongle and a sensor bar which is powered by battery? It should work?

[Locutus73]

have someone try to use a wiimote with just a bluetooth dongle and a sensor bar which is powered by battery? It should work?

I didn't try that route, but if I should, I would start from here: https://github.com/abstrakraft/cwiid

[EDIT]
Reading the git cwiid seems very very interesting, it lets you map everything (including acceleration sensor) to everything (mouse, keyboard, joypad) with both absolute or relative values!!!
Very interesting, I think I will fiddle with it this weekend.
[/EDIT]

Regards.

Locutus73