Mouse pointer accuracy and keyboard issue

Hey Kris

For the cursor:

Do you notice just the "click" in the wrong place? Or does the cursor itself seem displaced? You can normally notice when the "mouse over" highlight of things activates in the wrong location.

Do you notice that on all machines, or just some? Can you let us know an example of the display setup? For example, the resolution and scaling factors of the local and remote monitors.

For the keyboard;

It sounds like a modifier key is getting "stuck" on the remote system (perhaps we are sending a key down for a modifier, but for some reason not sending the corresponding key up which makes your future actions behave as though the modifier is held down).

Again, do you notice that on all machines, or just some? It sounds like there's some pattern of usage that might be triggering this. Can try changing the setting of "Session" > "Apply Key Combinations" that you are using and see if that has an effect?

Finally, it would be helpful to know if the remote machine(s) are using the .exe or .msi normally, and if they are running the latest version or something old.

Thanks and kind regards,

Hi,

about cursor:

only "click" is at wrong place, cursor is at normal position...
when I hold LMB, it's start select, it's start draw "select rectangle" from strange position

No I didn't notice any special machine.. it's very random.. I have it today for example, but I don't remember where, so probably I'll not check what you need, but as I wrote before.. sometimes it's start at first session, so I close connection, connect again and it's fine.

about keyboard:

I thing it's always start together with mouse cursor - the same users computers.
Generally I don't use exe file.. 99% i've installed msi files so it works as service.
90% user computer have newest version 2020.1.7, that's why I mentioned that it can be something wrong with this build... I didn't notice problems like this never before.

When I have it next time I'll back here with more details.

--
KriS

Small update how repro for keyboard problem: (mouse was fine this time)

1. use RDM
2. open wayknow remote session (tabbed)
3. use alt+tab to switch apps at local computer
4. do something (in my case it was chrome where I check smth)
5. use alt+tab again back to RDM
6. magic starts ;)

Workaround is hit left ALT one time or twice.

Hello

Thanks for the update. It sounds like from the description; when you Alt+Tab into a different application, when you release the "Alt" key then - of course - RDM doesn't have the focus, so the "Alt" key up is not send to the remote machine. This causes the "Alt" key to appear stuck down on the remote machine and as you say, magic starts.

In the Wayk Now client, we have some logic to protect against this: when the focus leaves the Wayk Now session for another window, we release any keys that we pressed on the remote machine. We then re-sync the keyboard state when the Wayk Now session gets the focus again.

I'll raise this with the RDM team and post back here once I have an update on that. Thanks again for the bug report and sorry for the inconvenience,

Hi,

about cursor:

only "click" is at wrong place, cursor is at normal position...
when I hold LMB, it's start select, it's start draw "select rectangle" from strange position

any chance you use high DPI settings on either machine? I think I did experience something similar a while ago, would need to check tho.

Hello,

I don't know at which customer and on which computer it was, but if it caused by some DPI settings it should work bad at every connection right?
In my case it was looks like:

1. connect customer
2. work few min
3. I start miss click - can't work
4. close connection and reopen - everything is fine.

Another case:

1. open connection - can't work cus miss clicks
2. reopen session - everything is fine

I don't need change anything at customer site.

--
KriS

Hello

Yes, exactly: if this was the result of improperly normalizing the mouse coordinates between the client and the server, we'd expect to see the same result for the whole duration of the session. The issue shouldn't come and go.

I have to admit to being quite confused on this one; we'll take another look at things next week but if you notice that behaviour and can identify any cause or way to reproduce it, that will be very helpful.

In the meantime, I saw that the RDM team committed a fix for the keyboard issue you experienced - I expect that will be in the next release, but I'll confirm that back in this post.

Thanks and best regards,

Small update with wayknow service logs.

I connected first time at customer computer, first login and again problem with mouse. Keyboard was fine.
I disconnect session, reconnect and everything was fine.

BTW - I recorded session, if you wan't I can share it with you, but it's unusable, because mouse pointer wasn't recorded so we don't see where mouse point is.

--
KriS

Hello again

Thanks for the update.

Yes, a session recording won't show much because we don't actually include the cursor in the display capture from the remote machine (so it won't be present in the recording). Unfortunately the log is not really helpful in this case either: input events are forwarded to the capturer, which is hosted in a separate process with it's own logging. We can ask it to trace the mouse messages but the result will be extremely noisy and hard to make sense from, so I'll call it a last resort.

Can you confirm that the mouse issue is when you connect using the Wayk Now client, or with RDM?

With regard to the keyboard bug you reported - I see the fix is integrated but doesn't have a version tagged yet. I expect it should be resolved in the next release of RDM, so please let me know if you still get that after the next update.

Thanks and kind regards,

Hi,

thx for update. If you have any idea more how I can help, don't hesitate tell.
It's not a critical bug and workaround is quick reconnect... so let's wait.. maybe I'll recognize any pattern when it's happen.

I'm not sure at 100%, but I thing it was at Wayk now stand alone client.

--
KriS

Maybe try a local recording using for example sharex which can show your mouse cursor.

Hello,

• it's happen at random computers (so I can't prepare recording software)
• it's customer computer so Idk if I can install any software
• try install new software when you don't know where you click :)

--
KriS

I think there might be a misunderstanding.

just to make sure, you are the one controlling your customers' PCs right?

because I meant using sharex (which also runs portable) on the side of the person who is doing the controlling. there by you can see both your local cursor, as well as where the actions land up being on the remote PC.

and also you can run most software even without any mouse input (having experience with a wireless mouse with terrible battery life can tell you a lot in terms of Keyboard usage outside of just typing. :-)

Hello again

A screen recording will probably not be very helpful, so don't trouble yourself to provide one. I understand the issue, but it's not clear right now why it would happen.

In Wayk Now, we don't render the cursor client-side. The cursor you see is your local computer cursor. When you use the mouse locally, we send the mouse events to the remote machine. You don't see the cursor of the remote machine, but you see the effect it has on the desktop (when clicking, dragging, etc).

When a mouse event occurs, here's what happens: on the client side, Windows gives us an X/Y coordinate within the session window. First we have to map that coordinate to an absolute position on the desktop image - this is affected by the zoom level and scroll position of the session window. The mapped coordinate is transmitted to the server, where it has to be further normalized: first, to account for display scaling, and then to map the coordinate to an absolute position within the entire virtual desktop. That coordinate is then used to create a mouse event that is injected into the OS.

A mistake at one point would cause the result you described. We have fixed similar bugs in the past - historically, it's usually been related to incorrectly scaling a mouse coordinate for a high DPI display. What I don't understand currently is how the issue could occur on connection, and then be resolved by reconnecting; or how that issue could start occurring during an active session.

That's why I've gone into some detail in how things work - perhaps you'll notice a particular pattern with regard to the influencing factors:

• display configuration of the remote machine
• changing displays of the remote machine
• zoom setting or level and/or scroll position in the Wayk Now window

Please also make note of if you see that with RDM or Wayk Now, as that is also a large factor.

Thanks again and kind regards,

@my1
I've some agreement between me and customer.. I can't install/run every software which I want, especially if problem is not related with my customer PC.
Yesterday I quick check getsharex webistine I found only installer, but now at github I see portable version... but it's still don't gonna help resolve issue and only show how it's looks like, but I thing Richard exactly know how it's.

@Richard
OK, let's wait then.. maybe we'll figure out whats going on, when I'll have it next time.

Hey Kris

I wanted to let you know that I did reproduce this issue once this evening. I determined that the problem was not on the client (I could see that the mouse coordinates were correct when dispatched to the server), but the remote machine was running a release build and I couldn't get any further insight into the problem.

We're no closer to determining the cause, but I wanted to give you a small update.

Thanks and kind regards,

Hi,

I just want to tell you that at latest version wayknow (not RDM) I did have the same issue.

--
KriS

Hi Kris

Thanks for the update. We didn't forget about this issue; we have had some sporadic reports of the same thing but have not been able to reproduce the problem.

It is helpful to know that you saw that with Wayk Now and not RDM; although I strongly suspect the issue is server-side anyway.

What makes it really confusing is that the problem seems transient and comes and gos without changing any configuration. As we said before, an offset cursor could be caused by incorrectly scaling mouse coordinates. But scaling is performed according to the display scaling factor - if that doesn't change, then cursor coordinate scaling should not be fine in one moment and then "wrong" in the next.

It's also extremely tricky to trace mouse inputs, especially via text logs. Mouse movement generates a lot of events and tracing them degrades performance significantly.

From our side, we'll take another look at this and see what diagnostics we can add to help track it down. In the meantime - if you notice any relevant pattern or oddity when you see this happen it would be very helpful. Are your customer machines typically multi-monitor? High DPI displays? Do you commonly switch between remote monitors, adjust the remote resolution or display scaling?

Thanks and kind regards,

Hi,

it's happen without any pater and totally random. I can connect for weeks at this customer without problem and one day it's happen.
I don't remember where I connect yesterday but it was probably laptop with 150% scaling.
Today I did have the same issue with another customer laptop (another company) and Im sure there was some % scaling but I don't know exact number, but window screen was smaller then 1920x1080.

If I well remember it's happen at desktop too, where I've connected monitor without scaling.

Some time ago I thought about internet connection/latency/speed, because when I've this problem mouse cursor move very slow.. and it's hard click something.. when I reconnect it's looks normally and cursor issue gone.

--
KriS

yesterday I also had some mouse weirdness. I connected from a windows 8.1 to a WIndows 10 (no idea which one it was) and a Win Server 2019 1809.

in both cases sending ctrl+alt+del via the menu, then resolved the issue, and you could just click cancel on the screen that appears.

Hello

Thanks Kris for the continued feedback and also my1 whose prior comment provided a good hint.

I'm happy to say I believe we've resolved this issue, and that fix will be available in the next release. I'll post back here with an update on that.

Thanks and kind regards,

Hello

Just a quick update to say that Wayk Now 2020.2.5 is now available for download, and contains a fix for the offset mouse issue. If you interested, there is a post-mortem of this bug on our blog.

Thanks and kind regards,

cool to know, also just by the way the signature doesnt look too good in dark mode especially the name being black on dark grey.

cool to know, also just by the way the signature doesnt look too good in dark mode especially the name being black on dark grey.

Thanks! It should be fixed now.