"Floating" vim selection in SSH

I select some text to copy elsewhere by dragging my mouse over.
Then when I start scrolling the text with my cursor keys, the selection "box" does not scroll, and just starts highlighting whatever scrolls into it. Confirmed this does not happen in Putty, with all else being the same (same box, same .vimrc, same account)

Example: I will select 40ms


scroll down a line or two, and now the highlight is here:



Interestingly, the actual clipboard text is still "40ms", but the highlight/selection box moving like that is very distracting.

Thoughts?
TIA,
-=Matt