Forum / Remote Desktop Manager - Beta

Performance release 2019.2.x

  • Create an Issue
  • Cancel


After doing some tests with different releases to investigate significantly lower performance, I note that the releases after release 2019.1.41 all have that performance prob. Going back to 2019.1.41.0 it was 'as usual'.

When selecting for example a credential-session the Dashboard-pane needs a few to many seconds before the data/information from that session is available and visible in the dashboard. During that time RDM does not respond and the Dashboard section is 'black'.
Found here in release 2019.2.3.0 and 2019.2.5.0 and after reinstalling release 2019.1.41.0 this was workable again. (Just to be sure, I went to release 2019.2.5 again and the problem was present again)

64 bit, MFA Azure SQL dtatabse, around 8000 sessions in maps.

Do more users have this behavior and is this known and reproducible for you?

Clock3 mths

Hello,

Could you enable the profiler (https://help.remotedesktopmanager.com/help_profiler.htm) and see if anything is logged in either the performance tab or the debug tab? For the performance tab, please check both "debug" and "SQL", and for the debug tab, "Debug", then reproduce the issue. You can send me these logs through the forum's private messaging system or to hmireault(at)devolutions.net

Can you also send us your application logs by going in Help -> Applications Logs -> Report -> Send to Support.

Regards,

Hubert Mireault

signaturesignature

Clock3 mths

Hello Hubert,

Thnx for your reply and I will ensure that these files become avaliable as quickly as possible for you.
At the moment I can't switch to the 2019.2.5 release given the work I'm currently working on. And naturally want to work 'quickly' with RDM :-)

I certainly try to give feedback this week.

Regards, Marcel

Clock3 mths

Hello,

No problem! We've also been looking and we identified what might have been causing this issue. We should have a fix for it in 2019.2.6.0. Still, it would be useful to have the result of your profiler logs in case something else is the cause on your workstation.

Regards,

Hubert Mireault

signaturesignature

Clock3 mths

Hi Hubert,

Just installed release 2019.2.6.0 and this version is working great and even beyond expectations. Very fast in the functionality which in the indicated versions before gave significant problems. Super!

In any case, if needed I can send you the profiler logs from here with the version 2019.2.5.0, I'll generate them if I see time for this.
(With this new version I don't see the message(s) "Tread was being aborted" in the Application Log)

Regards, and thanks for this new release ;-)

Marcel

Clock3 mths

Thanks for the update Marcel, I'm glad it improved the performance for you. If you find the time to generate the profiler logs on 2019.2.5.0 it would be nice so we can verify if there were other potential slowdowns.

Regards,

Hubert Mireault

signaturesignature

Clock3 mths

Hi Hubert,

Unfortunately I notice a performance problem when checking OUT en Checking IN a credential. RDM is frozen for a few seconds.
See that this querie takes a long time:


--> select a.*, s.*, p.*, p.ID as 'UserProfile.ID'
from dbo.UserAccount a
inner join dbo.UserSecurity s on a.ID = s.ID
left outer join dbo.UserProfile p on p.UserID = a.ID
where s.UserType = 0
and (upper(s.Name) = upper(@Name) or upper(s.Name) = upper(@Alias));

I send you the logfiles to your emailaddress.
Happens in all versions btw..

Regards, Marcel

Clock3 mths

Hello,

Thanks Marcel, I've received the files. We'll take a look and see if it's possible to improve the performance when checking in/out the entries.

Regards,

Hubert Mireault

signaturesignature

Clock3 mths

Hello Hubert,

A short question, considering that we're also looking internally what can cause this. Are there specific ports RDM necessarily needs?
This because we see a clear difference if we connect directly (from my home address) to the azure SQL database or if we connect trought the company VPN.

Regards, Marcel

Clock3 mths

Hi Marcel,

Have a look at your VPN configuration/ports/routing. It is not normal that your SQL Azure database responds differently when connecting from home vs via your VPN. You can maybe test using SSMS (SQL Server Management Studio) to see if you can pinpoint the issue.

As for the Check In/Out issue can you please send us your SQL Schema for analysis? File > Data Sources > Upgrade (tab) > Email Schema to Support. Please add "attn: Hubert/Stefane Topic 32486" in the subject line.

SQL Port: 1433

Best regards,

Stefane Lavergne

signaturesignature

Clock3 mths

Hello,

We received your information successfully. After discussion with Stefane, we would like you to try something.

Can you use SSMS (SQL Server Management Studio)? If so, connect to your SQL Azure database, and in a new query window, enter the following:

DECLARE @name nvarchar(256) = 'name';
DECLARE @alias nvarchar(256) = 'alias';

select a.*, s.*, p.*, p.ID as 'UserProfile.ID'
from dbo.UserAccount a
inner join dbo.UserSecurity s on a.ID = s.ID
left outer join dbo.UserProfile p on p.UserID = a.ID
where s.UserType = 0
and (upper(s.Name) = upper(@Name) or upper(s.Name) = upper(@Alias));


Then, in this query window, right click anywhere and at the bottom press "Include Actual Execution Plan":
2019+09+17+9+12+28

It will create something that looks like this:

2019+09+17+9+14+42

You can then right click it and press "save execution plan". Can you send the result to my email address? It will help us see where the bottleneck is.

Regards,

Hubert Mireault

signaturesignature

2019-09-17_9-12-28.png
2019-09-17_9-14-42.png
Clock3 mths

Marcel,

Thank you for the information. I was discussing this issue with Hubert and looking at the code this afternoon and we believe that the issue has been identified.

If you could run the attached script to help us validate our assumptions. This script will output the amount of data stored for each user in the database. We only care about the last column "Row size (MB)". The underlying issue is RDM is not properly hitting its local users cache and re-fetching the user on each call, this plus the fact that we believe your user(s) might have big "Private Vaults" making the data fetch costly.


Best regards,

Stefane Lavergne

signaturesignature

RDM User Row Sizes.sql
Clock3 mths

Tnx for the input/responce Stefane :-)

I'll run this script tomorrow (Time here GMT A'dam) and in any case I can state that under my account many Private 'things' will certainly be present, given my RDM history and using it :-) .Internally we also referred ofcourse to the private vault to all of our users with which we also want to prevent local Password Storage.

Let you know!

Regards, Marcel


Clock3 mths

Marcel,

Sneak peek feature, we will be migrating the user's private vault into the same structure as the normal vaults. Doing this will bring us many advantages: better performance, bigger private vaults & added security, for example. Should be available in the final v2019.2 release.

Best regards,

Stefane Lavergne

signaturesignature

Clock3 mths

Hi Stefane,

Nice feature :-).
I'll send you the output from the script in a private message and as already told, my personal size is big..
For now you can close this post.

I'll first carry out further internal research for the users that experience problems with RDM. This concerns a little part of all users and first I want to see if this has other causes.

Thnx for your answers and help.

Regards, Marcel

Clock3 mths

Marcel,

Thank you for the private message. I was honestly hoping for an even bigger number, 1.2MB is not very big but might still explain the difference between your home vs. VPN speed. Some of the calls in the profiler are taking 8+ seconds to execute & transfer 1.2MB that doesn't quite add up.

Are you the only user that is this slow? How about the second one in the list? They are only at 0.25MB, faster for them? Everyone else (<0.05MB)?

Best regards,

Stefane Lavergne

signaturesignature

Clock3 mths


Hi Stefane,


As there were general complaints with the use of RDM here I sent all users a questionnaire internally in which they could indicate performance problems.
Here I also asked to indicate where, home, customer, work and with or without VPN ect.

As soon as I have completed this overview I will first check with the user(s) whether this problem is indeed present, and will bundle it and then give feedback to you. So far I have received a feedback from about half of all users where 30% indicating that starting RDM takes a long time, checking in-out and switching from vaults. I will keep you informed about this.

Regards, Marcel

Clock3 mths


Hi Stefane,

For your information (I'll send you the output private) a query I started on our RDM DB:
Maybe you can get more information for some performance problems.


-- GET EXPENSIVE QUERIES [MGO]
SELECT TOP 25 SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,
((CASE qs.statement_end_offset
WHEN -1 THEN DATALENGTH(qt.TEXT)
ELSE qs.statement_end_offset
END - qs.statement_start_offset)/2)+1),
qs.execution_count,
qs.total_logical_reads, qs.last_logical_reads,
qs.total_logical_writes, qs.last_logical_writes,
qs.total_worker_time,
qs.last_worker_time,
qs.total_elapsed_time/1000000 total_elapsed_time_in_S,
qs.last_elapsed_time/1000000 last_elapsed_time_in_S,
qs.last_execution_time,
qp.query_plan
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt
CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp
-- ORDER BY qs.total_logical_reads DESC -- logical reads
-- ORDER BY qs.total_logical_writes DESC -- logical writes
ORDER BY qs.total_worker_time DESC -- CPU time


Regards, Marcel

Clock3 mths

Thank you for the query output it was indeed helpful.

A few things you can try:

  • Create a new index (execute the following):

IF NOT EXISTS (SELECT * FROM sys.indexes WHERE NAME = 'IX_Connections_ConnectionType_SecurityGroup' AND object_id = object_id('dbo.Connections'))
CREATE NONCLUSTERED INDEX IX_Connections_ConnectionType_SecurityGroup ON dbo.Connections (ConnectionType, SecurityGroup) INCLUDE (Data);

  • Perform index maintenance on your database (might want to this off hours, might take some time)

In RDM, Administration > Pack Data Source (Optimize) > Index Maintenance (button) or ask you DBA

  • Please send me your database schema for analysis, all looks normal from the data you sent me but I want to be 100% sure

File > Data Source > Upgrade (tab) > Email Schema to Support


Best regards,

Stefane Lavergne

signaturesignature

Clock3 mths

Hi Stefane,

Database Schema sended :-)
New index created and Index Maintenance done.

First impression after several times starting and using random functions:

Switching vault to private fault --> extremely faster as before
Check In and Check -Out --> Work perfectly, this has to be workable for the users
Overall performance quick look --> No complaints

I'll do everything again tomorrow during work-hours and inform you about this.

Regards, Marcel

Clock3 mths

Marcel,

Let me know what the performance looks like tomorrow when everyone is on the system.

The new index surely helped but the "Index Maintenance" is not for nothing. I would suggest you run the maintenance on a bi-weekly or monthly schedule to keep performance at its peak.

Best regards,

Stefane Lavergne

signaturesignature

Clock3 mths

Also, your schema is perfect.

Did notice that you have a vault with just short of 10,000 sessions, you might want to split it for performance reasons. We usually suggest 4,000 max range for best performance but it is not always possible.

Best regards,

Stefane Lavergne

signaturesignature

Clock3 mths

Hi Stefane,

Spoke todat a number of colleagues who previously reported problems and those I spoke indicate that things are going a lot faster :-) Also I performed many different functionalities and for now also a 'smiley'.
To get a good picture I 'll ask all the users for feedback next week as they've worked with it for more days. I will inform you about this.

Regarding the 'Index Maintenance' I already did run that a number of times before but that really gave minimal improvement compared to the index adjustment we made. We also several times deleted the Queryplans (freeproccache) but also minimal approvements.

Thanks for the support and I'll send as soon as possible the results/feedback.

Have a nice weekend!

Marcel

Clock3 mths