Version 2022.3.4.0 (November 10th 2022)

Core - Added a warning message if a vault has too many entries

Could you elaborate on this? Is there a limit to the number of entries in a vault, and if so, what is that limit? Is that a hard limit or a soft limit? We use multiple vaults and each vault tends to be filled quite a bit, and one reason we went with Devolutions Server was due to the no limit on the vaults.

Hello John,

Thank you for voicing your concern.

There are no hard limits for the number of entries in a vault; the warning is only to notify you the performance can be degraded if you have more than 5000 entries in the vault.

There are multiple factor that come into play, such as the connection between Devolutions Server and the SQL, if it is on-premise or in the cloud, how far away you are from the server hosting Devolutions Server, VPN, etc. The limit that we settled on is a general guideline, but if performance is good for you with 8000 entries, for example, you do not need to change anything!

Best regards,

Hi,

We see a bit of memory-leak-like-behavior in out test environemt (that is all the memory it can consume):


We know our Valuts are way too big but we didn't have this issue with older version (last one we tested was 2022.2.10)

Best regards,
Rok

Hello Rok,

Thank you for reporting this problem.

How many users are connected to your DVLS instance?

Is DVLS the only website running on this server?

Have you tried to restart the DVLS instance using the Stop Server/Start Server button on the DVLS Console to see if that help to reduce the w3wp.exe memory usage?

Best regards,

It is the only web site. No users yet this is initial start after reboot:


PS C:\WINDOWS\system32> get-process w3wp | select -Property *


Name : w3wp
Id : 3664
PriorityClass : Normal
FileVersion : 10.0.17763.1 (WinBuild.160101.0800)
HandleCount : 1881
WorkingSet : 847544320
PagedMemorySize : 2113691648
PrivateMemorySize : 2113691648
VirtualMemorySize : 500584448
TotalProcessorTime : 00:35:26.8593750
SI : 0
Handles : 1881
VM : 2233883578368
WS : 13732446208
PM : 14998593536
NPM : 481946
Path : c:\windows\system32\inetsrv\w3wp.exe
Company : Microsoft Corporation
CPU : 2126,859375
ProductVersion : 10.0.17763.1
Description : IIS Worker Process
Product : Internet Information Services
__NounName : Process
BasePriority : 8
ExitCode :
HasExited : False
ExitTime :
Handle : 2392
SafeHandle : Microsoft.Win32.SafeHandles.SafeProcessHandle
MachineName : .
MainWindowHandle : 0
MainWindowTitle :
MainModule : System.Diagnostics.ProcessModule (w3wp.exe)
MaxWorkingSet : 1413120
MinWorkingSet : 204800
Modules : {System.Diagnostics.ProcessModule (w3wp.exe), System.Diagnostics.ProcessModule (ntdll.dll)
, System.Diagnostics.ProcessModule (KERNEL32.DLL), System.Diagnostics.ProcessModule (KERNE
LBASE.dll)...}
NonpagedSystemMemorySize : 481946
NonpagedSystemMemorySize64 : 481946
PagedMemorySize64 : 14998593536
PagedSystemMemorySize : 650704
PagedSystemMemorySize64 : 650704
PeakPagedMemorySize : 2113691648
PeakPagedMemorySize64 : 14998593536
PeakWorkingSet : 903397376
PeakWorkingSet64 : 13788299264
PeakVirtualMemorySize : 602759168
PeakVirtualMemorySize64 : 2233985753088
PriorityBoostEnabled : True
PrivateMemorySize64 : 14998593536
PrivilegedProcessorTime : 00:25:41.2812500
ProcessName : w3wp
ProcessorAffinity : 3
Responding : True
SessionId : 0
StartInfo : System.Diagnostics.ProcessStartInfo
StartTime : 2022-11-17 15:41:51
SynchronizingObject :
Threads : {3668, 3864, 3940, 4016...}
UserProcessorTime : 00:09:45.6562500
VirtualMemorySize64 : 2233883578368
EnableRaisingEvents : False
StandardInput :
StandardOutput :
StandardError :
WorkingSet64 : 13732446208
Site :
Container :


After 50 minutes it is still struggling to start (it is not failing but it takes forever to start). Server (Win 2019) is dedicated has 16GB of RAM and SQL locally.

This is the memory usage graph (prior to yesterday we ran 2022.2.10, yesterday I refreshed from production and upgraded from 2020.3.18).


Best regards,
Rok Berlec

I found the source of the "memory leak"
We were getting

DevolutionsCryptoException - NativeError :  InvalidSignature  ===Original Message===  Exception of type 'Devolutions.Cryptography.DevolutionsCryptoException' was thrown.    ===Original StackTrace===     at Devolutions.Cryptography.Utils.HandleError(Int64 errorCode)     at Devolutions.Cryptography.Managed.Decrypt(Byte[] data, Byte[] key, ILegacyDecryptor legacyDecryptor)     at Devolutions.Server.ConnectionManager.DecryptDataAtRest(Guid contextId, String value, String context) 

few 100 times every second written in LogMessage table.

After I ran Remove security povider it started working normally.

Should I activate Encryption At Rest?

Best regards,
Rok

Hello Rok,

Thank you for your feedback.

That a good thing you have found that the problem comes from the Security Provider. That would have been my first recommendation to remove the Security Provider as it generates the CryptoException error messages.

Regarding the Encryption at REST, it's safer to enable it to encrypt the data entries in the database (connections, private vaults, documentation and attachments). We strongly recommend storing your Recovery Kit or the encryption keys in a secure, yet easy-to-remember location outside of Devolutions Server, such as in Password Hub Business, Azure Key Vault, or AWS Key Management Service.

Let us know if you have further questions about this.

Best regards,

Hi Erica,

We probably will need to adjust the timeout paramter in datasource that you introduced, is there a way to push that settings to all clients?


This is much too low for our way too big vaults,,,

Best regards,
Rok

Hi Rok,

Using the Custom Installer would allow you to propagate that setting by creating a new custom installer package with the new data source setting.

Sadly there is no method to push that setting globally.

Best regards,

Hi,

Are you willing to create an instalation package with 60 sec as default setting?

If we push new data source confgiuration we will overwrite a bunch of stuff that users have set to fit their needs (https://forum.devolutions.net/topics/34095/custom-path-for-default-installed-addons#168717)...

Best regards,
Rok

Hi Rok,

Thank you for your feedback.

Creating a installation package with 60 seconds is not in our plan.

You could try to export your options file including your data source with the Connection timeout set to 60 seconds in a .cfg file. Then, rename the file to default.cfg and place it in the RDM installation folder.



On opening RDM, the user should get the following dialog.



Use New Configuration (Lose Mine) will replace all data sources with the one exported in the file. But it will also replace all local settings and options the user may have set.

It's a workaround. Not the best one but it is possible to push the new settings this way.

Another workaround would be to export the data source in a .rdd file and ask the users to import it and remove the old data source.
https://help.remotedesktopmanager.com/datasource_importexport.html

Let me know if you have further questions about this.

Best regards,

Hi Erica,

we created a custom installation, pushed new datasource confgiuration, wrote instructions to users "how can they change parameter by themselves" - because most of the users will not read the instructions and at the same time are they afraid to touch the button saying LOSE MINE. Few hours after upgrade I am mostly dealing with connection timeout (appart from running bulk edit to fix the inheritance problem Devolutions credated with VPN value).
This is probably questions for some architect in your team: Why 15 seconds?

Best regards,
Rok

Hi,

Another thing that you / we can implement is hard limit of size of vaults (quota)...
In bigger organizations we let departments to create the structure of vaults the way they wanted to. Now we are dealing with the yellow baner. Is there any way to configure the limit so the vaults cannot grow over recommended size. Once they are too big it is not so easy to "undo" the growth. If we move stuff we need to inform responible and the users. How do we inform 500 users that the thing they are looking for was moved to another vault (3 months ago)?

Best regards,
Rok

And the bulk edit that would edit only folders that have no real-value on VPN settings overwrote ALL VPN settings to inherited (seems like in 2022.3.4 vpn.enabled behaves a bit differtnetly)...

if($connection.vpn.enabled -ne $true){
	$connection.vpn.mode = "inherited"
}
$RDM.Save();

Can we see from anywhere what the previous value was?

Hi Rok,

Thank you for your feedback.

We understand that not every users will read the notice you sent them to modify the Connection timeout parameter. For the 15 seconds, it's usually a value that will fit most data source cases. I will get back to you regarding the default value set to 15 seconds if I get a different statement from the engineering team.

About hard coding the number of allowed entries in a vault, I will definitely open an improvement ticket.

Finally, for the entries you have updated the VPN settings, you can use the history to compare what information has been updated. Please see the following online help page about it.
https://help.remotedesktopmanager.com/home_sessionhistory.html

Let me know if you have any further questions.

Best regards,

Hi,

One more thing I have noticed with shiny new data source config file we have pushed to all our clients.


Of course I cannot check rdi file that I created if I included username variable since it is not readable.
At the same time I exported rdd file and I can see I did have this paramter configured in my data source configuration

Funny thing is that RDM with this new custom rdi works as expected (UPN seems to come to web browser from somewhere, maybe that is the default setting), it is only that users now cannot change their time out parameter (sometimes even 60 sec in not sufficient) since username cannot be blank and they have to fill out that field first...
Yeah, I know it is not a big problem for you, I can instruct all our users to manually edit data source configuration correctly. I guess I must be doing something wrong...

Best reagrds,
Rok

OK, my mistake, I forgot to check Include data source credentials in the rdi...

This is a bit hard to test before you upgrade the servers...

Best regards,
Rok

Hi Rok,

That's a good news you have found that the option to include the data source credentials.

About testing this before upgrading your production environment, have you think about deploying the new version in a staging environment? Here is the method to create a staging DVLS instance based on your production instance.
https://kb.devolutions.net/dvls_staging.html

Best regards,

Hi,

Can you explain if around 100% more network traffic after upgrade from 2020.3.18 to 2022.3.4 is expected behavior?


We have vaults that are way too large to fetch over VPN even with max timeout 100 if the client's bandwith is not at least 10/10MBit. This was never an issue before.

Best regards,
Rok

Hi Rok,

Thank you for reporting this behaviour.

To troubleshoot this behaviour, I will open a ticket on your behalf and ask some reports from your RDM and DVLS installation.

Best regards,

Hi Erica,
probably I have found one of the causes and that is that rdi that I exported and created installation with has:
<ProfilerLevel>167</ProfilerLevel>

I know you should always reset the profile level, I have heard that before...
Insteresting though, the username variable didn't jump into config file because I forgot to add that option and this one is presumably one of the exported options every organization needs since you don't have to select anything (and you cannot deselect it either)?

Best regards,
Rok

Hi Rok,

I'm sorry for the late reply.

That's right, the profiler level should be set back to 1 once you have finished with the Profiler tool.

And about the username variable, we indeed need to enable the option to include it in the configuration file for the custom installer package. In specific situation, this option is not required as they want to let the user configure themselves the data source configuration in the RDM data source.

Let me know if you have further questions about the configuration file for the custom installer package.

Best regards,