A fix for this issue has been implemented in version 2025.2.14.0
Implemented

DVLS Bug: Mail Spam

avatar

DVLS 2025.2.12.0

Hi All

There is still a Mail Spam Bug available in current DVLS Release together with a Syslog Server.
When you reboot the SQL Server Instance while DVLS normal running, then DVLS spams Emails with Notifications, that the Syslog Server is not available anymore. And it do not recover. Only Solution was to completely disable the Syslog Server in DVLS Server.

Please implement a general Throttling for ALL Mailflow in DVLS, so this is impossible to happen.

Best Regards,
Andreas

All Comments (6)

avatar

Hello everyone,

I can confirm this bug. We have switched the SQL server to another node (SQL failover cluster) and are now constantly receiving emails to the Devolutions server administrators stating that the syslog server cannot be reached.

I have already restricted Devolutions server administrators to one user (I am the only one who should now receive emails), but the emails are still being sent to both previously stored Devolutions server administrators.

It almost seems as if DVLS has its own email queue through which emails are sent/forwarded?

I also removed the syslog server. Nevertheless, I am still being flooded with emails.
After removing the mail server connection and reactivating the syslog server, everything seems to be working “cleanly” again.

I will leave the mail server connection disabled for now.

Best regards,
Alex

avatar

Hi Andreas, hi Alex,

Thank you both for the detailed reports. I was able to reproduce the issue on my side, and we’ve opened a ticket with our development team to investigate and address this bug.
We’ll keep you updated as soon as we have more information or a fix available.

Best regards,

Jonathan Iannone

avatar

Hey Jonathan

There seems another Bug related to this Bug.
After this happing our DVLS Instance was incredible slow via RDM when changing Vaults or refreshing a Vault (up to 30s).
After that we troubleshooted via Profiler and looked also up into SQL Server.
In our case, the table dbo.NotificationEvent was filled up with 208953 rows which leaded into slow queries for this Table.

16:28:13.655630	REMOTE:    GET - /api/accessrequest/all : 0 ms
16:28:25.198640	            LoadConnections [Devolutions Server : ****** - Global Vault] - [Global] - Disabled - 64-bit : 13230 ms
...
16:28:37.386537	REMOTE:    GET - /api/notification : 7285 ms
16:28:37.386553	            DVLS - Api - .GetCurrentNotifications...
16:28:37.386556	                WebAPI.GetCurrentNotifications : 12164 ms
16:28:37.386557	            DVLS - Api - .GetCurrentNotifications : 12164 ms
...
16:28:37.452296	REMOTE:    GET - /api/secure-messages : 0 ms
16:28:37.452330	            DVLS - Api - .GetSecureMessages...
16:28:37.452336	                WebAPI.GetSecureMessages : 56 ms
16:28:37.452361	            DVLS - Api - .GetSecureMessages : 56 ms
16:28:37.452435	        LoadConnections UI - [Devolutions Server : ******] - [******] - Disabled - 64-bit : 25516 ms
16:28:37.452594	    RefreshConnections : 25516 ms
16:28:43.675271	REMOTE:    GET - /api/accessrequest/all : 0 ms
16:28:49.523261	REMOTE:    GET - /api/notification : 7417 ms
16:28:49.523278	    DVLS - Api - .GetCurrentNotifications...
16:28:49.523395	        WebAPI.GetCurrentNotifications : 12010 ms
16:28:49.523404	    DVLS - Api - .GetCurrentNotifications : 12011 ms
16:28:49.526904	ActionManager.RefreshConnections : 37592 ms
...


We cleaned up this table and now everything is fast again.

Here i have some very slow queries which took 4-6 seconds:

SELECT ne.*, ntne.ID AS RelationID, ntne.Status 
                FROM NotificationEvent ne 
                JOIN NotifierToNotificationEvent ntne ON ne.ID = ntne.NotificationEventID 
                WHERE ntne.NotifierID = @ID


DELETE TOP (1000) FROM dbo.NotificationEvent
WHERE 
    ((Processed = 1
    AND NOT EXISTS (
        SELECT 1 FROM NotifierToNotificationEvent nvn 
        WHERE NotificationEvent.ID = nvn.NotificationEventID)
	OR DATEADD(MONTH, 1, ProcessedDate) < GETDATE())
	AND NotificationCategory != 'EXPIRATION'
    )
    OR (
        NotificationCategory = 'EXPIRATION'
		AND Processed = 1
        AND (
            NOT EXISTS (
                SELECT 1 
                FROM dbo.Connections c 
                CROSS APPLY OPENJSON(NotificationEvent.Keywords, '$.FIELDS') 
                WITH (
                    NAME NVARCHAR(50) '$.NAME',
                    VALUE NVARCHAR(50) '$.VALUE'
                ) AS JSON_FIELDS
                WHERE JSON_FIELDS.NAME = 'lblEntryID'
                AND c.ID = JSON_FIELDS.VALUE
            )
            OR EXISTS (
                SELECT 1 
                FROM dbo.Connections c 
                CROSS APPLY OPENJSON(NotificationEvent.Keywords, '$.FIELDS') 
                WITH (
                    NAME NVARCHAR(50) '$.NAME',
                    VALUE NVARCHAR(50) '$.VALUE'
                ) AS JSON_FIELDS
                WHERE JSON_FIELDS.NAME = 'lblEntryID'
                AND c.ID = JSON_FIELDS.VALUE
                AND c.Expiration IS NULL
            )
        )
    )


Would be great if you can add at least a mechanism which avoids inserting that much entries, adding at least indexes for all called fields in your queries and optimize the queries for better performance.

Thanks and best regards
Andreas

avatar

Hi Andreas,
We released a fix for this syslog issue in our latest version 2025.2.14.0.
Could you please let me know if it works on your side?

Thank you

Marc-Andre Bouchard

avatar

Hello Marc-Andre,

The bug that caused users to be spammed with emails when the syslog server was unavailable has been fixed.
However, the syslog connection does not work with 2025.2.14.0.

I still have an instance running with 2025.2.12.0, where I receive syslog messages without any issues.

On a 2025.2.14.0 server in the Devolutions Console -> Logs, I found this entry:
SYSLOG_SERVER-ZBX_UNREACHABLE - Scheduler - Error
“Error connecting to host FQDN:1518” Error: “Unable to write data to the transport connection: An established connection was aborted by the software in your host machine..”

When the system was still running 2025.2.12.0, the syslog connection worked.

I don't know whether I should open a new report for this or whether it is related to this problem.

Best regards,
Alex

avatar

Hello everyone,

I just updated our test environment from 2025.2.12.0 to 2025.3.2.0.
2025.2.12.0:
Everything is running perfectly. The syslog server is displayed as UP in the system dashboard and the data is flowing smoothly into the syslog server.

2025.3.2.0:
As it stands at the moment, everything looks good. The syslog server is displayed as DOWN in the system dashboard, but in the syslog server setup, there is now a “Test connection” button that indicates a successful connection. The data flowing into the syslog server is also up to date.
So there is only a visual problem on the system dashboard page.

Best regards,
Alex

A fix for this issue has been implemented in version 2025.2.14.0