[2026.3.1] App Crash
Version: 2026.3.1
OS: 2025 Server Datacenter (latest Windows Updates - including the Out-Of-Band one)
.Net Core: 10.0.12 - Windows Server Hosting
IIS Hosted - App Pool running as a gMSA
Database: SQLite
Whenever I attempt to browse to PSU at all, even the /admin page, I get application crashes logged and the page errors out or the application pool crashes. This is the application crash I'm getting:
Faulting application name: w3wp.exe, version: 10.0.26100.1882, time stamp: 0xc6bd45bd Faulting module name: e_sqlite3.DLL, version: 3.53.3.0, time stamp: 0x4b3d3b00 Exception code: 0xc00000fd Fault offset: 0x0000000000115d7c Faulting process id: 0xB68 Faulting application start time: 0x1DD4B6FADEF5BD3 Faulting application path: c:\windows\system32\inetsrv\w3wp.exe Faulting module path: D:\Powershell Universal\e_sqlite3.DLL Report Id: 840d3897-5d38-49f7-9c2b-968392a41442 Faulting package full name: Faulting package-relative application ID:
(sometimes it references ntdll.dll, but that's typically a red herring)
This was happening on 2026.3.0 after upgrading from 2026.1.2, though that was routinely referencing D:\Powershell Universal\coreclr.dll.
What I've tried:
- Verified the ZIP download hash.
- Manually verfied files weren't blocked (I run a Get-ChildItem -Recurse | Unblock-File in the upgrade process)
- Re-extracted and copied over our appsettings.json and web.confg.
- Reviewed and updated the appsettings.json to contain any new settings that exist in the one from the zip
- Upped the logging to debug in appsettings.json, I see a lot of lines about loading license information, but nothing that looks useful
The only error (tagged warning but has error in the message) in the systemLog file for PSU is the following:
2026-09-23 11:26:35.921 -04:00 [INF][UniversalAutomation.Services.UniversalConfigurationService] Reading configuration for Variable
2026-09-23 11:26:35.925 -04:00 [INF][ProtoBuf.Grpc.Server.ServicesExtensions.CodeFirstServiceMethodProvider] RPC services being provided by UniversalAutomation.IPublicGitSyncService: 5
2026-09-23 11:26:36.048 -04:00 [INF][Microsoft.AspNetCore.Hosting.Diagnostics] Request starting HTTP/1.1 GET https://<internalURL>/login?returnUrl=%2Fadmin - null null
2026-09-23 11:26:36.109 -04:00 [WRN][Microsoft.AspNetCore.Session.SessionMiddleware] Error unprotecting the session cookie.
System.Security.Cryptography.CryptographicException: The payload was invalid. For more information go to https://aka.ms/aspnet/dataprotectionwarning
at Microsoft.AspNetCore.DataProtection.Cng.CbcAuthenticatedEncryptor.DecryptImpl(Byte* pbCiphertext, UInt32 cbCiphertext, Byte* pbAdditionalAuthenticatedData, UInt32 cbAdditionalAuthenticatedData)
at Microsoft.AspNetCore.DataProtection.Cng.Internal.CngAuthenticatedEncryptorBase.Decrypt(ArraySegment`1 ciphertext, ArraySegment`1 additionalAuthenticatedData)
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
at Microsoft.AspNetCore.Session.CookieProtection.Unprotect(IDataProtector protector, String protectedText, ILogger logger)
2026-09-23 11:26:36.334 -04:00 [INF][Microsoft.AspNetCore.Routing.EndpointMiddleware] Executing endpoint '/Login'
2026-09-23 11:26:36.354 -04:00 [INF][Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.PageActionInvoker] Route matched with {page = "/Login", action = "", controller = ""}. Executing page /Login
2026-09-23 11:26:36.372 -04:00 [INF][Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.PageActionInvoker] Executing handler method PowerShellUniversal.Login.OnGet - ModelState is "Valid"
2026-09-23 11:26:38.618 -04:00 [INF][UniversalAutomation.Services.UniversalConfigurationService] Reading configuration for Translation
2026-09-23 11:26:38.623 -04:00 [INF][UniversalAutomation.Services.UniversalConfigurationService] Reading configuration for Settings
Both reverting to the 2026.1.2 folder and reverting the VM restores functionality.
For reference, my upgrade process is as follows:
- Create new folder, set ACL security on the folder for the gMSA.
- Extract the ZIP.
- Unblock recursively
- Copy web.conf and appsettings.json from the existing PSU folder to the new folder
- iisreset /stop
- Rename folders (old one gets datetime stamped, new one gets renamed to the original path)
- iisreset (to start IIS)
The database/repository folders are separate and outside the path of the IIS site, and are not touched. Database access looks functional based on the systemLog, as in debug logging, I see it sucesfully executing queries:
2026-09-23 11:52:15.845 -04:00 [DBG] Creating DbConnection. 2026-09-23 11:52:15.846 -04:00 [DBG] Created DbConnection. (0ms). 2026-09-23 11:52:15.846 -04:00 [DBG] Opening connection to database 'main' on server 'D:\PSUDB\database.workflows.db'. 2026-09-23 11:52:15.846 -04:00 [DBG] Opened connection to database 'main' on server 'D:\PSUDB\database.workflows.db'. 2026-09-23 11:52:15.846 -04:00 [DBG] Creating DbCommand for 'ExecuteReader'. 2026-09-23 11:52:15.846 -04:00 [DBG] Created DbCommand for 'ExecuteReader' (0ms). 2026-09-23 11:52:15.846 -04:00 [DBG] Initialized DbCommand for 'ExecuteReader' (0ms). 2026-09-23 11:52:15.846 -04:00 [DBG] Executing DbCommand [Parameters=[@filter_UsableAsActivity='?' (DbType = Boolean)], CommandType='"Text"', CommandTimeout='30'] SELECT "w"."Id", "w"."BinaryData", "w"."CreatedAt", "w"."Data", "w"."DefinitionId", "w"."Description", "w"."IsLatest", "w"."IsPublished", "w"."IsReadonly", "w"."IsSystem", "w"."MaterializerContext", "w"."MaterializerName", "w"."Name", "w"."OriginalSource", "w"."ProviderName", "w"."StringData", "w"."TenantId", "w"."ToolVersion", "w"."UsableAsActivity", "w"."Version" FROM "WorkflowDefinitions" AS "w" WHERE "w"."UsableAsActivity" = @filter_UsableAsActivity ORDER BY "w"."CreatedAt" 2026-09-23 11:52:15.846 -04:00 [INF] Executed DbCommand (0ms) [Parameters=[@filter_UsableAsActivity='?' (DbType = Boolean)], CommandType='"Text"', CommandTimeout='30'] SELECT "w"."Id", "w"."BinaryData", "w"."CreatedAt", "w"."Data", "w"."DefinitionId", "w"."Description", "w"."IsLatest", "w"."IsPublished", "w"."IsReadonly", "w"."IsSystem", "w"."MaterializerContext", "w"."MaterializerName", "w"."Name", "w"."OriginalSource", "w"."ProviderName", "w"."StringData", "w"."TenantId", "w"."ToolVersion", "w"."UsableAsActivity", "w"."Version" FROM "WorkflowDefinitions" AS "w" WHERE "w"."UsableAsActivity" = @filter_UsableAsActivity ORDER BY "w"."CreatedAt" 2026-09-23 11:52:15.846 -04:00 [DBG] Closing data reader to 'main' on server 'D:\PSUDB\database.workflows.db'. 2026-09-23 11:52:15.846 -04:00 [DBG] A data reader for 'main' on server 'D:\PSUDB\database.workflows.db' is being disposed after spending 0ms reading results. 2026-09-23 11:52:15.846 -04:00 [DBG] Closing connection to database 'main' on server 'D:\PSUDB\database.workflows.db'. 2026-09-23 11:52:15.846 -04:00 [DBG] Closed connection to database 'main' on server 'D:\PSUDB\database.workflows.db' (0ms). 2026-09-23 11:52:15.846 -04:00 [DBG] 'ManagementElsaDbContext' disposed. 2026-09-23 11:52:15.846 -04:00 [DBG] Disposing connection to database 'main' on server 'D:\PSUDB\database.workflows.db'. 2026-09-23 11:52:15.846 -04:00 [DBG] Disposed connection to database 'main' on server 'D:\PSUDB\database.workflows.db' (0ms).
I will note that I don't see any logs about it talking to the main DB, just this workflow db.
The final lines in the systemLog are always:
2026-09-23 11:54:28.550 -04:00 [DBG][UniversalAutomation.Common.Interfaces.ILicenseService] Loading license information. 2026-09-23 11:54:29.129 -04:00 [INF][UniversalAutomation.Services.UniversalConfigurationService] Reading configuration for Translation 2026-09-23 11:54:29.135 -04:00 [INF][UniversalAutomation.Services.UniversalConfigurationService] Reading configuration for Settings 2026-09-23 11:54:29.139 -04:00 [DBG][UniversalAutomation.Services.ConfigurationScript] Reading configuration file settings.ps1 2026-09-23 11:54:29.140 -04:00 [DBG][UniversalAutomation.Services.ConfigurationScript] File exists. Deserializing items. 2026-09-23 11:54:32.682 -04:00 [DBG][Universal.Server.Routing.UniversalEndpointDataSource] Endpoints changed. Notifying change token.
Any help on where to start further troubleshooting would be appreciated.
All Comments (4)
Hello @SunnyJ
Thank you for providing such a detailed report and for documenting the troubleshooting you have already completed.
We have been reviewing several areas and behaviors in the latest PSU releases, and I am going to add this IIS and SQLite crash scenario to our current lab testing so we can compare the behavior under similar conditions.
The fact that reverting to 2026.1.2 restores functionality, while both 2026.3.0 and 2026.3.1 exhibit the crash, gives us a useful baseline for testing. We will also take into account the IIS hosting configuration, the gMSA identity, the SQLite database location, and the `0xc00000fd` crash being reported against `e_sqlite3.dll`.
For now, please confirm whether this is affecting a production environment and whether the system is currently stable again after reverting to 2026.1.2.
I will continue reviewing this against our lab environment and update the thread with the findings.
Best regards,
Ruben Tapia
Hello @SunnyJ
Apologies for the delay in getting back to you. I have been working through several lab investigations in parallel, and I wanted to make sure I had a meaningful update before posting back here.
From the first round of testing, I was not able to reproduce the crash in my lab with PSU 2026.3.1 under IIS using SQLite. A clean InProcess deployment remained stable, and I also tested an upgrade-style scenario using a 2026.1.2 baseline with the 2026.3.1 runtime without reproducing the w3wp.exe crash.
I was also able to confirm that the session cookie CryptographicException shown in the logs can occur without terminating the PSU process, so at this point I do not consider that warning to be the primary cause of the crash.
Since that first lab cycle, additional diagnostic results have been received through our internal support process. The engineers involved are coordinating those findings with the ongoing reproduction work so that the public thread and the internal investigation remain aligned and we do not ask you to repeat tests you have already completed.
The case is currently under active internal technical review. At this stage, we are not treating OutOfProcess as a validated workaround, nor are we attributing the root cause to the session cookie warning, SQLite, or any specific faulting module without stronger evidence.
No additional action is required from you through the forum at this moment. We are reviewing the latest evidence internally, and I will update this thread again today as we consolidate the confirmed findings.
Best regards,
Ruben Tapia
Hello @rubentapia
I tried updating to 2026.3.3 as it mentioned a fix for something that was likely affecting this server:
> Fixed a system hang on startup when an invalid license file was installed.
There is an old license installed from the old way that dev instances were licensed (via generating a dev license from a licensed PSU server). PSU version 2026.3.3 works and I am able to load pages/the admin interface now. You can mark this item as resolved.
Hello @SunnyJ
Thank you very much for the update and for confirming the results after upgrading to PSU 2026.3.3.
I am glad to hear that the application is loading normally again and that you are able to access the Admin Console.
The additional detail regarding the older license file is also very helpful, especially considering the related fix included in 2026.3.3. We will keep that information with the case findings as part of the final technical context.
Thank you as well for all the testing, logs, and follow-up information you provided throughout the investigation.
This topic can now be marked as resolved.
Best regards,
Ruben Tapia