[ISSUE] DPH PAM Service module stopped working?

[ISSUE] DPH PAM Service module stopped working?

avatar

I cant remember exactly when it stopped working but its been a few weeks now and since then ive tried reinstalling the MSI or upgrading to new versions, also added new Application Users with the required permissions. But still for some reason my local Service is failing to make contact with my DPH Bus instance, Application Log Event Errors im seeing are:

Category: Devolutions.Hub.Services.Business.Services.HubContextService
EventId: 0

Error : Hub Key could not be decrypted. ManagedError :
InvalidParameter
===Original Message===
Exception of type 'Devolutions.Cryptography.DevolutionsCryptoException' was thrown.

Category: Devolutions.Hub.Services.Pam.PamBackgroundService
EventId: 0

[Devolutions Hub PAM Service] Could not connect to Devolutions Hub. Make sure to configure your Application User properly in the installer.

Like I mentioned I tried reinstalling and recreating the App Users after seeing these errors but it hasn't helped.

Any Ideas

Thanks

JK
Devolutions Force Member (and Long time Devolutions Fan)

All Comments (9)

avatar

Hi John,

This problem can occur if the application secret and application key are inverted in the configuration setup. Could you validate that the app secret and app key are indeed properly set and not switched? If the problem persists, please open a ticket at service@devolutions.net, we'll setup a call to diagnose the issue with you.

Best regards,

Luc Fauvel

Luc Fauvel

avatar

I did and have checked and double checked I'm using the right fields, i use the downloaded app id pdfs from my hub that plainly shows the field name and value which match up exactly with the hub service installer (side topic, is there a pwsh cmd to change the hub service app id values? I've been uninstalling and reinstalling to change values which is long so wondered what the alternative is), should I still try reversing the values then or not, also just so you know it was running so was setup correctly but recently I performed a clean install and since then haven't been able to regain the connection. What else should I try? I've gone through the docs for setting this up and setup a new app id tied to my PAM service on hub, but for whatever reason my hub service just isn't able to speak to my hub instance?? I have also used Sysinternals TCPVIEW and was able to see the service contact an IP so i'm assuming its making a connection to hub instance, so from your reply guessing its authentication related issue.


If you have any other suggestions please provide.

Thanks,

JK
Devolutions Force Member (and Long time Devolutions Fan)

avatar

You can change the application identity values by opening the file in C:\ProgramData\Devolutions\Hub\PAM Service\appsettings.json

As for your issue, just off the logs you provided we can't really tell what might be the cause and we cannot reproduce internally. However, in the appsettings file, you can change the log level from Information to Debug, this will allow us to see in more detail what the issue could be.

Best regards,

Luc Fauvel

Luc Fauvel

avatar
You can change the application identity values by opening the file in C:\ProgramData\Devolutions\Hub\PAM Service\appsettings.json

As for your issue, just off the logs you provided we can't really tell what might be the cause and we cannot reproduce internally. However, in the appsettings file, you can change the log level from Information to Debug, this will allow us to see in more detail what the issue could be.

Best regards,

Luc Fauvel


@Luc Fauvel

Will give that a go....

Thanks

PS. Sorry got another completely diff topic again soz, only simple one this time. I am using the latest RDM beta but i keep seeing a notification banner telling me that my RDM and DPH data versions are out RDM wants 2025.2 but my DPH is 2025.1, normally that gets updated after an upgrade on startup doesn't it. Any idea why that didn't happen this time?

JK
Devolutions Force Member (and Long time Devolutions Fan)

avatar

apologies for delayed response, I think the PAM module service finally took the App creds well i didn't see the crypt errors this last time round. But the installer form for the details is still looping twice and its done that the last few updates, which is very annoying, you fill out the host url, ID and SECRET hit test or just next and you lose the field data lol, but the 2nd go it works fine lol...... Also the last time i installed I installed all modules PAM, Encryption and Reports, normally I skip the Encryption component, but installing it caused a the double loop to occur twice lol so 4 times I had to enter the field data this go. You may need to look into that as I've clean installed win 11 twice over last couple months and its always don't the same so I can rule out my install at least.

Anyway, this try with PAM I ran into yet another issue, there must been a change I hadn't noticed regarding Approver members? I have 2 checkout policies for my PAM with each set on 2 folders in the PAM vault, one folder I use checkout approvals the other I don't, both also allow users to approve themselves and provide a reason as an optional. Well I received an Error prompt again, this time from what I could pick out was not having any Approval members?? But it did the same for both checkout policy paths, which confused me further seeing as one policy I've tried to set to not need any additional inputs as there for my main tenant creds where as my other policy is for my 365 dev tenant I use...

So long story short I've had to snooze using PAM now so I can just use RDM without issues using user vault cred searches. On this topic has RDM still not rolled out that Linked creds (External Vault) feature yet, I've been looking for it but haven't noticed it yet.

Thanks

JK
Devolutions Force Member (and Long time Devolutions Fan)

avatar

@john.kenny

But the installer form for the details is still looping twice and its done that the last few updates, which is very annoying,


That's strange, we've never reproduced that internally, could you send a video of the behavior? (Without any sensitve information) at https://devolutions.sharepoint.com/:f:/s/SessionRecordings/EiYJqpvo4F9FhZFOth2eHPEBW-uH5zBus7dMnKain4NOvw the 1 prompt per service is normal as you'd have to create seperate app user credentials for each, but more than that isn't.

Anyway, this try with PAM I ran into yet another issue, there must been a change I hadn't noticed regarding Approver members? I have 2 checkout policies for my PAM with each set on 2 folders in the PAM vault, one folder I use checkout approvals the other I don't, both also allow users to approve themselves and provide a reason as an optional. Well I received an Error prompt again, this time from what I could pick out was not having any Approval members?? But it did the same for both checkout policy paths, which confused me further seeing as one policy I've tried to set to not need any additional inputs as there for my main tenant creds where as my other policy is for my 365 dev tenant I use...


I'll look into this and get back to you, this is with the latest PAM service version 2025.1.500 correct? Could you also provide the error you are getting?

On this topic has RDM still not rolled out that Linked creds (External Vault) feature yet, I've been looking for it but haven't noticed it yet.


The new Linked Credential (External Vault) is only supported in RDM with SQL server at the moment, we're working on getting available using DVLS and Hub.



Luc Fauvel

avatar
@john.kenny
But the installer form for the details is still looping twice and its done that the last few updates, which is very annoying,

That's strange, we've never reproduced that internally, could you send a video of the behavior? (Without any sensitve information) at https://devolutions.sharepoint.com/:f:/s/SessionRecordings/EiYJqpvo4F9FhZFOth2eHPEBW-uH5zBus7dMnKain4NOvw the 1 prompt per service is normal as you'd have to create seperate app user credentials for each, but more than that isn't.

Ive uploaded the info, does PAM and Reporting in the module need different app creds then??

Would that be why its looping twice? Although I notice that if i select PAM and Reporting it loops twice, if i select all 3 inc Encryption then it loops 4 times, so that could explain the extra loop of the duplicate loop

Anyway, this try with PAM I ran into yet another issue, there must been a change I hadn't noticed regarding Approver members? I have 2 checkout policies for my PAM with each set on 2 folders in the PAM vault, one folder I use checkout approvals the other I don't, both also allow users to approve themselves and provide a reason as an optional. Well I received an Error prompt again, this time from what I could pick out was not having any Approval members?? But it did the same for both checkout policy paths, which confused me further seeing as one policy I've tried to set to not need any additional inputs as there for my main tenant creds where as my other policy is for my 365 dev tenant I use...

I'll look into this and get back to you, this is with the latest PAM service version 2025.1.500 correct? Could you also provide the error you are getting?


Latest PAM service yes

once I pass the member error the 2nd error is still the same as it was before,


On this topic has RDM still not rolled out that Linked creds (External Vault) feature yet, I've been looking for it but haven't noticed it yet.

The new Linked Credential (External Vault) is only supported in RDM with SQL server at the moment, we're working on getting available using DVLS and Hub.


Thanks




@Luc Fauvel

JK
Devolutions Force Member (and Long time Devolutions Fan)

avatar

OK so looks like the dual loop is because I'm selecting the multiple functions of the service which I didn't know before, although I'm about to count the loops with all 3 now, where are the hub service functions designated on hub administration or is that done via permissions? I'm just wondering why a single app cred cant do all the services jobs, I do understand why It would but it would be handy to be offered another option regarding the use of separate app creds in the installer if we wanted too.....

Right so installing the 3 functions it does the double look for app creds then it asks for a url and port for the encrypt service which I've no idea where to point too, so that answers my original question. If the documentation has been updated so that it mentions the use of separate app creds for PAM and Reporting then I missed it again, although I cant say I've every noticed the extra parts including where to point the Encryption part to URL and Port?? Also, remember I mentioned this but I did start with PAM working and not changed when It originally stopped working which after that point I've not been able to reinstate it. I will take a look but the latest version seems doesn't loop extra when Encryption is installed so my query has mostly been answered regarding the looping.

To add, does look like the Docs DO NOW show the separate app creds on different pages, I'm going to try PAM in RDM with just PAM running to see if I still see the original error.

Thanks for that,

JK
Devolutions Force Member (and Long time Devolutions Fan)

avatar

No, so no error anymore, so the original error most likely was linked to using Reporting in the Service perhaps without the DPH PAM app permissions set, I think Im going to start over from the very start, dump PAM resources from Entra, DPH and RDM and start over, hopefully the Docs on Devo have caught up with the releases and see how it goes, as now i see an EntraID cannot reach error which I know for a fact was working when I originally posted about the RDM error lol...

Think Ill post a Feature request while I'm at it for DPH or the Service to be able to generate PS1's to setup EntraID and DPH app creds for the PAM service..... (don't mean to create more work but I've had nothing but probs every few updates using PAM to the point I'm thinking about dropping to come back at future date...)

JK
Devolutions Force Member (and Long time Devolutions Fan)