Possible v9 Bug - Phantom Entry/Caching Issue

Possible v9 Bug - Phantom Entry/Caching Issue

avatar

Hello to my good friends at RDM Support!

I've just upgraded to v9.0.0.0 here (2 users so far - the other 4 are still using 8.4.6.0 until I gain confidence in v9), and have found some issues for you to look at. I will log each in separate forum topic for easier tracking.

The first relates to a Phantom entry when using Intelligent Caching. I have already investigated this one for you, and have resolved the issue, however I wanted to log it so that if there is a code bug you can fix it for future releases.

High-Level Summary

Symtpom - After deleting a Group/Folder using v9.0.0.0, the folder was still visible on my PC, despite vanishing from everybody else's PC (we use a shared SQL Server DB).

Cause - What I've determined is that the entry was still present in my offline.db file, which was presumably why it was still visible for me.

I know that you've dramatically changed the way that Caching and Offline Mode works in v9 to support higher performance and offline read/write use, and I suspect that this bug stems from those new features somehow.

Details

After I deleted the folder and it was still visible, I checked the Shared Logs to verify that it showed me deleting the entry - indeed it did, however the entry was still visible to me.

I tried using F5 to refresh the Data Source - no luck - the entry was still visible.

I tried closing RDM and then opening it up again - the entry was still there.

As I said, I then checked with other users, and for them the entry had successfully vanished.

When I looked at the dbo.Connections table in the SQL DB itself, I couldn't find the entry - it truly did appear to be deleted. But when I used PowerShell and did a Get-RMD-Session, the entry was still listed (presumably this occurs because the PowerShell obtains its data from the client, and would therefore pick up the same connections that are visible in the UI?).

By way of testing, I then created a new folder (verified that others could see it), then deleted it. It successfully vanished from all PC's (mine and others). So this phantom entry problem may only occur when a Group/Folder that existed PRIOR to v9 is deleted - TBC.

Since my PC was the only one affected by the problem, I realised that it was probably a Caching issue, so I started to look into that.

At the time I deleted the entry, my Data Source was configured as follows :

- Offline mode = Read-only (the default - I hadn't changed this since the client upgrade) Note that I wasn't using Offline Mode, I only include this for completeness.

- Caching mode = Intelligent (again, the default - I hadn't changed this setting)

I realised that my offline.db file needed refreshing.

Question : Shouldn't F5 do this? If you are successfully online to the SQL Data Source, a refresh surely should re-initialise the entire offline.db contents? Perhaps this is not the way you've designed it to work with "Intelligent" Caching...

I couldn't think of any other way of forcing the offline.db to be refreshed (considered deleting it, but was worried that would upset things).

So I re-configured my Data Source to DISABLE both Offline Mode and Caching Mode, and the entry vanished successfully. Upon re-enabling them (Read-Only/Intelligent again), however, THE ENTRY RE-APPEARED! It seems that RDM just went back to using the offline.db file that existed previously without refreshing it!

This whole issue raises some very important questions in my mind about your new Caching and Offline Modes :

- Should the Intelligent Caching mode be enabled by default? (Prior to version 9, our settings were to "Allow" Offline Mode, but nobody had used it yet - so presumably the default behaviour in previous versions was stay online only and not build an offline cache of the DB until someone clicked "Go Offline"? Also, this unsolicited building of the offline.db file slows the startup time of v9 RDM noticably)

- Should the UI indicate somewhere whether Caching and/or Offline Mode is being used, and which type/mode? (Like Outlook does with Cached Mode)

- Why is the size of my SQL DB 135 MB, however the size of the offline.db file is only 4 MB? This doesn't give me confidence that the database has cached all entries properly. (I know that the SQL DB contains tracking, logs, possibly attachments (?) and other things that don't get copied to the offline.db file, but our DB is relatively new, so I would expect that the majority of data in the DB is still raw credential and connection data).

- Is there another way to force a full refresh of the contents of the offline.db file?

- What is the difference between "Intelligent" and "Simple" Caching mode? (Nothing on online help yet)

The problem I see with all this is that this will affect ALL of your customers upgrading to v9, even when they aren't intending to use Caching and/or Offline Mode (it's the default setting now in v9)!

Sorry to dump such a monster problem on your lap - as a Programmer myself I know the pain of someone pointing out something of this magnitude!

P.S. - I have tried to replicate this issue since, and have been unsuccessful, unfortunately.

Regards

Des

edited by ceo_rdm on 10/3/2013

All Comments (18)

avatar

I will anser the question one/two at a time :-)

- Is there another way to force a full refresh of the contents of the offline.db file?

CTRL + refresh will clear the client cache and reload everything from the DB.

Stéfane Lavergne

avatar

- Should the Intelligent Caching mode be enabled by default? (Prior to version 9, our settings were to "Allow" Offline Mode, but nobody had used it yet - so presumably the default behaviour in previous versions was stay online only and not build an offline cache of the DB until someone clicked "Go Offline"? Also, this unsolicited building of the offline.db file slows the startup time of v9 RDM noticably)

For a new data source the default is: Offline Mode - Disabled, Caching mode - Intelligent.

If the you had "Allow offline" that maps to "Read/Only" offline in v9.

Prior to v9, if allow offline was checked the offline fill was synced on every action (add/edit/delete/refresh). As of v9, this is still the case, but done in a background thread.

"synced" in prior v9 was to write an XML file of the entire session list, in v9 only deltas get updated to cache (Intelligent cache). In all our tests this was faster in v9. Could you please start RDM with the "/Profiler" switch and post the result. This will display cache load times, DB refresh times and UI load times. Should help us diagnose why you are seeing a slow start of RDM.

Stéfane Lavergne

avatar

I've started the help topic for Offline Mode & Caching Mode but it's not done yet :-(

Quick overview.

Offline Mode feeds off Caching Mode. So I will start with Caching Mode.

Caching Mode is how we sync changes from the server.

- Disabled - on a client refresh action (edit, refresh, open...) the entire list of sessions is sent over from the server to the client. (slow)

- Simple - client keeps a "cache ID" locally, on a client refresh action the client verifies the cache ID against the server cache ID, in the case where it is different, the entire list of sessions are sent over from the server to the client.

- Intelligent - client keeps a cache version, on client refresh action the client sends to the server the cache version then the server responds with the delta of changes to be applied locally (add, edit, delete)

Offline Mode

- Disabled - no offline use of the data

- Read-Only - session data can be browsed and launched without connecting back to the server (SQL, RDMO, RDMS...)

- Read/Write - session data can be browsed/launched/modified

So even if you have offline mode disabled, the offline file (now a SQLite file, no longer a simple XML) may be still be present due to the Caching Mode specified.

Changed: I had wrongly specified MySQL instead of SQLite as the file structure.

edited by stefane on 10/16/2013

Stéfane Lavergne

avatar

- Should the UI indicate somewhere whether Caching and/or Offline Mode is being used, and which type/mode? (Like Outlook does with Cached Mode)

Bottom right corner of RDM we have the offline status.

edited by stefane on 10/4/2013

Stéfane Lavergne

avatar

- Why is the size of my SQL DB 135 MB, however the size of the offline.db file is only 4 MB? This doesn't give me confidence that the database has cached all entries properly. (I know that the SQL DB contains tracking, logs, possibly attachments (?) and other things that don'tget copied to the offline.db file, but our DB is relatively new, so I would expect that the majority of data in the DB is still raw credential and connection data).

Double check your DB file size, maybe the file growth.

You can run disk usage report (SSMS):

Right click on the database

Navigate to Reports > Standard Reports > Disk Usage By Table

[/ol]

The offline file on stores info from Connections table and a few computed values. It is normal that it is smaller than the entire DB.

Stéfane Lavergne

avatar

As for the actual bug, thank you fro the detailed explanation. I will have a look and try to reproduce the issue.

Have I missed anything? Let me know.

Best regards,

Stéfane Lavergne

avatar

CTRL + refresh will clear the client cache and reload everything from the DB.

Perfect! Thank you! This gives me the exact tool I wanted on Friday.

A suggesting for future releases :

Allow this "full" refresh to be invoked by keyboard shortcut also, eg - CTRL-F5

Thanks

Des