Possible v9 Bug - Phantom Entry/Caching Issue

I will anser the question one/two at a time :-)

CTRL + refresh will clear the client cache and reload everything from the DB.

For a new data source the default is: Offline Mode - Disabled, Caching mode - Intelligent.

If the you had "Allow offline" that maps to "Read/Only" offline in v9.

Prior to v9, if allow offline was checked the offline fill was synced on every action (add/edit/delete/refresh). As of v9, this is still the case, but done in a background thread.

"synced" in prior v9 was to write an XML file of the entire session list, in v9 only deltas get updated to cache (Intelligent cache). In all our tests this was faster in v9. Could you please start RDM with the "/Profiler" switch and post the result. This will display cache load times, DB refresh times and UI load times. Should help us diagnose why you are seeing a slow start of RDM.

I've started the help topic for Offline Mode & Caching Mode but it's not done yet :-(

Quick overview.

Offline Mode feeds off Caching Mode. So I will start with Caching Mode.

Caching Mode is how we sync changes from the server.

- Disabled - on a client refresh action (edit, refresh, open...) the entire list of sessions is sent over from the server to the client. (slow)

- Simple - client keeps a "cache ID" locally, on a client refresh action the client verifies the cache ID against the server cache ID, in the case where it is different, the entire list of sessions are sent over from the server to the client.

- Intelligent - client keeps a cache version, on client refresh action the client sends to the server the cache version then the server responds with the delta of changes to be applied locally (add, edit, delete)

Offline Mode

- Disabled - no offline use of the data
- Read-Only - session data can be browsed and launched without connecting back to the server (SQL, RDMO, RDMS...)
- Read/Write - session data can be browsed/launched/modified

So even if you have offline mode disabled, the offline file (now a SQLite file, no longer a simple XML) may be still be present due to the Caching Mode specified.

Changed: I had wrongly specified MySQL instead of SQLite as the file structure.
edited by stefane on 10/16/2013

Bottom right corner of RDM we have the offline status.


edited by stefane on 10/4/2013

Double check your DB file size, maybe the file growth.

You can run disk usage report (SSMS):

Right click on the database
Navigate to Reports > Standard Reports > Disk Usage By Table
[/ol]
The offline file on stores info from Connections table and a few computed values. It is normal that it is smaller than the entire DB.

As for the actual bug, thank you fro the detailed explanation. I will have a look and try to reproduce the issue.

Have I missed anything? Let me know.

Best regards,

Perfect! Thank you! This gives me the exact tool I wanted on Friday.

A suggesting for future releases :

Allow this "full" refresh to be invoked by keyboard shortcut also, eg - CTRL-F5

Thanks
Des

OK, after a little more testing, I withdraw my comment that Intelligent Caching slows startup speeds, but to clarify why I made that comment in the first place :

- When you run v9 for the first time, it defaults to "Intelligent Caching" mode, and therefore needs to build the initial offline.db file. This initial build does slow load times that first time (as evidenced by the attachments "Intelligent Cache initial build - Test 1.txt" and "Intelligent Cache initial build - Test 2.txt") compared to subsequent load times, after the offline.db has already been built for the first time (see "Intelligent Cache subsequent load - Test 1.txt" and "Intelligent Cache subsequent load - Test 2.txt"). So the real question is - how do those load time compare when RDM is not using offline.db? Well, it turns out that you are right, the load time from the SQL data source when RDM in not using caching at all is slightly slower - see "Profiler with NO Intelligent Cache.txt".

- I believe that my comment may have also been fueled by another problem - slower startup speeds with using the 64-bit client (as per http://forum.devolutions.net/topic7313-bit-a-lot-slower-than-bit-client.aspx). I may have assumed at one point that the slower speed was due to the building of the offline.db file, but once I switched back to the 32-bit client, performance is much better.

So, after all that is said - no further action is required by you or me for now on this particular topic (load times using Intelligent Caching).

Cheers

Des

Thanks for the great overview of how Offline/Cachine mode works now in the new v9 - very helpful. Hopefully you'll include all those details once you get around to updating help.

I do have one extra question though, which is about delta updates. Your post said :

I don't understand how this can occur with an SQL Data Source. Surely this "delta" intelligence can only work with RDMO or RDMS back-ends?

So, does that mean that "Simple" and "Intelligent" actually do the same thing if we use an SQL Data Source?

Regards

Des

Hmmm - this doesn't work for me as depicted in your screenshots. If I'm in Offline mode, sure enough, I get the "red cross" icon (see Offline.jpg), however if I'm in normal, online mode, there is no "green tick" icon as you seem to get (see Online.jpg). I checked both 64-bit and 32-bit versions - same result.

As a suggestion for future version, perhaps you could include a logo that indicates the status of "Caching" as well? Something to differentiate the three modes - Disabled, Simple, Intelligent.

Regards

Des

Thanks for the great idea - I hadn't thought of checking it that way. I still got a discrepancy, though - see attached.

In SQL - the dbo.Connections table is showing around 16MB, but on disk, the offline.db is only 4MB.

I'm no longer too concerned about this, but I thought I would share.

Regards
Des

Once again, your responses to all the cases I logged on Friday has been superb - fast and helpful! Well done!

Keep up the good work.

Cheers

Des

Have a look at the system option (Always show "Go Offline" button)


edited by stefane on 10/9/2013

No, the Intelligent & Simple cache are very different. Simple cache refreshes all sessions if something changes. Performance degradation will occur if you have a few thousands sessions even a few hundred sessions.

Intelligent is currently available with SQL, MySQL, RDMS & RDMO data sources. The logic is actually in the DB not the server.

We are not sending deltas (fragments) of sessions but rather the delta of the data source. The delta is a combination of new/edits/deletes(removals) since that last refresh. In other words what has changed in the data source since the client last verified.

Deletes(removals)? What is that? Due to security group and such a session may no longer be available for a given user. If ever security has changed the client receives "remove" action for the affected sessions which forces the client to delete the sessions locally.

Note:
RDMS has it's own cache that caches the sessions in memory so it doesn't have to hit the DB every time a client hits the server. A query from the client will be served from memory. When a client call comes in, Same technology as the client but it's on the server.

Interesting, we do store more information in SQL but not 4 times as much. I would guess the size differences here are due to the differences in the underlying storage engines (SQL Server vs SQLite)

Thanks for the logs. I will investigate further the the "initial build" issue. You will notice that with the initial build files the line:


OfflineManager.GetDataSourceSettings : 1280 ms
So it takes 1.2 seconds to get the data source settings from the offline file, but since the offline file does not yet exist it is created. This could be done on a back ground thread make the initial load faster.

Note:ThreadQueue.AddRow : 167 msThe rows are added to the offline file in batches via a background thread.

P.S. I'm still working on the online documentation of the Offline mode & Caching modes. I will using most of what I wrote here as a starting point :-)

Stefane - Thanks for your thourough replies - all great info!

Des