VMs deduplication ratio = 0

Hi @plpawzaww ! Unfortunately I don't have a prod environment to compare this to but I can get to my test machine after a few days. Would it be possible for you to share the NetWorker binaries that you have installed with me? I can try and reproduce this in my environment. 

What exacly do you mean? What binaries?

can you show the exact output, so to know what/which 0 you mean? the entries after v1:?

Also this is only stating dedupe of that one single backup, and says nothing about the total dedupe achieved for that VM for all of its backups, which - for accounting/billing reasons - is still a big omission, needing to use DPA for that to try to extract that information instead of NW being able to request that from the DD in question.

KB https://www.dell.com/support/kbdoc/en-us/000023290?lang=en states how to interpret the 4 values after v1:
# mminfo -q "ssid=3190711499" -S
ssid=3190711499 savetime=10/03/2014 10:02:55 AM (1412326975) vm-lego-231:/etc
 
*ss data domain backup cloneid: 1412326975;
*ss data domain dedup statistics: "v1:1412326975:50823364:1625378:738007";

For this save set:
cloneid                                       :  1412326975
backup size                               :   50823364
pre local-compression size   :   1625378
post local-compression size :   738007

But there also have been issues in the past with the way NW data was ingested and internally copied to its location on a DD, no longer showing any dedupe statistics.

KB articles like https://www.dell.com/support/kbdoc/en-us/000029940?lang=en , https://www.dell.com/support/kbdoc/en-us/000038202?lang=en , https://www.dell.com/support/kbdoc/en-us/000037010?lang=en all suppose these issues were addressed and fixed in later (nw9.x) versions. But might be a regression. Wouldn't be the first (nor last). I will have a look at what nw19.5 shows.

"Cause

In the NW logs for a VM it states the backup path being used:

"BackupPath": " / // / /xx/xx/xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx",

NW device name might depend on your own naming convention...

On the DD it then shows for that structure and all files belonging to that backup:
DD# filesys show compression /data/col1/ / /xx/xx/xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx
Total files: 14; bytes/storage_used: 625.5
Logical Bytes: 161,063,040,372
Original Bytes: 161,529,216,136
Globally Compressed: 277,142,565
Locally Compressed: 257,383,297
Meta-data: 856,984

If you know the long NW SSID name of a backup (using mminfo query with "ssid(60)", then you can list all the files beneath a certain DD folder from another system running an ssh command towards the DD to grep for the long SSID name:

ssh "files show compression /data/col1/ / recursive"|grep xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx-xxxxxxxx

With / I mean :

= nw server shortname mtree name as shown by :

# ddboost storage-unit show

= subfolder of mtree/storage unit as shown by

# ddboost storage-unit show

So that are the exact same numbers as mentioned in the backup of the VM on NW server end. I f however those are actually accurate, is - as always - the question...

when looking at a nw19.5.0.6 environment with a DD running 6.2.1.30-663869, I see that mminfo -S shows information for all 4 fields:

"v1:1677169830:161529216136:277142565:257383297";

for a :

*vm_backup_level: incr;

So that would be a backup size of around 161GB (NW shows 157GB so that matches, probably a difference between GB and GiB), while pre local-compression size is 277MB and post local-compression size would be 257MB.

While the stats in the logfile (as it would also state in the logs on the vproxy):

DiskStats: CSV: Generate stats in CSV format for friendly import to spreadsheet.
DiskStats: CSV: VM MORef,VM Name,Disk Label,Mode,Bytes Moved,Total Time (sec),Overhead (sec),VDDK Wait (sec),Read Rate (b/s),Write Rate (b/s)
DiskStats: CSV: vm-xxxx, ,Hard disk 1,hotadd,7325810688,245,30,0,50211748,204174189
DiskStats: CSV: vm-xxxx, ,Hard disk 2,hotadd,65536,60,30,0,134346563,48889658

So there is still a difference between what the VC states regarding the amount of CBT data that has changed and would need to be protected (7GB for disk 1 and 64KB for disk 2) and the amount of data that actually needed to be transferred to the DD.

Hence I wonder what you see in your case for both the logs on NW logs end vs. what mminfo -S states?

From mminfo:

*ss data domain dedup statistics: "v1:1677017103:68762732187:0:0"

I don't have logs from NW log now. I will try to post those later.

thank you for your answer, in my opinion deduplication works fine in my case but I would like to check which virtual machines have a worse dedup ratio and which a better one.

Accuracy of statistics is not very important to me, I just want to have a reference point.

Currently, all I have is information that the level of deduplication ratio is 0 (which is obviously not true).

In your case you see this informations, I wonder why I don't see it. I'm wondering if it's a problem on the Networker's or DDOS side, the links you provided above indicate that the problems occurred in both cases.