Unsolved
This post is more than 5 years old
10 Posts
0
719
October 26th, 2009 04:00
XAM Java API and global options (embedded.data.threshold) not working ?
Hi,
I'm trying to use emmbedded CDF data, via the Java XAM API
I've set com.emc.centera.embedded.data.threshold (on the XamLibrary) and com.emc.centera.buffersize (on the XSystem) to 100K, but when I use the CLI to check pool capacity, it still shows me that it creates 2 C-Clips per file (and hence 4 Objects)
Since I need to copy about 15 million small documents there will be a big capacity loss (or early limit reach) if I can't solve this issue.
PS: same documents stored via CUA (70k embedded) shows an almost 1on1 relation between C-Clips and files
Can anyone give some working examples on how to achieve this ? Or is someone prepared to review my current code ? [ largely based on "StoreContent" sample ]
I'm trying to use emmbedded CDF data, via the Java XAM API
I've set com.emc.centera.embedded.data.threshold (on the XamLibrary) and com.emc.centera.buffersize (on the XSystem) to 100K, but when I use the CLI to check pool capacity, it still shows me that it creates 2 C-Clips per file (and hence 4 Objects)
Since I need to copy about 15 million small documents there will be a big capacity loss (or early limit reach) if I can't solve this issue.
PS: same documents stored via CUA (70k embedded) shows an almost 1on1 relation between C-Clips and files
Can anyone give some working examples on how to achieve this ? Or is someone prepared to review my current code ? [ largely based on "StoreContent" sample ]
No Events found!
holgerjakob_c0722c
2 Intern
•
337 Posts
0
October 26th, 2009 06:00
In order to find out the increase in object count I would use the node list in Centera Viewer and check what the effective increase in object count is.
I'm not 100% sure but would expect the pool capacity output to show you the clip and the file.
As for the increase in clips you should really only see one clip per file. The embedding has nothing to do with the clip count. There is one per clip written to Centera.
Holger
twinkeloogje
10 Posts
0
October 26th, 2009 07:00
when I do "show pool capacity" it returns the following list :
rfnd_pp 10 MB 1,297 GB -- 21588013 21584420
cata500prd 8 MB 328 GB -- 11912310 5956087
default -- 0 GB -- 0 0
ConsoleArchive -- 0 GB -- 0 0
SystemArchive -- 0 GB -- 0 0
Total -- 1,626 GB -- 33500505 27540507
And Centera viewer confirms a total Object count of about 75,000,000 objects used
holgerjakob_c0722c
2 Intern
•
337 Posts
0
October 26th, 2009 08:00
The frnd_pp pool shows a 1-2-1 relationship between clips and files.
cata500prd shows 11 Million Clips and 5.9 Million Files. Only half of the clips have an associated blob.
I would open the node list, display the column used object count. If you happen to have a stable amount of used objects, write your object and see if the object count increases by 2 (if embedded blob). (right click on the node to choose the displayed columns)
Should you not have a stable object count I can also send you a profile that you can used on my internet Centera's. I can then look into the clips and see if your object was really embedded or not.
Best regards, Holger
twinkeloogje
10 Posts
0
October 26th, 2009 15:00
the rfnd_pp pool is the one which has been loaded via a CUA
I'm currently halfway in loading the same files via my own Java/XAM program - I need both ways to be able to compare access performance on a large dataset - but I'm afraid that the results will be biased if the XAM solution needs to lookup/load 2 clips instead of 1
holgerjakob_c0722c
2 Intern
•
337 Posts
1
October 27th, 2009 00:00
Are you writing to the same pool with your XAM Application?
As said, I would inspect the CDF with the CLI Tools (read the CDF) that would show you if the blob is embedded as the CDF would directly containt the base64 encoded blob in the CDF.
Contact me on holger.jakob@informatio.ch if you need a copy of the CLI Tools. I can also provide a short description on how to use them.
Best regards, Holger
twinkeloogje
10 Posts
0
November 5th, 2009 05:00
Embedded Blob settings work correctly as intended, but additional C-Clips get created by XAM API to store Non-Binding fields.
For more info, I can refer to some documentation from Stuart Graham :
https://community.emc.com/servlet/JiveServlet/previewBody/1271-102-2-1404/Transitioning%20from%20the%20Centera%20SDK%20to%20XAM.pdf