retrieve file is slow with java/XAM ?

Question

Hi,

I'm developing an application in Java to read files from Centera.

During testing of the application, the recovery time of the Centera files are very large. Here is my code:

private ObjetoEletronicoVO criaVO(XSet xSet) throws XAMException, IOException,APICenteraException {
  
  Map
  
    propriedades = new HashMap
   
    ();
   
  
  ObjetoEletronicoVO vo = new ObjetoEletronicoVO();
  XStream xStream = null;
  String field;
  String valor;
  XIterator it = null;

  try{
  
       vo.setIdSistema(xSet.getString(APICenteraCAS.ID_SISTEMA));
       vo.setObjetoReferencia(xSet.getString(APICenteraCAS.OBJETO_REFERENCIA));
       vo.setTipoDocumento(xSet.getString(APICenteraCAS.TIPO_DOCUMENTO));
  try{
  
       it = xSet.openFieldIterator("br.cnpq");
       while(it.hasNext()){
  
            field = (String) it.next();     
            valor = xSet.getString(field);          
            propriedades.put(field, valor);
       }
  }finally{
  
       if(it!=null){
  
            it.close();
       }
  } 
  xStream = xSet.openXStream(APICenteraCAS.ARQUIVO, XStream.MODE_READ_ONLY);
  vo.setBytes(getBytes(xStream));
  }catch(FieldDoesNotExistException e){
  
       throw new APICenteraException(ERRO_ARQUIVO_NAO_ENCONTRADO);
  }finally{
  
       if(xStream != null){
  
            xStream.close();
       }
  }

  return vo;
}

private ObjetoEletronicoVO recuperarPorId(XSystem xSystem, XAMXUID xuID)
                 throws Exception {//removed other exceptions to make the code short for the post.
  XSet xSet;
  ObjetoEletronicoVO vo;
  xSet = xSystem.openXSet(xuID, XSet.MODE_READ_ONLY);
  try{
  
       vo = criaVO(xSet);
  }finally{
  
       if(xSet!=null){
  
            xSet.close();
       }
  }
  return vo;
}

mfh2 · Answer

Hello Cruzcunha - How big are the streams you are reading back from Centera? And when you say the recovery time is large, how much time are we talking about? Regards, Mike Horgan

cruzcunha · Answer

File size, time to retrieve 140 MB , : 2h15m 2 MB ,  68s 15 MB,  556 s

cruzcunha · Answer

Hi, Mike,

A EMC's technician did a check in my machine with JCenteraVerify, and the retrieve time was 10 times lower.

I realized that the versions of the JCenteraVerify's .DLL are newer than the ones I have. So, I downloaded the Centera_SDK-3.3.718, there's a version of XAM compatible with this version?

Thanks.

mfh2 · Answer

Yes, those times seem well below the normal range.

That said, if this is a new system you are developing, the recommended integration path is through the Centera SDK. XAM support for Centera is being deprecated by EMC, see the announcement toward the bottom of the main community page (https://community.emc.com/community/edn/centera).

I don't know if EMC has produced a XAM package compatible with the 3.3.718 SDK series, perhaps someone from the CSE team could comment.

Good Luck,

Mike Horgan

cruzcunha · Answer

No, this isn't a new system, it was already developed with XAM.

mckeown_paul · Answer

Normally when there is a difference in reading via XAM it's due to using too small a buffer size for the IO, make it as large as the file if possible.  Certainly not something small like 8KB

cruzcunha · Answer

It seems I'll hava to change the implementation to the standard SDK Lib, instead of XAM.

But I'm having some issues: the XUID isn't the same identifier as the ClipID.

I tried to query the last files added to CAS and restore the content with the examples that came with SDK.

But the data format is diferent too.

Does anyone has a solution to this?

mckeown_paul · Answer

I'm in the process of writing a whitepaper which discusses these sort of differences between the two API so an extract from that...

Under the covers the Xset is actually a "normal" centera CDF and the XUID has the "real" content address in it.

If you base64 encode the XUID when you get it back from a commit, base64 decode it and then the first 8 bytes can be discarded and the content address starts at byteoffset 8 (from 0). You can use that with the Centera API to open the clip.

If you are using non-binding fields you cant directly access these values but if you really need to get at it let me know.

cruzcunha · Answer

Thank you,  Paul. It's working now! Let me ask another question: how can I retrieve the properties that I create in XAM with xSet.createProperty(String,boolean,String) ?

mckeown_paul · Answer

If you havent figured it out for yourself by now by looking at CDFs....

XAM adds xfields to the underlying Centera CDF in a different way to the Centera API. With the Centera API you would porbably just see the equivalent for xfields as custom attributes. However the XAM sdk actually creates each xfield as a tag with the tag attributes of type and value. Here is an example of an xset with three fields as viewed from jcasscript

CASScript>crv

...

so to get the value of your xfields you would need to use FPClip_GetTopTag() to position yourself at the first tag and then use FPTag_GetTagName() to see if you are the correct tag. If not call FPClip_FetchNext() and see if thats it and repeat if not

If it is you can then use FPTag_Get Attribute() providing the Attribute name "value", Note there is no float or double type so you can just use string and do the conversion yourself.

cruzcunha · Answer

Thanks Paul,

but I couldn't get the values yet. I did what you said, but I got this:

name:EMC Centera Binding XSet Clip

creation time:1368061724000

number of blobs:1

number of attributes:18

retention period:0

size:142916608

number of tags:1

retention class name:

->creation.poolid:1dc7b8b8-1dd2-11b2-9400-c58e15cf4352-7

->retention.period:0

->retention.base.enabled:true

->sdk.version:4.0.698

->modification.poolid:1dc7b8b8-1dd2-11b2-9400-c58e15cf4352-7

->type:xam.binding

->name:EMC Centera Binding XSet Clip

->creation.date:2013.05.09 01:08:44 GMT

->modification.date:2013.05.09 01:09:17 GMT

->creation.profile:appcentera

->modification.profile:appcentera

->numfiles:1

->totalsize:142916608

->142916608

->refid:31LAN5FPP4GSRAV9UIQ639N2IB

->clusterid:1dc7b8b8-1dd2-11b2-9400-c58e15cf4352

->prev.clip:

->clip.naming.scheme:MD5

->numtags:1

....Top TAG

tag name: com.filepool.fplibrary.FPLibraryException: Operation not (yet) supported

....calling fetch Next:

tag name:stream:

->read-only:false

->type:application/octet-stream

->size:0

....calling fetch Next:

not found(returned null)

mckeown_paul · Answer

i used jcasscript to do this which is just a centera api application. COuld you email me your code and I'll try and find some time to look at this again paul.mckeown@emc.com

cruzcunha · Answer

I'll try to use jcasscript first to learn how it is saved in the c-clip. Thanks.

Centera

retrieve file is slow with java/XAM ?

Was this post helpful?