Unsolved
This post is more than 5 years old
8 Posts
0
3869
June 3rd, 2010 03:00
lost pages
Ive been asked by our developers about the possible impact of increasing FP_OPTION_MAXCONNECTIONS to 200 in relation to a recurring problem (im not a developer..i only support the hardware).
Apparently applications reading >100pages at a a time have regularly returned 10-15% of the pages as corrupt gibberish.
This has not been a problem in the past with very few records being written this large..but is now becoming a problem.
Previously for one offs the developers have a script to pull large records one page at a time.
Its been suggested that the FP_OPTIONS_MAXCONNECTIONS setting (currently left to the 100 default) be increased to 200.
Any comments on the problem , if this is right direction to go on ...or if this parameter will bring other peformance improvement or degredation??
The original notes from th developers attached below.
Thanks
Marc
We are planning to set the MaxConnections per FPPool object used to 200. Currently we use 1 FPPool object per Weblogic cluster node.
Currently we have 4 weblogic clusters in production so totally WorkSmart can have 800 connections(at max) with Centera.The SDK will initially allocate a smaller number of connections and allocate more (up to the limit) when required.
We are using the FPLibrary(Centera SDK) to open connections
The existing code snippet
com.filepool.fplibrary.FPPool;
centeraConnectionPool
= new FPPool(poolAddress + "?" + peaFileLocation);
For production we use the following 2 entries in the config file
%CENTERA_PEA_FILE_LOCATION%=/opt/Centera_SDK/wms_prod-rw.pea
%CENTERA_POOL_ADDRESS%=centera02.production.local
we looked into the API document of FPPool.After API analysis found
1. we set only one global option
"openstrategy" to 0
2. we are not setting
FP_OPTION_MAXCONNECTIONS Global option on the FPpool object.
FP_OPTION_MAXCONNECTIONS
("maxconnections") - The maximum number of connections in the connection pool maintained by the FPPool object. If your application uses multiple threads, each thread will use a separate connection up to the limit specified here. The default value is 100; the maximum value is 999. To take advantage of this feature, the application must use multiple threads against a single FPPool object.
Daeja viewer uses ImageRetrievalServlet to retrieve individual pages from Centera using Document Retrieval Service (multithreaded access to FPPool)
Newly added code snippet
Currently we have added the following code to set the Max Connections global option to 200.
FPPool.setGlobalOption(FPLibraryConstants.
FP_OPTION_MAXCONNECTIONS, 200);
Please let us know if this change is fine with Centera.
mckeown_paul
409 Posts
0
June 3rd, 2010 06:00
Marc
Why do they think that increasing this will stop the pages being read back as corrupt?
FYI - we recommend that in total no more than 20 current IO threads be used per access node (rule of thumb you could probably run with a bit more) so if they are intending to have 800 IO threads then you will need approx 40 Access Nodes.
I suspect that they are probably not intending to do this but please check with them.
Normally the limit of 100 is fine as typically most application never actually reach the 20 IO threads per Access Node anyway and have a much smaller requirement.
If they are getting corrupt content back then only the following scenarios can occur
- the content was corrupted before it was sent to the Centera, so the Centera thinks its what you really want to store
- when they read it back from the centera their logic in some way corrupts it.
If the content was written correctly to the centera then the centera will always return it correctly or will return an error. It will never return corrupted data. It can validate the data as when it reads back the content, it reruns the hashing algorythm and if the key generated does not match the content address supplied by the application then it knows its been corrupted and returns an error. It could be its corrupted on the centera or in transit across the network but in either case the sdk would return a blob id mismatch error or in some network cases a socket error.
If you want please get them to post more detail on the issue or email the CenteraCSE@emc.com mailing list
Also I would recommend that if this is happening on a production system that they log a fault call with EMC Customer Services as if there really is corrupt data on the Centera then you will need to engage EMC CS
MarcG2
8 Posts
0
June 3rd, 2010 06:00
Thanks for the reply Paul...im going to pass your comments on now.
I forgot to add i dont have a definition of 'corrupt' ie if its gibberish or what.
However i do know that repeated requests for the same pages will return different pages 'corrupted'.
ie the source data on the Centera is correct, but is not consistently returning it correctly via the API.
Marc