This post is more than 5 years old
3 Posts
0
3607
January 14th, 2011 07:00
.NET Centera with Stream override
I have a need to override Stream in my application because we write many files to Centera in the same blob.I have a Stream class that overrides system.io.stream, and it reads the files in sequence and supplies them to the wrapper Read function.
I have tried the regular FPStream( filename, intptr) and it works fine, but I tried to supply my stream to the FPGenericStream class. It kinda looks like this:
FPPool thePool = new FPPool(threadData.URL);
FPClip clipRef = new FPClip(thePool, "StreamClip");
FPTag fileTag = clipRef.AddTag("FileBlob");
MultiFileInputStream mfis = new MultifileInputStream( fileList); // this over-rides system.io.stream
FPGenericStream gfs = new FPGenericStream(mfis, userData);
fileTag.BlobWrite(gfs,FPMisc.OPTION_CLIENT_CALCID);
Where MultFileInputStream overrides system.io.Stream, so when FPGenericStream reads, it reads from mfis.
This seems to work great, I have traced thru the .NET wrapper to the point where the info->mEOF is set to true, but then the Centera reports an error, and I get a Seek back to zero, and this continues in a tight loop.
I have attached a Centera log file, you can see that the data is transferred OK, then EOF is set, but at about line 409, we get an error:
message=BlobIDMismatchException: BlobIDMismatch on node 14977c10-1dd2-11b2-8f1e-8c434d072249 [0UM368STVC812AHAPLOSG9L6Q0, 09CG0HVCOO7EK7SGNOTIC0J079]
followed by a reset and Seek to zero.
any help would be appreciated.
gstuartemc
2 Intern
•
417 Posts
0
January 19th, 2011 05:00
Dale - as we discussed via email, the rest to the first byte only happens once and is correct. This is becaase you are using CLIENT CALCID (which reads the data to calculate the MD5 and then rewinds to stream it across if it is not on the cluster) rather than CLIENT CALCID STREAMING (which streams the data over as it calculates the MD5).
The GenericStream implementation assumes that the Read method of the underlying stream will return the number of bytes requested. If less than that number are returned this this viewed as the end of the stream. In your case this must have happened on one of the passes, resulting in different MD5s for the calulate and data transfer passes.
docHacker
3 Posts
0
January 19th, 2011 06:00
Thanks for your help, changing to streaming worked.
Since in our application we override the standard .NET Stream class, we will be returning values that match the needs of the SDK. Users of the standard IO Read function in .NET need to be aware that Read will not always return the number of bytes requested when the IO system is heavily loaded, which will fool the Blob Write function into quitting early.
I would suggest when using the SDK to override the .NET Stream class so it matches the SDK's needs, or making a small change to the SDK to always drain the Stream to the requested number of bytes.
Thanks again for all your help. Things are working great!!