Ask the Expert: Object Storage into the Cloud

This Ask the Expert session is now open for questions. For the next couple of weeks our Subject Matter Experts will be around to reply to your questions, comments or inquiries about our topic.

Let’s make this conversation useful, respectful and entertaining for all. Enjoy!

Robert, thanks for setting this up.

Can someone outline the Best Practices for Centera to ECS Migrations?  Also, is there a formula for calculating how quickly the migration will complete?

Thanks,

Mike Horgan

Excellent, thanks Dean. I'd be very interested to know a baseline ECS Transformation duration expectation for something like 100M (100,000,000) small clips, say 50KB each from an imaging application.  Lets assume the Centera has 16 nodes and 4 access nodes which would seem pretty typical.

Best Regards,

Mike Horgan

Timely thread for me! 

Can someone explain the technical difference between Atmos and ECS or Point me to a detailed comparision?  I have looked at a number of "technical" presentations and everything I see suggests that ECS is an update/rewrite of Atmos with a few  changes such as containerization, xfs and a more modern stoage algorithm.  Conceptually, I see no major difference and the HW is basically identical. 

Thanks.

Simon

Hi Simon,

Though the hardware of ECS and Atmos can be similar and both have support multi protocol access (Note: ECS also has support for Hadoop and OpenStack Swift, refer to the ECS 2.2.1 Data Access Guide for details on ECS API support) , the underlying Architecture layers in each product work very differently. Comparing the documents below should help you here I think. Let me know if you need anything clarified.

ATMOS

Atmos 2.4 Administrators Guide -> https://support.emc.com/docu68366_Atmos-2.4-Administrator%E2%80%99s-Guide.pdf?language=en_US

Atm,os 2.4.1 Relesae Notes -> https://support.emc.com/docu70762_Atmos-2.4.1-Release-Notes.pdf?language=en_US

ECS

ECS 2.2.1 Planning Guide -> https://support.emc.com/docu70102_ECS_2.2.1_Planning_Guide.pdf?language=en_US

ECS 2.2.1 Data Access Guide -> https://support.emc.com/docu70101_ECS_2.2.1_Data_Access_Guide.pdf?language=en_US

ECS 2.2.1 Administrators Guide ->

https://support.emc.com/docu70100_ECS-2.2.1-Administrator's-Guide.pdf?language=en_US

Good morning guys,

I have currently set up an ECS 2.2 Single Node Docker.  I have been using this for archiving data to this endpoint, my problem is, I have used up all the space on the endpoint, after seeing this I then deleted the archives that I had sent to this endpoint but I dont see the available space increasing.  I use the EMC2 Dashboard.to monitor the space and efficiency of the endpoint.

Is there a way to reclaim the space?

I appreciate any help that can be given.

Thanks

Emma

Hi Mike,

While Dean looks into your query there is a Centera to ECS migration White Paper which you may find useful available @ http://www.emc.com/collateral/white-papers/h15115-centera-to-ecs-migration.pdf

The biggest difference is the policy engine on ATMOS.

If application simply need to set/update the proper metadata of an object. With the corresponding policy definition on ATMOS, these user- or system metadata are used as trigger for different protection/distribution of objects. This also covers retention enforcement, erasure service or migration to “public” cloud (if this is an ATMOS or ECS system).

--

With kind regards/Mit freundlichen Grüßen

Helmut Gotsche

Advisory Systems Engineer, EMEA

Object Storage

Software Defined Solutions@Emerging Technology Division

EMC² Computer Systems Austria GmbH

(Firmenbuch Gericht: HG Wien, FN 160018x, VATno: ATU43027201)

Am Euro Platz 1/4,

A-1120 Wien / Austria

Phone: +43 1 599 52 490

Fax: +43 1 253 67 22 1830

Mobile: +43 664 411 7466

e-Mail: Helmut.Gotsche@emc.com

More about ViPR: http://www.emc.com/cloud/vipr/index.htm

More about Atmos: http://www.emc.com/storage/atmos/atmos.htm

More about Centera: http://www.emc.com/archiving/centera.htm

For Developers: https://community.emc.com/community/edn/

The information contained in this communication is confidential.

It is intended solely for the person(s) to whom this email message is addressed.

If you have received this email message in error, please notify the sender immediately by telephone or email

and destroy the original message without making a copy.

P Please don't print this e-mail unless you really need to.

Thank you.

Thanks Michael. I browsed the paper and it looks very well done, kudos to the author.

Best Regards,

Mike Horgan

Hi Dean,

The data was deleted 2 days ago (Tues 21st).  Hard to say what the average size was to be honest.  We were archiving projects that were between 2GB and 240GB.

I was testing archiving a project with 0.5TB to the endpoint that only had 300GB space available, I wanted to see what would happen and how it would handle the large file.  It failed at 240GB and the archive was deleted (if an archive fails with our software, the archive is then deleted from the destination) however it still registered on the dashboard that the space was used and we were unable to archive large projects.

I hope this helps.

Thanks

Emma

A question concerning XOR recovery.

Assume that we have 8 DataCenters/VDCs (as shown in the ECS Storage Overhead illustrations in various places),  how many sites can we lose and still be able to recover all data? 

Thanks.

Simon

To expand on Dean's response a bit, normally content is XOR'd with content from all other VDCs, thus only allowing us to tolerate one failure.  However, there's an option in ECS when setting up a Replication Group to "Replicate to All Sites".  In this case, we maintain a full copy of the data on all sites in the RG and we could tolerate multiple losses.  It's really just a trade-off between availability and storage overhead.