ECS/Cloud Object Storage - DevOps Day

1© Copyright 2015 EMC Corporation. All rights reserved.
OBJECT STORAGE
TECHNICAL DISCUSSION

DATA GROWTH IS BREAKING TRADITIONAL STORAGE
SCALE-OUTFILE &OBJECT STORAGE CONTINUES TO GROW
• Overly complex
– Multiple data protection schemes,
protocols, management tools
• Can’t economically scale
– Inefficient, high overhead,
especially at geo-scale
• Not cloud-ready
– Not architecturally suited and no
self-service
Source: IDC EMC Digital Universe Study 2014

BLOCK FILE FILE BLOCK FILE
Today’s Storage Infrastructure
BLOCK

WHY OBJECT?

Object Storage Characteristics
Linear Scalability
Scales to billions of
objects
No Locking
No lock on write or
create operations
Geo-scale
Geo-replicated and
distributed
Support for large files
Object sizes are in TBs
Web friendly
Firewall friendly, http,
REST accessibility
Metadata and
extensibility
Objects can be extended
to multiple policies
(Immutability, retention,
etc…)

An Object platform offers …
 A flat namespace of millions of buckets
 Buckets that scale to billions of objects
 Geo distribution, protection, access
 User meta data as a first-class entity
 Snapshot consistency semantics
 Multi-tenant access & metering
 Multiple data access methods including
via REST/HTTP (S3, Hadoop, CAS,
Atmos & Swift)
OBJECT
OBJECT
OBJECT
OBJECT
OBJECT
OBJECT

File and Object Storage Comparison
File Object
Writing to a file requires exclusive lock Object supports multiple writes, no locking
Limit on number of files in a directory. Objects are limitless in size, 1 MB to TBs, Objects scale
across multiple files
File meta-data is fixed by file system, no user
meta-data
Objects support extensible meta-data
Large files hard to seek Objects can be viewed with no limitation
File create operations require directory to be in
exclusivity lock
No locking required to create files
CIFS/NFS access not Web or firewall-friendly –
relies on file/folder access control and session-
based authentication
Easy, fine-grained authentication and access control
(per object),
HTTP, REST-based access

Object Use Cases are Expanding
• Existing Use Cases:
– Scalable content store for cloud-based
applications/services
– Scalable content storage for vertical
applications
– Tape rationalization/elimination
• Emerging Use Cases:
– Storage for Big Data/Hadoop
– NAS replacement/augmentation
– Public IaaS alternative
• Migrate to alternative providers or “in-sourcing”
Applications, analytics and data growth drive Object
Source: 451 Research

MODERN APPS ARE BREAKING TRADITIONAL STORAGE
NOT DESIGNED FOR CLOUDAND BIG DATA APPLICATIONS
• Architecture is too complex
– Locking, replication, High Availability, geo-
distribution is complex
• Not Web or firewall friendly
– Distributed (WAN) access is complex
• Storage silos impede development
– Different hardware for every data type and
access protocol

USE CASES

GLOBAL CONTENT REPOSITORY
ON-PREMISE UNSTRUCTURED STORAGE PLATFORM
PROBLEM
• Can’t cost-effectively manage or scale storage to
support explosive growth in unstructured content.
• Traditional storage not suited for new Web, mobile
and cloud applications.
• Difficult and costly to manage data lifecycle and
retention policies across archive silos and sites
VALUE
• Reduce complexity and cost–one globally
accessible, geo-efficient archive that serves
multiple applications and content types at lower
cost than public cloud.
• Anywhere data access – All data globally
accessible by Web, mobile and cloud apps.
• Enterprise-grade data protection – Efficient
geo-protection and policy-based retention for
basic compliance and governance.
https://accesspoint.yourcompany.com
U.K.L.A.
Memphis
Applications Tiering, Archiving,
Backup

MODERN APPLICATION PLATFORM
EFFICIENT GEO-CAPABLE STORAGE & ANYWHERE ACCESS
U.K.L.A.
Memphis
PROBLEM
• Traditional storage architecture not optimized for
multi-site, mobile access to content
• Writing to multiple file systems and proprietary
APIs complicates development
• Can’t access or process large data sets
VALUE
• Anywhere access - Provides anywhere access to
geo-replicated content
• Simpler, faster development - Supports
multiple industry standard APIs/protocols and
anywhere access with strong consistency
• Unmatched access and efficiency - Geo-
protection, active-active architecture optimizes
both access and storage efficiency for Big Data –
large and small files

GEO-SCALE BIG DATA ANALYTICS
EFFICIENT GEO-SCALE STORAGE & GLOBAL BIG DATA ANALYTICS
U.K.L.A.
New York
ANALYTICS
PROBLEM
• Large (and growing) data volumes lead to
exponential storage costs
• Traditional Hadoop replication leads to
unmanageable DC footprint with data growth
• Always have to move data to the analytics cluster
VALUE
• Cost Efficient Storage
• HDFS Archive – Bring state of the art patented
technology to provide highly dense storage for
Hadoop
• Global Analytics –Bring analytics to geo-
distributed data and archives

PROBLEM
• Unstructured data growth - Reclaim costly Tier 1 storage
• Current solutions aren’t scalable or cost efficient
• Instant access to cold-stored data is required
• “No Public Cloud” policy - Data needs to be on-premises
VALUE
Costs less than public cloud - Provides on-premises security
U.K.L.A.
Memphis
LAN/
WAN
Video
Unstructured
Data
Sensory Data
Images
COLD ARCHIVE
COSTEFFECTIVE LONG TERMRETENTION

PROBLEM
• Need cost effective solution to store hue amounts of
unstructured data generated by IOT and sensors
• “No Public Cloud” policy - Data needs to be on-premises
• Data collection via modern cloud applications requires
compatibility with APIS’s like S3 and OpenStack
• Analytics workflow is slow, expensive and complicated using
Hadoop direct attach or public cloud storage
VALUE
• Cost per GB is less than public clouds
• Provide high availability with on-premises security
• Compatible with S3, OpenStack, and other popular API’s
• HDFS compatible and enables a streamlined Hadoop workflow
for “data in place” analytics
‘IOT’ CLOUD STORAGE PLATFORM
‘INTERNET OF THINGS’ –SENSORY & TELEMETRY DATACOLLECTION

EMC & OBJECT STORAGE

 Hyper Scale - Sales Out to Billions of Objects
 “Public Cloud-Like” – Secure Access, Anytime, Anywhere
 Comprehensive Multi-Tenant Management
 Active/Active Geo-Distributed Architecture
 Multiple Protocol Support – REST and HDFS Ready
 Compelling Economics –Appliance or SW Only (DIY)
ECS HYPERSCALE CLOUD STORAGE

CUSTOMERS CAN LEVERAGE COMMODITY PLATFORMS
SOFTWARE-DEFINED STORAGE
Software-Defined
Storage
Commodity
Platforms

COMMODITY HARDWARE VALUE PROPOSITION
• Utilize standardized, open
technologies and mass market
components
• Individual components provide
lower performance, reliability, etc.
• At sufficient scale, with the right
software, the component pool
provides superior characteristics

ECS
SOFTWARE
Enterprise & SPs
Object & HDFS
DIY Commodity
ECS
APPLIANCE
Enterprise & SPs
Object & HDFS
integrated appliance
ViPR
DATASERVICES
Enterprise & SPs
Object & HDFS
File-based Arrays
CHOICE AND FLEXIBILITY

EMC OBJECT ECOSYSTEM
Enterprise Information
Archiving
Enterprise Content
Management
Analytics
Cloud Gateways
Migration
Sync & Share
Analytics
Protocols CAS

Learn | Try | Develop | Collaborate
Explore how-to
videos, helpful
guides, and
training
Download ViPR
FREE with no
time limit – for
non-production
use
Access SDKs,
FAQs, forums,
technical
documentation,
sample apps,
and more
Ask the
experts, talk to
peers, share
ideas and
experiences
www.emc.com/viprcommunity
JOIN THE ViPR COMMUNITY…

ECS TECHNICAL DETAIL
• ECSSTORAGE ENGINE
• WRITE PATH, READ PATH, BOX CARTING
• ECSGEO-CAPABILITIES

ECS ARCHITECTURE OVERVIEW
Object
Storage Engine
HDFS NFS*
• Multi-head access - ability to access same data concurrently
through multiple access protocols.
• Provides High Availability and Scalability.
• Manages transactions and persistent data.
• Protects data against failures, corruption and disasters
ECS
Appliance
Commodity
EMC and 3rd party
file arrays

COMPREHENSIVE DATA ACCESS
COMPATIBILITY WITH COMMONINDUSTRY API’S
• Simultaneous access to underlying data
through multiple interfaces
– Object, HDFS, File (future)
• HDFS compatible with Cloudera,
Hortonworks, Pivotal etc.
• Support for S3, Swift, Atmos and Centera
CAS APIs object
• Extensions to APIs
– Byte-Range updates, Atomic appends, Rich
ACLs etc.
ATMOS

DESIGN PRINCIPLE: LAYERED ARCHITECTURE
 Limitless Scale: Each layer is
independently scalable, highly
available, and has no single
point of failure.
 Scale-Out Architecture: Scale
by adding more nodes, no
special nodes or roles
 Global Namespace: Any node
has full system view off data and
meta-data
Persistence Layer
Storage Engine
JBODs
OBJECT
HDFS
OBJECT
HDFS
OBJECT
HDFS

TRANSACTION FLOW (WRITE)
Node 1
1. Create Object request
(Name, data, metadata)
Node 1 Node 4 Node 5
2. Write of data and
metadata in chunk
All three copies written in parallel. Write successful only if all copies ack
Node 2
5. Back to client
3. Index update (name,
location) to the owner
partition
4. Journal write

ERASURE CODING
 Data is written into chunks - 3 copies
 Erasure coding begins as the chunks are shipped
 Once EC completes, the data becomes fully
protected and the 3 copies are deleted
A
A
AA

GARBAGE COLLECTION
 In Append-only systems updates/deletes cause files
to have blocks of data that are unused
 This is done at the level of chunk
 Unused chunks reclaimed by a background task

Node 1
1. Read Object request
3. Read data
Node 2
4. Send data back
2. Get Location
TRANSACTION FLOW (READ)

BOX CARTING: CHUNK WRITE
Node
Buffered Writer
Acks
(PARALELL SYNC WRITE)

DATA WRITTEN IN APPEND ONLY CHUNKS
• Data is written in an append-
only pattern.
• No data is overwritten or
modified.
• No locking required for I/O.
• No cache invalidation
required.
• Journaling, snapshot and
versioning natively built-in
• ECS stores all types of data
and index in “chunks”
• Chunks are
– Logical containers of
contiguous space
(128MB)
– Written in an append-
only pattern
• All data protection operations
are done on chunks

ECS STORAGE ENGINE: KEY BENEFITS
• All nodes can process write requests for the same
object simultaneously, and write to different sets of
disks.
• Throughput takes advantage of all spindles and NICs
in cluster.
• Payload from multiple small objects are aggregated
in memory and written in a single disk write
• Efficient storage for both small and large data

 Unstructured configurations
– Object & HDFS
 Available in multiple capacities within
a rack
 Clustering across racks scales to 100s
of PBs
HARDWARE CONFIGURATIONS

ANALYTICS

SAMPLE HADOOP WORKFLOW
HDFS
Analytical Models
(Hive, HAWQ)
Data Visualizations
(Tableau)
Variety
of Data
Sources
Data
Cleansing
Ingest
Data Scientists
Store
Analyze
Surface

HADOOP PROCESSING MODEL
SHARED STORAGE MODEL
VNXVMAX Isilon CommodityVNX Isilon 3rd Party Commodity
• Enables common Data
Lake for LOB application
storage and analytics
• Scale compute & storage
independently
• Multiple distributions,
clusters connect to the
same data
ECS

CHALLENGES WITH HDFS
• HDFS not Enterprise-Grade
– Requires three full copies of data, no erasure
coding
– No Geo-distribution, Limited DR, Multi-tenancy
– Inefficient for handling small files
• High Availability Still In Progress
– No Active-Active Failover even with the
secondary NN
• DAS architecture not suitable for some
customers
– Lack of Enterprise Data Governance Features

HDFS DATA SERVICE OVERVIEW
• Addresses limitations of off-the-
shelf HDFS
• Brings HDFS to existing storage
hardware
• Enables HDFS/Object/File scenarios
• Flexible software model allows for
future co-location of compute and
storage

HDFS DATA SERVICE OVERVIEW
• API head
– Custom client/server protocol optimized for
high scale
– Uses the same unstructured storage engine as
ECS/ViPR Object data service
• Client library over the HDFS API
– Provides a viprfs:// drop-in replacement for
HDFS 2.0
– Can be seamlessly added to existing Hadoop
distributions
• Implemented as a Hadoop Compatible
Filesystem (HCFS)
– Supports HDFS 2.0 and 2.2

HDFS ARCHITECTURE
RM / AsM
Commodity Compute & Storage
Node Manager
Data Store
MapReduce Task
Client
Node Manager
Data Store
MapReduce Task
Node Manager
Data Store
MapReduce Task
NAME
NODE
SEC
NAME
NODE

ViPR/ECS HDFS ARCHITECTURE Client
Node Manager
MapReduce Task
ViPR/ECS
Client
Node Manager
MapReduce Task
ViPR/ECS
Client
Node Manager
MapReduce Task
ViPR/ECS
Client
NAME
NODE
RM/AsM
SEC
NAME
NODE

Customer’s Hadoop
Compute Cluster
HDFS – ECS APPLIANCE DEPLOYMENT
ViPR Controller
VMViPR Controller
VMViPR Controller
VM
Data Read/
Write
Object/HDFS
…
Object/HDFS

HDFS DATA SERVICE/ECS ARCHITECTURE
ECS Storage Engine
HDFS
API
Head
S3
API
Head
Customer’s Hadoop
Compute Cluster
ViPR Data Service Node
Data Read/Write
via ViPR HDFS

HDFS DATA SERVICE VALUE PROPOSTION
• High Availability Built-In, No SPOF
• Avoids multiple copies of data
• Erasure Coding Support
• Geo-Distributed Across Sites
• Multi-tenancy, Metering, Chargeback
• Allows Byte-Range Updates Through
S3 Interface
• ViPR Controller aids in Management &
Monitoring

UPCOMING ECS INTEGRATIONS

ISILON CLOUDPOOLS
SMART TIERING TO OBJECT STORES
Key Features
Benefits
 Stub to Cloud of choice
 Extending SmartPools workflow to
CloudPools
 Ability to send encrypted data to the
cloud
 Compression for efficient transport
 Simple policy based management
 Combine file & object store benefits
 Use stubs to optimize local storage
space, with offsite archive protection
 Seamless placement and availability
of data per policy
 One Accessible namespace
SmartPools ->
CloudPools
Clients
SMB | NFS | REST| HDFS | SWIFT
OneFS
Service
Provider
Public
Cloud
ECS

EMC CLOUDBOOST
LONG TERM RETENTION TO THE CLOUD
LAN LAN/WAN
CloudBoost
appliance
Desktops
Laptops
Files NAS/NDMP
VMware &
Hyper-V
Databases
Email Applications
DB
ROBO
Protected by NetWorker,
Avamar, NetBackup
Key Features
Benefits
 Long-term retention to ECS for
NetWorker, Avamar, NetBackup
 Inline variable de-duplication and
compression
 Data encrypted in-flight and at rest
 Cloud choice: private/public clouds
 Appliance cache for ROBO
 Capacity of up to 6PB logical per
appliance
 Central management via a cloud
portal
 Lower storage cost per TB
 Efficiency: Reduced network and
storage consumption
 Lower risk, operational overhead
than tape.
 Airtight security

WRITE PATH, READ PATH,BOX CARTING

TRANSACTION FLOW (WRITE)
Node 1
1. Create Object request
(Name, data, metadata)
Node 1 Node 4 Node 5
2. Write of data and
metadata in chunk.
All three copies written in parallel. Write successful only if all copies ack.
Node 2
5. Back to
client
3. Index update
(name, location) to
the owner partition.
4. Journal write

ERASURE CODING
 Data is written into chunks 3 copies
 Once a chunk fills to 128 MB, erasure coding starts
 Once it is completed and data is protected the 3
copies are deleted.
A
A
AA

GARBAGE COLLECTION
 In Append-only systems updates/deletes cause files
to have blocks of data that are unused.
 This is done at the level of chunk
 Unused chunks reclaimed by a background task

Node 1
1. Read Object request
3. Read data
Node 2
4. Send data
back
2. Get Location
TRANSACTION FLOW (READ)

BOX CARTING: CHUNK WRITE
Node
Buffered Writer
Acks
(PARALELL SYNC WRITE)

• ECSGEO-STORAGE OVERVIEW
• DATAPROTECTION
• GLOBAL DATAACCESS
ECS GEO-CAPABILITES

ECS GEO-STORAGE OVERVIEW
• Data Protection
– Protection against data center failure
– Seamless failover and recovery
• Global Data Access
– Global namespace
– Ability to read/write data from any site
• Optimized Storage
– Low storage overhead
– WAN Optimization
• Applicable to all unstructured Storage Engine based
heads
– Object, HDFS
– File when available

DATA PROTECTION

INDUSTRY SOLUTION: MIRROR COPY
• Mirrored copy in a backup site
• Benefit: Achieves Local
reconstruction on hardware failure
• Shortcoming: Storage overhead
-> 2.66xPrimary Secondary

INDUSTRY SOLUTION: DISTRIBUTED ERASURE
CODING
• Distributing fragments across
sites
• Benefit: Achieves low Storage
Overhead ~ 1.6x
• Shortcoming: Disk/Node failure
requires fragments to be fetched
over the WAN.
Site 1 Site 2
Site 3 Site 4

ECS MODEL: BEST OF BOTH WORLDS
• Achieves low Storage Overhead
~ 1.8x
• Local hardware failure recovery
requires no WAN traffic.
• Handles local hardware and full
data center failures
– Disk, Node, Rack, Data Center are
failure domains
Site 1 Site 2
Site 3 Site 4

GLOBAL DATA ACCESS

INDUSTRY SOLUTION: SEGREGATED NAMESPACE
• Customers are asked to pick a
location for each bucket.
• Shortcoming: sites are vertical
silos, unaware of each other’s
namespaces.
Site 1 Site 2
app app
Bucket BBucket A

INDUSTRY SOLUTION: MULTI-ACCESS WITH
EVENTUAL CONSISTENCY
• Global Namespace with read only
replicas.
• Replicas have eventual consistency
• Shortcomings: Difficult to write
applications against eventual
consistency models
Site 1 Site 2
app
Bucket A
app
read only

ECS: MULTI-ACCESS WITH STRONG
CONSISTENCY
• Global Namespace: buckets
stretches across sites
• Global Access: Any data can be
read and written to any site
• Strongly consistent: Always
returning latest version without
requiring synchronous write.
Site 1 Site 2
Bucket A
app app app

OPTIMIZED STORAGE
• Low Storage Overhead: ~1.8x
replication over head across 4 sites
• WAN Optimization: All node and
disk failures are repaired within the
site, without any WAN traffic.

STORAGE OVERHEAD
# of Data Centers Overhead
1 1.33 x
2 2.67 x
3 2.00 x
4 1.77 x
5 1.67 x
6 1.60 x
7 1.55 x
8 1.52 x

ECS GEO KEY DIFFERENTIATORS
• Tolerates one site disaster along
with up to 2 node failures in all the
rest of the sites.
• Component failures are recovered
using fragments from local site
without WAN traffic
• Geo-efficient (~1.8 copies across 4
sites) without WAN read/write
penalties

ECS APPLIANCE

 Not Contradictory!
 Components are Commodity
 x86 Servers
 Ethernet Networking
 SATA Disk Drives
 Innovation in how they’re put
together to enable reliability,
availability, and serviceability!
COMMODITY INNOVATION

ECS APPLIANCE CHARACTERISTICS
• Use COTS Components
– Economies of scale
• Density Optimized
– Up to 72TB Raw / Rack Unit
– Saves Power/GB, Real Estate costs, etc.
• Labor Optimized
– Manage the cluster, not the devices
– Maximize Serviceability
• Protection Efficiency
– Geo-efficient storage

ECS/Cloud Object Storage - DevOps Day

ECS/Cloud Object Storage - DevOps Day

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie ECS/Cloud Object Storage - DevOps Day

Ähnlich wie ECS/Cloud Object Storage - DevOps Day (20)

Mehr von Bob Sokol

Mehr von Bob Sokol (13)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

ECS/Cloud Object Storage - DevOps Day

Hinweis der Redaktion