Solution architecture for big data projects solution architecture,big data,hadoop,hive,hbase,impala,spark,apache,cassandra,SAP HANA,Cognos big insights

1. Solution Architecture for bigdata
Business Architecture
Information Architecture
Infrastructure Architecture
Data Architecture
Integration Architecture
Service Architecture

2. Solution Architecture
BI Solution
Requirements
Corporate
Strategy
BI Stakeholders
BI Vision
BI Strategy
BI Mission
Statement
BI Solution
Architecture
Business
Architecture
Infrastructure
Architecture
Data
Architecture
Service
Architecture
Integration
Architecture
Information
Architecture

3. Business Architecture
Business Architecture
Business Objective
Business Strategy
Business Capabilities
Business Process Areas
Ways of Working
Interviews
Requirement Workshop
Business Structure

4. Information Architecture
Information
Architecture
Social Media:Facebook
data from API
Legacy System
Social Media Data:
Twitter
Data Capture System
Web Application data
KANBAN Process
ERP (Manufacturing)
Landing Space
Staging Tables
ODC
Denormalized
Column
Families
Row KeyERP 3NF Tables
Reference Data
Data Ownership Data Contracts
Structural Design Shared Info Env. Flow & Lineage Meaning and Use

5. Application Architecture
Application
Architecture
noSQL databases: HBASE, Cassandra, MongoDB
Relational databases (Source /Target):
Oracle,SQL Server,Terradata
Graph databases: Neo4j and Giraph
Hadoop Technologies: HDFS
In memory/MPP/Search:
Shark,Spark,Impala,SAP HANA,
Visualisation/Reporting:
Tableau,Pentaho,qlickview
Data Integration:
(Talend/Informatica/BODI,Pentaho DI)
How All Application Hook Together
Data Warehouse:
(Hive, Relational Datawarehouse,SAP HANA)

6. Infrastructure Architecture
Infrastructure
Architecture
Storage
Communication
Data Federation
Data Virtualization
Cloud
Security
HA/DR
Recovery/Backup
Licensing Models
Versioning / Patching
Hardware/ VM
Procurement
Sizing
Production
Test
Development

7. Data Architecture
Data
Architecture
ETL
Sources
Data Governance
Metadata
Data Quality
Data Profiling
MDM
Big data
Near Real time
Real time
ODS
Reference Data
Physical /logical Data model
Dimensional Modelling
Entity Modelling
Data Warehouse

8. Data Integration Architecture
Data Integration
Architecture
ETL
ETL Subsystem
Sources
Data Governance
Metadata
Data Quality
Data Profiling
MDM
Big data
Near Real time
Real time
ODS
Reference Data
Physical /logical Data model
Dimensional Modelling
Entity Modelling
Data Warehouse
Details Next Slide

9. ETL Subsystems
Profiling
Extrac
tion
Data
Change
Mgmt.
Error Event
Schema
Audit
Dimension
Deduplication
Data
Cleansing
Surrogate
Generation
SCD Mgmt.
Fact
Update
Dimension &
Fact
Conformity
Hierarchy
Mgmt.
Surrogate
Mgmt.
Bridge
Table
Sorting
Job
Scheduler
Lineage
Dependencies
Parallel
Processing
Bulk Load
TransformExtract Load
Recovery
& Restart
Security
Systems
Package
Versioning
Compliance
Metadata
Repository
Recovery
& Restart
Problem
Escalation
Package
Lifecycle
Workflow
Monitoring
Backup
Systems
Manage

10. Integration Architecture
Integration
Architecture
Security (Cloud Security, Data security, Top 10)
Portal Integration
Coarse grain Integration (Web Services)
Social Media API
Distributed Hadoop Java Customization
Infrastructure AWS API publication-consumption
Service Oriented Architecture
Big data Analysis Java Components
Data Analysis javascript Libraries
Real time data feed API (node.js)
3rd party Visualisation API like Adobe Flex
Analytics API
Integration Governance Framework
Other 3rd party system Integration API
Process Modelling: BPM/BPEL components
Supporting Platforms Integration

11. Service Architecture
Service
Architecture
Compliance
Backup and Recovery
Disaster Recovery and High Availability
Service Library
Defects/ Fixes/Impact Management
Upgrades/Maintenance
Service Operations
Change Requests CR, Request For Change RFC
Release Policy
Service Transition
Service Design Package
Capacity Plan
Service Level Agreement
Service Design
Recovery Time Objective/ Recovery Point Objective
Service Strategy

12. More details
• http://thebigdatatrends.wordpress.com
• http://enterprisearchitectview.wordpress.com
• http://thedatascience.wordpress.com/
• http://thecloudview.wordpress.com/
• http://businessarchitectview.wordpress.com/
• https://sandyclassic.wordpress.com