SlideShare ist ein Scribd-Unternehmen logo
1 von 11
Vimeo on EC2,[object Object],1,[object Object]
Vimeo?,[object Object],2,[object Object]
Why AWS?,[object Object],3,[object Object]
The Workflow,[object Object],4,[object Object]
On-demand, Reserved,[object Object],5,[object Object]
Upload/Transcoding Infrastructure,[object Object],6,[object Object]
Our Bidding Strategy,[object Object],Keep it simple,[object Object],One time spot requests; one instance per request;across all availability zones,[object Object],Spot requests expire in 10 minutes,[object Object],Never bid more than threshold. It is currently set to 80% of on-demand price,[object Object],Not more than 10 open spot requests at any time,[object Object],Bid 10% more than the average price over last hour ,[object Object],Never bid more than threshold. It is currently set to 80% of on-demand price,[object Object],7,[object Object]
Reserved Instance Utilization,[object Object],8,[object Object],on-demand,[object Object],% busy,[object Object],• 96.23% utilization with 54 instances on low-traffic day,[object Object],• High priority jobs: Buy reserve instance capacity to meet non-peak hour loads,[object Object]
Ondemandvs Spots,[object Object],9,[object Object],Happy Place,[object Object],,[object Object],#Ondemand,[object Object],#On-demand,[object Object],#spots,[object Object],#Spots,[object Object]
Pro Tips,[object Object],Use Spots for your low priority and less time critical jobs,[object Object],Never kill spots. Let Amazon do it.,[object Object],Have more retries for jobs running on spots.,[object Object],Watch out for open spot requests. Add expiry to your requests.,[object Object],Long running jobs, bid higher or use on-demand,[object Object],Failover to on-demand when spot market is saturated,[object Object],10,[object Object]
Result,[object Object],Some months we save 50% over pure on-demand instances,[object Object],11,[object Object]

Weitere ähnliche Inhalte

Was ist angesagt?

Webinar: AWS Partner Strategies For Success
Webinar: AWS Partner Strategies For SuccessWebinar: AWS Partner Strategies For Success
Webinar: AWS Partner Strategies For SuccessAaron Klein
 
Botmetric- Automate your AWS Cloud
Botmetric- Automate your AWS Cloud Botmetric- Automate your AWS Cloud
Botmetric- Automate your AWS Cloud Nutanix Beam
 
Handling customer fears
Handling customer fearsHandling customer fears
Handling customer fearsAshraf Osman
 
BeyondAutomation
BeyondAutomationBeyondAutomation
BeyondAutomationSean Zhang
 
ERPNext Open Day - August 2015
ERPNext Open Day - August 2015ERPNext Open Day - August 2015
ERPNext Open Day - August 2015Anand Doshi
 
Kanban for Game Studios
Kanban for Game Studios Kanban for Game Studios
Kanban for Game Studios Perforce
 
Facilitating the Elephant carpaccio exercise
Facilitating the Elephant carpaccio exerciseFacilitating the Elephant carpaccio exercise
Facilitating the Elephant carpaccio exercisePeter Antman
 
AWS Well Architected Framework
AWS Well Architected FrameworkAWS Well Architected Framework
AWS Well Architected FrameworkJohn McCormack
 
Queueing at the Checkout
Queueing at the CheckoutQueueing at the Checkout
Queueing at the CheckoutWilliam Tracz
 
AWS account migration for BBC iPlayer Radio
AWS account migration for BBC iPlayer RadioAWS account migration for BBC iPlayer Radio
AWS account migration for BBC iPlayer RadioSimone Spaccarotella
 
International software testing conference 2017 fergal hynes
International software testing conference 2017 fergal hynesInternational software testing conference 2017 fergal hynes
International software testing conference 2017 fergal hynesFergal Hynes
 
Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...
Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...
Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...National Retail Federation
 

Was ist angesagt? (15)

Webinar: AWS Partner Strategies For Success
Webinar: AWS Partner Strategies For SuccessWebinar: AWS Partner Strategies For Success
Webinar: AWS Partner Strategies For Success
 
Botmetric- Automate your AWS Cloud
Botmetric- Automate your AWS Cloud Botmetric- Automate your AWS Cloud
Botmetric- Automate your AWS Cloud
 
Handling customer fears
Handling customer fearsHandling customer fears
Handling customer fears
 
BeyondAutomation
BeyondAutomationBeyondAutomation
BeyondAutomation
 
Datacomm VMWare Hybrid Cloud
Datacomm VMWare Hybrid CloudDatacomm VMWare Hybrid Cloud
Datacomm VMWare Hybrid Cloud
 
ERPNext Open Day - August 2015
ERPNext Open Day - August 2015ERPNext Open Day - August 2015
ERPNext Open Day - August 2015
 
Kanban for Game Studios
Kanban for Game Studios Kanban for Game Studios
Kanban for Game Studios
 
Facilitating the Elephant carpaccio exercise
Facilitating the Elephant carpaccio exerciseFacilitating the Elephant carpaccio exercise
Facilitating the Elephant carpaccio exercise
 
AWS Well Architected Framework
AWS Well Architected FrameworkAWS Well Architected Framework
AWS Well Architected Framework
 
Queueing at the Checkout
Queueing at the CheckoutQueueing at the Checkout
Queueing at the Checkout
 
Paving The Way To The Hybrid Cloud
Paving The Way To The Hybrid CloudPaving The Way To The Hybrid Cloud
Paving The Way To The Hybrid Cloud
 
AWS account migration for BBC iPlayer Radio
AWS account migration for BBC iPlayer RadioAWS account migration for BBC iPlayer Radio
AWS account migration for BBC iPlayer Radio
 
International software testing conference 2017 fergal hynes
International software testing conference 2017 fergal hynesInternational software testing conference 2017 fergal hynes
International software testing conference 2017 fergal hynes
 
Effect of Batch Size on Work Flow
Effect of Batch Size on Work FlowEffect of Batch Size on Work Flow
Effect of Batch Size on Work Flow
 
Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...
Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...
Shop.org 2017 Tech talk website speed for ecommerce why it matters and how to...
 

Ähnlich wie Vimeo EC2 transcoding

SRV301 Getting the most Bang for your buck with #EC2 #Winning
SRV301 Getting the most Bang for your buck with #EC2 #WinningSRV301 Getting the most Bang for your buck with #EC2 #Winning
SRV301 Getting the most Bang for your buck with #EC2 #WinningAmazon Web Services
 
SRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #WinningSRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #WinningAmazon Web Services
 
SRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #WinningSRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #WinningAmazon Web Services
 
Get the Most Bang for your Buck with #EC2 #Winning
Get the Most Bang for your Buck with #EC2 #WinningGet the Most Bang for your Buck with #EC2 #Winning
Get the Most Bang for your Buck with #EC2 #WinningAmazon Web Services
 
Cloudreach Voices EC2 Making Sense of the Cost Options
Cloudreach Voices EC2 Making Sense of the Cost Options  Cloudreach Voices EC2 Making Sense of the Cost Options
Cloudreach Voices EC2 Making Sense of the Cost Options Cloudreach
 
AWS Cloud Kata | Bangkok - Getting to Profitability
AWS Cloud Kata | Bangkok - Getting to ProfitabilityAWS Cloud Kata | Bangkok - Getting to Profitability
AWS Cloud Kata | Bangkok - Getting to ProfitabilityAmazon Web Services
 
Cut AWS Costs: Using Spot Instances for More Than Batch
Cut AWS Costs: Using Spot Instances for More Than BatchCut AWS Costs: Using Spot Instances for More Than Batch
Cut AWS Costs: Using Spot Instances for More Than BatchRightScale
 
Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...
Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...
Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...Amazon Web Services
 
5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)
5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)
5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)parkmycloud
 
AWS Meetup - Exploring ways to buy EC2 capacity
AWS Meetup - Exploring ways to buy EC2 capacityAWS Meetup - Exploring ways to buy EC2 capacity
AWS Meetup - Exploring ways to buy EC2 capacityAntti Siiskonen
 
Getting the most Bang for your Buck with #EC2 #Winning
Getting the most Bang for your Buck with #EC2 #WinningGetting the most Bang for your Buck with #EC2 #Winning
Getting the most Bang for your Buck with #EC2 #WinningAmazon Web Services
 
Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017
Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017
Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017Amazon Web Services
 
Hidden Costs of Chasing the Mythical 'Five Nines'
Hidden Costs of Chasing the Mythical 'Five Nines'Hidden Costs of Chasing the Mythical 'Five Nines'
Hidden Costs of Chasing the Mythical 'Five Nines'DevOpsDays DFW
 
Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...
Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...
Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...Amazon Web Services
 
Running Lean Architectures: How to Optimize for Cost Efficiency
Running Lean Architectures: How to Optimize for Cost Efficiency Running Lean Architectures: How to Optimize for Cost Efficiency
Running Lean Architectures: How to Optimize for Cost Efficiency Amazon Web Services
 
AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...
AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...
AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...Amazon Web Services
 
AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...
AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...
AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...Amazon Web Services
 
Getting Started with EC2 Spot - November 2016 Webinar Series
Getting Started with EC2 Spot - November 2016 Webinar SeriesGetting Started with EC2 Spot - November 2016 Webinar Series
Getting Started with EC2 Spot - November 2016 Webinar SeriesAmazon Web Services
 
Optimizing Costs and Efficiency of AWS Services
Optimizing Costs and Efficiency of AWS ServicesOptimizing Costs and Efficiency of AWS Services
Optimizing Costs and Efficiency of AWS ServicesAmazon Web Services
 

Ähnlich wie Vimeo EC2 transcoding (20)

SRV301 Getting the most Bang for your buck with #EC2 #Winning
SRV301 Getting the most Bang for your buck with #EC2 #WinningSRV301 Getting the most Bang for your buck with #EC2 #Winning
SRV301 Getting the most Bang for your buck with #EC2 #Winning
 
SRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #WinningSRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #Winning
 
SRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #WinningSRV301 Getting the Most Bang for your Buck with #EC2 #Winning
SRV301 Getting the Most Bang for your Buck with #EC2 #Winning
 
Get the Most Bang for your Buck with #EC2 #Winning
Get the Most Bang for your Buck with #EC2 #WinningGet the Most Bang for your Buck with #EC2 #Winning
Get the Most Bang for your Buck with #EC2 #Winning
 
Cloudreach Voices EC2 Making Sense of the Cost Options
Cloudreach Voices EC2 Making Sense of the Cost Options  Cloudreach Voices EC2 Making Sense of the Cost Options
Cloudreach Voices EC2 Making Sense of the Cost Options
 
AWS Cloud Kata | Bangkok - Getting to Profitability
AWS Cloud Kata | Bangkok - Getting to ProfitabilityAWS Cloud Kata | Bangkok - Getting to Profitability
AWS Cloud Kata | Bangkok - Getting to Profitability
 
Cut AWS Costs: Using Spot Instances for More Than Batch
Cut AWS Costs: Using Spot Instances for More Than BatchCut AWS Costs: Using Spot Instances for More Than Batch
Cut AWS Costs: Using Spot Instances for More Than Batch
 
Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...
Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...
Dollars and Sense Technical Tips for Continual Cost Optimization AWS-Partner-...
 
5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)
5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)
5 Ways to Control your AWS Spending (or, How to Make Your CFO Happy)
 
AWS Meetup - Exploring ways to buy EC2 capacity
AWS Meetup - Exploring ways to buy EC2 capacityAWS Meetup - Exploring ways to buy EC2 capacity
AWS Meetup - Exploring ways to buy EC2 capacity
 
Getting the most Bang for your Buck with #EC2 #Winning
Getting the most Bang for your Buck with #EC2 #WinningGetting the most Bang for your Buck with #EC2 #Winning
Getting the most Bang for your Buck with #EC2 #Winning
 
Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017
Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017
Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017
 
Hidden Costs of Chasing the Mythical 'Five Nines'
Hidden Costs of Chasing the Mythical 'Five Nines'Hidden Costs of Chasing the Mythical 'Five Nines'
Hidden Costs of Chasing the Mythical 'Five Nines'
 
Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...
Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...
Reducing Cost & Maximizing Efficiency: Tightening the Belt on AWS (CPN211) | ...
 
Running Lean Architectures: How to Optimize for Cost Efficiency
Running Lean Architectures: How to Optimize for Cost Efficiency Running Lean Architectures: How to Optimize for Cost Efficiency
Running Lean Architectures: How to Optimize for Cost Efficiency
 
AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...
AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...
AWS Summit 2013 | Auckland - Optimizing Your AWS Applications and Usage to Re...
 
AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...
AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...
AWS re:Invent 2016: Dollars and Sense: Technical Tips for Continual Cost Opti...
 
Cost Optimisation on AWS
Cost Optimisation on AWSCost Optimisation on AWS
Cost Optimisation on AWS
 
Getting Started with EC2 Spot - November 2016 Webinar Series
Getting Started with EC2 Spot - November 2016 Webinar SeriesGetting Started with EC2 Spot - November 2016 Webinar Series
Getting Started with EC2 Spot - November 2016 Webinar Series
 
Optimizing Costs and Efficiency of AWS Services
Optimizing Costs and Efficiency of AWS ServicesOptimizing Costs and Efficiency of AWS Services
Optimizing Costs and Efficiency of AWS Services
 

Kürzlich hochgeladen

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 

Kürzlich hochgeladen (20)

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 

Vimeo EC2 transcoding

  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.

Hinweis der Redaktion

  1. High quality video, first to do HD, work hard on supporting every format and getting the most out of the video.Video player, close second with HTML5 player, we want videos on vimeo to be playable everywhere. iOS, desktop, TV.Friendliest and most supportive community. Lots of positive people who like making videos.Good tools for sharing and privacy. You don’t have to share your videos with the whole world.Two types of users – free and plus.
  2. It was thereManaged hosting, expensive storage in 2007. We moved to source file storage to S3 because we could do it really easily. No contracts. It was cheaper than what we had. It had more features than what we had. We needed something now with low commitment.Our first auto scaling EC2 transcoders went up in 2008. Our encoding load at peak was 3-4x higher than non-peak. Our users don't care whether it's a peak time or not, and we prefer not to pay for transcoding machines the five days a week we aren't using them. This workload is perfect for EC2.Since then our workload has normalized a bit, but that was the situation at the time.
  3. Upload machine (Long connections, doesn't scale well, high IO)S3Transcoding machine (Jobs take minutes to transcode, open source toolkit, as tightly coupled as can be -- controlled from our datacenter via ssh. Works great if you are small. Thousands of ssh connections though... don't recommend this)
  4. We have had a pretty good auto scaling system in place since April 2009. We constantly improve it. We're refreshing it now to make it stateless.At peak few hundred c1.xlarges, dozens of m1.large for uploads. We're experimenting with GPU and cluster compute instances.We buy reserved instances to bring our costs down. In the past we've bought enough to keep them at 100% utilization. If our lowest utilization on a weekday was 50 instances, we bought that number.We've also buy them to guarantee capacity for our plus members. Our plus members shouldn't wait, even when Amazon is low on on-demand instances.The availability guarantee is important. We have had trouble getting capacity at times, for as long as a few days.Now we are buying them to get to 75% utilization. You save money if you use 55% or more.
  5. Don't manage spots manually:) Amazon has awesome api support. Leverage on it. We found python boto to be very stable and easy to use. We likeaws-lib for Node.js. We use it for SQS."Thinking of spots for your web servers and database machines. DON'T DO IT”
  6. We Keep it simple: Onetime spot request with one instance per request with an expiry of 10 minutes across all availability zonesWe get average price over the last 1hr and Bid 10% more than that.  It is little more complex than that.Watch how many spots you are launching. We have no more than 10 spot requests "open" at any time.We never exceed our bids more than 80% of the on demand price. 80 came from 80-20 rule. When in doubt pick 80. let us just say we just keep tweaking threshold and currently it is set to 80%
  7. However Amazon recommends to bid at the  max price you are comfortable with. We don't do that primarily because we never kill spots.. That is not completely true. When we need to scale down, we terminate spots only when we have no more on demand instances to kill.Why we never have to kill spots? We carefully estimate how many machines we need to keep running all the time and buy reserve instance capacity to meet that demand.
  8. Spots can be saturated during peak hours.  Deal with it.
  9. We terminate machines that have no jobs running on them or have least impact. Also we terminate machines 5 minutes before start of next billing hour.jobs running on spots are retried twice as many times the jobs running on reserve/on demand. If a job failed far too many times and/or is delayed beyond acceptable wait time, it gets to run on an on demand machine.Use spots for your low priority and less time critical jobs.If you have long running jobs bid higher or just use on demand instances.Understand your workload and tweak the spot algorithm to suit your needs.. the single most important thing you can take from this talk.