Submit Search
Upload
San Antonio DevOps: Fluentd
•
1 like
•
692 views
Hart Hoover
Follow
An overview of fluentd for the San Antonio DevOps Meetup
Read less
Read more
Technology
Report
Share
Report
Share
1 of 25
Download now
Download to read offline
Recommended
Gemless
Gemless
Brittany Martin
Android webservices
Android webservices
Krazy Koder
Dicas e truques de otimização de websites python
Dicas e truques de otimização de websites python
Fabiano Weimar
jSession #4 - Maciej Puchalski - Zaawansowany retrofit
jSession #4 - Maciej Puchalski - Zaawansowany retrofit
jSession
Solving The Open Stack Talent Gap
Solving The Open Stack Talent Gap
Rackspace
Butter Web Browsing with Margarine
Butter Web Browsing with Margarine
Wayne Walls
Enterprise Cloud Forum Rackspace IT: Journey to the Cloud
Enterprise Cloud Forum Rackspace IT: Journey to the Cloud
Rackspace
RMS Security Breakfast
RMS Security Breakfast
Rackspace
Recommended
Gemless
Gemless
Brittany Martin
Android webservices
Android webservices
Krazy Koder
Dicas e truques de otimização de websites python
Dicas e truques de otimização de websites python
Fabiano Weimar
jSession #4 - Maciej Puchalski - Zaawansowany retrofit
jSession #4 - Maciej Puchalski - Zaawansowany retrofit
jSession
Solving The Open Stack Talent Gap
Solving The Open Stack Talent Gap
Rackspace
Butter Web Browsing with Margarine
Butter Web Browsing with Margarine
Wayne Walls
Enterprise Cloud Forum Rackspace IT: Journey to the Cloud
Enterprise Cloud Forum Rackspace IT: Journey to the Cloud
Rackspace
RMS Security Breakfast
RMS Security Breakfast
Rackspace
1 encu-escritra-clau-correcion
1 encu-escritra-clau-correcion
Diego Solano
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Rackspace
Enterprise Open Cloud Forum: The Cloud is Making it Rain
Enterprise Open Cloud Forum: The Cloud is Making it Rain
Rackspace
Straight Up Startup…Featuring Eli Rosenberg, COO & Co-Founder of Food Genius
Straight Up Startup…Featuring Eli Rosenberg, COO & Co-Founder of Food Genius
Rackspace
Scaling Xen Within Rackspace Cloud Servers
Scaling Xen Within Rackspace Cloud Servers
Rackspace
Curing the 'Migration Migraine' with SharePoint Hosting
Curing the 'Migration Migraine' with SharePoint Hosting
Rackspace
Jason Pannell Resume
Jason Pannell Resume
Jason Pannell
Rackspace Analytical Compute Grid (ACG)
Rackspace Analytical Compute Grid (ACG)
Rackspace
Rackspace::Solve NYC - The Future of Applications with Ken Cochrane, Engineer...
Rackspace::Solve NYC - The Future of Applications with Ken Cochrane, Engineer...
Rackspace
CI/CD with Kubernetes
CI/CD with Kubernetes
Hart Hoover
CI/CD with Docker on AWS
CI/CD with Docker on AWS
Hart Hoover
Ignite Talk: "Building a DevOps Community" / DevOps Days Austin 2015
Ignite Talk: "Building a DevOps Community" / DevOps Days Austin 2015
Hart Hoover
San Antonio Chef Users Meetup, Jun 2014 - Chef Metal
San Antonio Chef Users Meetup, Jun 2014 - Chef Metal
Hart Hoover
Making TDD [Somewhat] Bearable on OpenStack
Making TDD [Somewhat] Bearable on OpenStack
Hart Hoover
Deploying a Chef Server
Deploying a Chef Server
Hart Hoover
Rapid Chef Development with Berkshelf, Test-Kitchen and Foodcritic
Rapid Chef Development with Berkshelf, Test-Kitchen and Foodcritic
Hart Hoover
Vagrant Intro
Vagrant Intro
Hart Hoover
"Unlocked: The Hybrid Cloud" Business Track
"Unlocked: The Hybrid Cloud" Business Track
Hart Hoover
Isolated Networks in the Cloud
Isolated Networks in the Cloud
Hart Hoover
Why Open Matters
Why Open Matters
Hart Hoover
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
Lonnie McRorey
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
Hervé Boutemy
More Related Content
Viewers also liked
1 encu-escritra-clau-correcion
1 encu-escritra-clau-correcion
Diego Solano
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Rackspace
Enterprise Open Cloud Forum: The Cloud is Making it Rain
Enterprise Open Cloud Forum: The Cloud is Making it Rain
Rackspace
Straight Up Startup…Featuring Eli Rosenberg, COO & Co-Founder of Food Genius
Straight Up Startup…Featuring Eli Rosenberg, COO & Co-Founder of Food Genius
Rackspace
Scaling Xen Within Rackspace Cloud Servers
Scaling Xen Within Rackspace Cloud Servers
Rackspace
Curing the 'Migration Migraine' with SharePoint Hosting
Curing the 'Migration Migraine' with SharePoint Hosting
Rackspace
Jason Pannell Resume
Jason Pannell Resume
Jason Pannell
Rackspace Analytical Compute Grid (ACG)
Rackspace Analytical Compute Grid (ACG)
Rackspace
Rackspace::Solve NYC - The Future of Applications with Ken Cochrane, Engineer...
Rackspace::Solve NYC - The Future of Applications with Ken Cochrane, Engineer...
Rackspace
Viewers also liked
(9)
1 encu-escritra-clau-correcion
1 encu-escritra-clau-correcion
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Enterprise Open Cloud Forum: The Cloud is Making it Rain
Enterprise Open Cloud Forum: The Cloud is Making it Rain
Straight Up Startup…Featuring Eli Rosenberg, COO & Co-Founder of Food Genius
Straight Up Startup…Featuring Eli Rosenberg, COO & Co-Founder of Food Genius
Scaling Xen Within Rackspace Cloud Servers
Scaling Xen Within Rackspace Cloud Servers
Curing the 'Migration Migraine' with SharePoint Hosting
Curing the 'Migration Migraine' with SharePoint Hosting
Jason Pannell Resume
Jason Pannell Resume
Rackspace Analytical Compute Grid (ACG)
Rackspace Analytical Compute Grid (ACG)
Rackspace::Solve NYC - The Future of Applications with Ken Cochrane, Engineer...
Rackspace::Solve NYC - The Future of Applications with Ken Cochrane, Engineer...
More from Hart Hoover
CI/CD with Kubernetes
CI/CD with Kubernetes
Hart Hoover
CI/CD with Docker on AWS
CI/CD with Docker on AWS
Hart Hoover
Ignite Talk: "Building a DevOps Community" / DevOps Days Austin 2015
Ignite Talk: "Building a DevOps Community" / DevOps Days Austin 2015
Hart Hoover
San Antonio Chef Users Meetup, Jun 2014 - Chef Metal
San Antonio Chef Users Meetup, Jun 2014 - Chef Metal
Hart Hoover
Making TDD [Somewhat] Bearable on OpenStack
Making TDD [Somewhat] Bearable on OpenStack
Hart Hoover
Deploying a Chef Server
Deploying a Chef Server
Hart Hoover
Rapid Chef Development with Berkshelf, Test-Kitchen and Foodcritic
Rapid Chef Development with Berkshelf, Test-Kitchen and Foodcritic
Hart Hoover
Vagrant Intro
Vagrant Intro
Hart Hoover
"Unlocked: The Hybrid Cloud" Business Track
"Unlocked: The Hybrid Cloud" Business Track
Hart Hoover
Isolated Networks in the Cloud
Isolated Networks in the Cloud
Hart Hoover
Why Open Matters
Why Open Matters
Hart Hoover
More from Hart Hoover
(11)
CI/CD with Kubernetes
CI/CD with Kubernetes
CI/CD with Docker on AWS
CI/CD with Docker on AWS
Ignite Talk: "Building a DevOps Community" / DevOps Days Austin 2015
Ignite Talk: "Building a DevOps Community" / DevOps Days Austin 2015
San Antonio Chef Users Meetup, Jun 2014 - Chef Metal
San Antonio Chef Users Meetup, Jun 2014 - Chef Metal
Making TDD [Somewhat] Bearable on OpenStack
Making TDD [Somewhat] Bearable on OpenStack
Deploying a Chef Server
Deploying a Chef Server
Rapid Chef Development with Berkshelf, Test-Kitchen and Foodcritic
Rapid Chef Development with Berkshelf, Test-Kitchen and Foodcritic
Vagrant Intro
Vagrant Intro
"Unlocked: The Hybrid Cloud" Business Track
"Unlocked: The Hybrid Cloud" Business Track
Isolated Networks in the Cloud
Isolated Networks in the Cloud
Why Open Matters
Why Open Matters
Recently uploaded
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
Lonnie McRorey
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
Hervé Boutemy
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
Scott Keck-Warren
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
BookNet Canada
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
RankYa
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
Florian Wilhelm
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
Pixlogix Infotech
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Sergiu Bodiu
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
Manik S Magar
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
null - The Open Security Community
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
Stephanie Beckett
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
Kalema Edgar
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
Alan Dix
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Precisely
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
Sri Ambati
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
LoriGlavin3
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
charlottematthew16
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
Rizwan Syed
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
Fwdays
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Mark Billinghurst
Recently uploaded
(20)
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
San Antonio DevOps: Fluentd
1.
Hart Hoover |
@hhoover Josh O’Brien | @OBrienCommaJosh @SADevOps
2.
fluentd
3.
Log all the
things to all the things!
4.
…in JSON!
5.
6.
7.
8.
input&{& &&file&{& &&&&path&=>&"/var/log/http.log"& &&}& }& filter&{& &&grok&{& &&&&match&=>&[&"message",&"%{IP:client}&% {WORD:method}&%{URIPATHPARAM:request}&% {NUMBER:bytes}&%{NUMBER:duration}"&]& &&}& }
9.
10.
11.
12.
Fluentd Events Time Tag Record passed
from source or adding a parsed time message routing in fluentd JSON
13.
#&receive&events&via&HTTP&& <source>&& type&http&& port&8888&& </source>
14.
#&read&logs&from&a&file&& <source>&& type&tail path&/var/log/httpd.log& format&apache tag&apache.access&& </source>
15.
#&DOCKER&OMG& <source>& &&type&tail& &&format&json& &&path&/var/lib/docker/containers/ID/IDYjson.log& &&pos_file&/var/lib/docker/containers/ID/IDY json.log.pos& &&tag&docker.container& &&rotate_wait&5& &&read_from_head&true& </source>
16.
#&save&alerts&to&a&file&& <match&alert.**>&& type&file&& path&/var/log/fluent/alerts&& </match>
17.
#&save&access&logs&to&MongoDB&& <match&apache.access>&& type&mongo& database&apache& collection&log&& </match>&
18.
#&Post&to&IRC& <match&**>& &&type&irc& &&host&localhost& &&port&6667& &&channel&fluentd& &&nick&fluentd& &&user&fluentd& &&real&fluentd& &&message¬ice:&%s&[%s]&%s& &&out_keys&tag,time,message& &&time_key&time& &&time_format&%Y/%m/%d&%H:%M:%S& &&tag_key&tag& </match>
19.
#&forward&other&logs&to&servers&& <match&**>&& type&forward& &&<server>&& &&&&host&192.168.0.11&& &&&&weight&20&& &&</server>& &&<server>&& &&&&host&192.168.0.12&& &&&&weight&60&& &&</server> </match>
20.
#&Send&logs&to&ElasticSearch& <match&**>& &&type&elasticsearch& &&logstash_format&true& &&host&localhost& &&port&9200& &&index_name&fluentd& </match>
21.
22.
Client Libraries Ruby Python Java PHP Node.JS Scala
23.
https://github.com/treasure-data/chef-td-agent Install with Chef Vagrantfile &
kitchen.yml included!
24.
https://github.com/dmytro/fluentd-cookbook Possibly Better https://gist.github.com/hhoover/4fceb09148a73f45136a
25.
@SADevOps http://meetup.com/SanAntonioDevOps
Download now