Business today is starting to understand the value of data, and some organisations are outperforming their competition by putting data at the heart of their thinking. Leveraging data to change business models, understand their customers and employees better and deliver new revenue streams is the driving force in this new data centric era.
Jon Woodward - MSFT
Dave Coplin - MSFT
Mike Bugembe - JustGiving
Gary Richardson - KPMG
2. UK Business Lead for BI & Analytics
Jon Woodward : Connect & Follow
@JLWoodward
www.linkedin.com/in/jonathanwoodward
#DataCulture
3. Data Culture Industry Immersions Community Other
Summits Events Events
Sept, London
22nd Sept Data Culture Summit – Business
23rd Sept Data Culture Summit - Technical
22nd Sept Data Culture Dinner - Executives
Dec, Birmingham
8th Dec Data Culture Summit, Business
9th Dec Data Culture Summit, Technical
8th Dec Data Culture Dinner, Executive
March, London
8th Mar Data Culture Summit, Business
9th Mar Data Culture Summit, Technical
8th Mar Data Culture Dinner, Executive
May, London
9th May Data Culture Summit, Business
10th May Data Culture Summit, Technical
9th May Data Culture Dinner, Executive
Future Decoded, Nov 10-11th, London
Data Culture Tracks
https://futuredecoded.microsoft.com/
10th Nov, Futures Data Platform Roundtable
10th Nov, Dashboard in an Hour
10th Nov, Data Culture Panel
10th Nov, Data Culture Dinner
Gartner BI Summit , Feb 29th-1st March,
London
http://www.gartner.com/events/emea/busin
ess-intelligence
29th Feb, Data Culture Dinner
6th October, Reading
Dashboard in a Day
27th October, Reading
Platform Modernisation
20th January, London
Dashboard in a Day
Platform Modernisation
11th February, Reading
Dashboard in a Day
Platform Modernisation
21st March, London
Dashboard in a Day
Platform Modernisation
21st April, Edinburgh
Dashboard in a Day
Platform Modernisation
11th May, Reading
Dashboard in a Day
Platform Modernisation
8th June, London
Dashboard in a Day
Platform Modernisation
10/11th September
Cambridge SQL Saturday
http://www.sqlsaturday.com/41
1/eventhome.aspx
October SQL Relay
http://www.sqlrelay.co.uk/
7th Nottingham
8th London
12th Reading
13th Bristol
14th Cardiff
15th Birmingham
28th Nov, London
Data Culture PowerBI Edition
http://www.eventbrite.com/e/d
ata-culture-day-london-power-
bi-edition-tickets-18258788528
5th Nov, London
IRM Data Science Track
http://www.irmuk.co.uk/ed
bi2015/postworkshops.cfm
UK DATA CULTURE EVENTS
26th Nov, London
Dashboard in a Day
Platform Modernisation
Cloud RoadShow, Feb 29th-1st March,
London
SQLSaturday, Exeter – 11/12
March
SQLBits, May
SQL Saturday, Edinburgh –
10/11 June
4. Data Culture for Marketing
Data Culture Summits – Sept/Dec/Mar/May
Data Culture for IT Executives
Data Platform Modernisation
Data Culture for Finance
IoT Track
Machine Learning and Analytics Track
Visualisation and Data Discovery Track
Big Data and Data Management Track
Day 1 - Business Day Day 2 - Technical Day
Dashboard in a Day
5. Dave Coplin – Chief Envisioning Officer, Microsoft
09.30 – 10.00 Mike Bugembe – Chief Data Officer, JustGiving
Break
10.15 – 12.30 Morning Tracks
Lunch
13.30 – 16.30 Dashboard in a Day Continues
Close
28. "Aoccdrnig to a rseecharer at Cmabrigde Uinervtisy,
it deosn't mttaer in waht oredr the ltteers in a wrod
are, the olny iprmoatnt tihng is taht the frist and lsat
ltteers be at the rghit pclae. The rset can be a toatl
mses and you can sitll raed it wouthit porbelm. Tihs
is bcuseae the huamn mnid deos not raed ervey
lteter by istlef, but the wrod as a wlohe."
61. So this is what we planned to build
14 years of giving behaviour, online, web traffic, academic research
Engagin
g
Machine
Learning
Social
Graph
Theory
Personal
62. To achieve our vision we built
an intelligent machine that…
Give
Care about
Engaging content
72. Intro + Welcome
10.30 – 11.00 Benefits of a Data Culture
11.00 – 11.45 The Future is Data Driven
11.45 – 12.15 Enabling Data Culture in your Organisation
12.15 – 12.30 Next Steps
Lunch
73. UK Business Lead for BI & Analytics
Jon Woodward : Connect & Follow
@JLWoodward
www.linkedin.com/in/jonathanwoodward
#DataCulture
74. Director of Data Engineering, KPMG
Gary Richardson: Connect & Follow
@GaryData
https://uk.linkedin.com/in/richardsongary
84. Algorithm’sThings
• Create new business models
• Provide better service and improve customer
experiences
• Respond to changes in the market faster
• Improve product availability and usage
• Open new revenue streams
85. Algorithm’sIntelligence
Predictions – Ray Kurzweil
2010- Supercomputer to emulate human intelligence
2020 – Human intelligence computing available for
$1000
2029 – Pass Turing Test
2030 – non-biological computation will surpass capacity
of all living human intelligence
2045 – Singularity
Tomorrow
138. Leverage the Edge
Tune your business to
embrace Machine Intelligence
138
What Should we do Today
Build a Data Culture Plan
Move beyond Big Data
Hinweis der Redaktion
For those that don’t know me, I am the business lead for Microsoft covering Business Intelligence and Advanced Analytics.
This covers a whole host of technologies from SQL Server to PowerBI to Azure Machine Learning and a list of other exciting data technologies.
Follow and connect to me via Twitter and Linkedin as you can see here – I post all my decks and thoughts that way..
So lets start with a question
For those that don’t know me, I am the business lead for Microsoft covering Business Intelligence and Advanced Analytics.
This covers a whole host of technologies from SQL Server to PowerBI to Azure Machine Learning and a list of other exciting data technologies.
Follow and connect to me via Twitter and Linkedin as you can see here – I post all my decks and thoughts that way..
So lets start with a question
Source: Henderson, Bobby (2005). "Open Letter To Kansas School Board". Venganza.org. Archived from the original on 2007-04-07.
Focusing on the ultimate outcomes, not the individual processes.
.
This is what I shared with you last year/now progress etc
This is what has genereated that 3 billion donations……..
You can see there is a fundraising story, an event that they are doing, donations from a friends….so much information
Worked really well – this is a great example of the interesting things that people would do for charity – but would you do this for any cause or would it make a difference if you were really passionate about your cause! We also saw that we could get a huge uplift if we also made that easier for you to do.
All of this stuff flows through networks
We live in a connected world,
and these networks are appearing everywhere.
Facebook
Linked in
Twitter
Skype
Phone book
email
Even the 15 year old fundraising page, one person decides to do something for a cause that she cares about
Then shares with their friends who come and support. 24 million people have used JustGiving so there is a good chance that their friends have given to someone else
It looks something like this it is a network a connection of people connected to people. When you zoom out to see how the 26 million people on our platform are connected to each other you end up with the givegrapgh – a graph is a complex system of nodes and edges.
This means we have can now use big data
To do graph calculations
We call these betweeness and centrality calculations
They tell us strengths of relationships
Nature of relationships
Influencers,
Getting the word around quickly
Important nodes
The givegraph is important allows us to understand how generocity flows through the network. It allows us to work out relative strength of what you care about. Now without big data, some of these calculations were impossible in the past, if you i
An engaging product – the feed, with notifications like may engaging products out there supported by machine learning for personal content to appear on that, and the graph to bring social connections into the picture…
We built a machine that could work out how you give, (its not about money, some like to give time, energy), the algorithms can work out what you care about at that time, remember what you care about is not static, it changes with circumstances and associations but most importantly, it not only knows how to keep you engaged, it has been trained to interact and engage with you. All of this makes up the give graph, a machine level understanding of the world of giving and how to operate in it.
Which you can see here. This is the product that uses and consumes all of the intelligence that is presented. Changing giving from a transactional action to a more engaging social action, enabling people to give, removing the barriers that we have created for giving. As I finish of just now, I really cannot emphasise enough how impactful this is going to be. And one of the main reasons why we are in a position to do this is our vast amounts of data collated over the years and because of the ease of use and flexibility of the tools available on the Azure platform that really enabled us to focus on the task of ensuring that every great cause gets the funding that it requires.
Thank you.
Our team is made up of scientists and engineers and we needed to address the problem fast without access to a large operations teams that specialize in managing a Hadoop infrastructure. With Azure we really enjoyed the benefits of the platform as a service. It means our scientists and engineers can focus on solving the challenges that I outlined earlier.
And we selected Microsoft Azure. This offered us the key features that ensures that we didn’t have to compromise on security, development language and robustness.
- its a platform as a service (Paas) - allows us to focus on our applications whilst Azure manages the rest, this signifiicantly reduces complexity allows us to move so much faster
- flexibility - allowing us to use a range of languages and operating systems to enable us to get whatever we need to work
- its always up, always on
- designed for security - unmatched experience running trusted online services.
More specifically we used
HDInsight for running Hadoop jobs to build the giving graph.
Azure Cloud Services (with Azure Autoscaling & Azure Storage Queues) to upload the output from big data analysis into Azure Table Storage where the results are persisted.
Azure Service Bus to manage delivery of event messages & updates to the graph.
F# to implement the algorithm to calculate the Target Value.
Azure websites (with Azure Autoscaling) to host the Api that drives the product.
There was some skeptism in
These are the components that we used. To make these algorithms real we use a combination of batch and real time processing, the batch is the back bone of this process and at the center of this is HdInsight.
Our TS data is extracted from our on premise SQL servers and uploaded to Azure blob storage and the then we use map reduce to build the graph and do the calculations, Most of our existing Jobs are Java Map reduce but we are actively looking at Spark for the machine learning, this will allow us to use python and scala. This is kicked off using a job schedule and the orchestration service which spin up a cluster and schedules the map reduce jobs. Once the jobs are finished the cluster is destroyed and the data is imported to table storage ready for consumption.
For Real time, you can see from the diagram that we combine F# mailboxes with Azure service bus. All events that take place on the platform are streamed through or on to the service bus. Allowing us to give the user an interactive experience.
The results are merged with the batch results in table storage
To present the data we use Azure websites to host our API’s again this is where the platform as a service enables us to have a managed service and elastic scalability out of the box. We run Azure distributed cache to allow us to present the data that is persisted in Azure storage and simultaneously manage any spikes in traffic.
All of this manifests its self on the product as a feed.
Ensure that every great cause gets the funding that it deserves. Our vision now this is pretty challenging, there are billions of people in the world who have the ability to give to a cause that they care about and if they all did we could eradicate poverty, we could put cancer out of business. But why are they not giving and how is technology going to help solve that problem. It turns out that a few years a go we found that we could address this problem with machine learning and big data. We had to use our data to address the following challenges
Our team is made up of scientists and engineers and we needed to address the problem fast without access to a large operations teams that specialize in managing a Hadoop infrastructure. With Azure we really enjoyed the benefits of the platform as a service. It means our scientists and engineers can focus on solving the challenges that I outlined earlier.
For those that don’t know me, I am the business lead for Microsoft covering Business Intelligence and Advanced Analytics.
This covers a whole host of technologies from SQL Server to PowerBI to Azure Machine Learning and a list of other exciting data technologies.
Follow and connect to me via Twitter and Linkedin as you can see here – I post all my decks and thoughts that way..
So lets start with a question
For those that don’t know me, I am the business lead for Microsoft covering Business Intelligence and Advanced Analytics.
This covers a whole host of technologies from SQL Server to PowerBI to Azure Machine Learning and a list of other exciting data technologies.
Follow and connect to me via Twitter and Linkedin as you can see here – I post all my decks and thoughts that way..
So lets start with a question
Key Points:
Data is currency in the twenty-first century
Companies that take advantage of data opportunities have the potential to outperform those that do not
Talk Track:
What asset is most leveraged by today’s thriving companies? Data.
Data will be a key differentiator for businesses today and in the future.
You constantly hear in the news about new ways in which businesses are using data as a competitive advantage.
You hear how people in those organizations are making fast, informed decisions like never before possible.
So the question is, what are these thriving companies doing with data?
But Lets start by looking at the acceleration we are seeing in industry and many of you will be experiencing this first hand
We live in a world today, where many countries have more mobile devices than humans
We live in a world where we have created more data in the last 2 year than in the history of humankind. We are creating 2.5 quintillion bytes of data per day.
We live in a world where 200Bn devices and sensors are being connected to the internet, accelerating the data deluge we already live in
We live in hybrid mobile first cloud first world where innovation is accelerating. , where the telephone took 75 years to get to 100 million users and Instagram took 2.4 years…
And it this data with added intelligence that is driving the experience we are having with hundreds of millions of users around the world.
From skype translate, where you can select the language you want to speak in, and in real time your speech will be translated
To Cortana which enables you to ask a question, to Delve which understands and surfaces interesting information directly to you and PowerBI that allows you to ask questions of your data and without developing anything it will display graphs automatically based on your question.
And it is these data companies that are defining this new breed of organisation
From uber that has no cars
From facebook that has no content of its own
To Airbnb that has no real estate on its books
To Alibaba, which has no inventory.
These organisations are multi-billion $ organisations due to one thing, their business model is based on a data culture.
And it is with these three areas that we enter a new data world order.
In the age of algorithms, very much where we are today, organisations are starting to differentiate based on outputs from algorithms. The best recommendation, the best search.
And now we are also starting to see with the age of things, where everything is fully connected, brand new business models, significant efficiencies and also we need to cope with the deluge of data that this brings.
And in the age of intelligence where we are imbueing differeing levels of machine intelligence into business processes, applications and augment humans
Taking a look at your business through the lens of these three aspects and looking at how you can change your business process, create a new business model, spin out a new business based on the data you traditionally capture, is the future..
And it is with these three areas that we enter a new data world order.
In the age of algorithms, very much where we are today, organisations are starting to differentiate based on outputs from algorithms. The best recommendation, the best search.
And now we are also starting to see with the age of things, where everything is fully connected, brand new business models, significant efficiencies and also we need to cope with the deluge of data that this brings.
And in the age of intelligence where we are imbueing differeing levels of machine intelligence into business processes, applications and augment humans
Taking a look at your business through the lens of these three aspects and looking at how you can change your business process, create a new business model, spin out a new business based on the data you traditionally capture, is the future..
And it is with these three areas that we enter a new data world order.
In the age of algorithms, very much where we are today, organisations are starting to differentiate based on outputs from algorithms. The best recommendation, the best search.
And now we are also starting to see with the age of things, where everything is fully connected, brand new business models, significant efficiencies and also we need to cope with the deluge of data that this brings.
And in the age of intelligence where we are imbueing differeing levels of machine intelligence into business processes, applications and augment humans
Taking a look at your business through the lens of these three aspects and looking at how you can change your business process, create a new business model, spin out a new business based on the data you traditionally capture, is the future..
And it is with these three areas that we enter a new data world order.
In the age of algorithms, very much where we are today, organisations are starting to differentiate based on outputs from algorithms. The best recommendation, the best search.
And now we are also starting to see with the age of things, where everything is fully connected, brand new business models, significant efficiencies and also we need to cope with the deluge of data that this brings.
And in the age of intelligence where we are imbueing differeing levels of machine intelligence into business processes, applications and augment humans
Taking a look at your business through the lens of these three aspects and looking at how you can change your business process, create a new business model, spin out a new business based on the data you traditionally capture, is the future..
And it is with these three areas that we enter a new data world order.
In the age of algorithms, very much where we are today, organisations are starting to differentiate based on outputs from algorithms. The best recommendation, the best search.
And now we are also starting to see with the age of things, where everything is fully connected, brand new business models, significant efficiencies and also we need to cope with the deluge of data that this brings.
And in the age of intelligence where we are imbueing differeing levels of machine intelligence into business processes, applications and augment humans
Taking a look at your business through the lens of these three aspects and looking at how you can change your business process, create a new business model, spin out a new business based on the data you traditionally capture, is the future..
And it is with these three areas that we enter a new data world order.
In the age of algorithms, very much where we are today, organisations are starting to differentiate based on outputs from algorithms. The best recommendation, the best search.
And now we are also starting to see with the age of things, where everything is fully connected, brand new business models, significant efficiencies and also we need to cope with the deluge of data that this brings.
And in the age of intelligence where we are imbueing differeing levels of machine intelligence into business processes, applications and augment humans
Taking a look at your business through the lens of these three aspects and looking at how you can change your business process, create a new business model, spin out a new business based on the data you traditionally capture, is the future..
Gartner predicts tat by 2017, 20% of all market leaders will lose their position to a company founded after the year 2000
And if we take a look at the company index – the company lifespan is getting shorter and shorter with a new number of new entrants into the market not even existing today.
And if we look at the research.
IDC and Microsoft undertook research across a large set of organisations to understand the differential between data driven organisations and lets call them traditional organisations.
The research found that organisations that gather diverse data, apply new analytics, make that available to more people and do that at speed, unlock what they termed the Data Dividend. Essentially across areas such as employee productivity, operations and customer facing projects, unlocking the data dividend can have a top and bottom line affect. The research suggests that there is circa £53B available for organisations to go after by unlocking this dividend, making themselves more productive, engaging better with their customers and employees and driving new innovation.
How do we tap into this – we start with having a modern data platform and apply a data culture to our organization.
Key Points:
Data is currency in the twenty-first century
Companies that take advantage of data opportunities have the potential to outperform those that do not
Talk Track:
What asset is most leveraged by today’s thriving companies? Data.
Data will be a key differentiator for businesses today and in the future.
You constantly hear in the news about new ways in which businesses are using data as a competitive advantage.
You hear how people in those organizations are making fast, informed decisions like never before possible.
So the question is, what are these thriving companies doing with data?
We have seen the benefits through the data dividend
We have seen the power of a modern data platform
We have seen the future of analytics and data science
But how, practically do we drive a Data culture
As we look at all the research and also how we have learnt on the MSFT journey towards a data culture, I have called out 4 areas of focus, which cover Leadership, organisarion, decision making and data. The aim of this section is to discuss how we make this culturally work inside our organisations; and please contribute where you have seen what has worked.
The research suggests, along with almost all other organisational and cultural change, it needs to be lead from the top.
As the number one successful strategy, this is probably also one of the hardest, as in many cases the top table don’t fully understand data
Which leads onto the next point
We need to put someone in control of the data and in larger organisations you may have different viewpoints/ownership.
From operational data control to data exploitation
A key aspect to focus on to get buy in is around data quality and governance.
How do you ensure a set of managed data sets that decisions are taken on. At managed data set creations (i.e. sales, pipeline) are controlled and published on the required frequency. There is strong goverance around the data and when we come to monthly and half year business review, the data pack, which is significant is shared to all required from a single source.
Many organisations are starting to think about data as money, and actually individual differentiated algorithms as key IP. We see many organisations valued at significant multiples based on their collection, use and exploitation of data. The future of having a balance sheet item around data is very near.
Data Culture is only valuable if you get it out to as many people as possible. Do not hold data to restricted sets of people, but carefully think about who should get to see the datasets. Enabling collaborate decision making based on data can only be made when everyone in the decision loop has that data..
Look to were you can refine and create roles that manage, disseminate and oversee the data
Look to educate the entire organisation in the value of data driven thinking, from the use of dashboarding tools for the wider population to specific forecasting/analytics for the data analysts/scientists within your organisations.
Be transparent with the use of the data. Ensure people know where it came from and what decisions will be taken off the back of the data.
Start every question with asking for the data to back up the recommendation or support the argument. If no data driven thinking has taken place, send them back to the drawing board to collect and analyse the data.
Ensure that you look for all opportunities to leverage data and build it into your thinking.
One area that we have found that drives wider data culture is to get everyone onto using a company scorecard.
Everyone In the UK org is very aware of the UK scorecard, which products are doing well, which needs further support
We also collect data that support leading indicators, i.e data that determines later success. i.e. how many opportunities does a seller have
We have determined the threshold for each of these indicators
We also then benchmark ourselves against the similar sized subsidiaries to understand and learn from the data on what is going well and where to improve
Real time dashboards, and we are not here on this yet, takes the dashboard to the next level.
What we do have is end user refreshable dashboards – i.e. the end user is in control.
Start with the data – and look for the gaps in the data, data anomalies that need further review – and constantly drive for better and better data
The Data Culture journey keeps going… as you ask more and questions with data insight, you will undoubtedly drive a requirement for further data.
As you start to collect more and more data, probably from many different locations both internal and external to the organisation, this is where data operations becomes a key aspect to drive automation into.
You don’t want the data burden to include too many humans in the loop, as a) it will take too long and b) it will introduce errors
Also look for ways to automate the decision making and actually remove the human from the loop completely.. We are starting to see several organisations start to manage their workforce through having an intelligent machine as the supervisior ..
Finally actually on the data and kind of covered in the other areas.
Build versions of the truth and ensure that all aspects of the business leverage the correct version – test this until you are sure they are using the right data or algorithm… how many times has an excel spreadsheet been presented as the answer, and we are not sure of the source…
Many tools exist today to enable data sharing, data discovery and data visualisation that non technical people can pick up through to advanced data science tooling. Ensure you look to the best way to leverage these tools to deliver on the data dividend.
Expand the data footprint; look for ways to encompass and bring into the decision making process more and more relevant data.
Organisations are using weather, traffic, and other external factors to tune the business process in real time.
For all the hype around Big Data and interesting its has now dropped of the Gartner hype cycle, so its actually mainstream !!
But my point here is to move beyond Big Data and actually embrace the concept of all data.
When looking at your business processes, look to leverage the edge where possible. Leverage sensors that give you insight into your operations. We have a lift company that has 1 million plus lifts that they have instrumented with sensors and this has allowed them to shift from annual maintence schedules on all their lifts to predictive maintenance based on machine learning.
In summary, from the research Top down as a strategy worked best and interestingly adding additional data analytst/scientists did not..
So what does the future look like
Well as we think about the areas we should be focussed on – it is enabling Humans with Machine Intelligence, All Data and All Things to drive forward our companies.
Now back to today. We are firmly in the world of Narrow Intelligence, but even this is redefining how organisations are working, and organisations that are starting to leverage machine intelligence are starting to outperform their peers and have better relationship with their customers. Nearly all my conversations with customers now, are now around how they leverage machine intelligence to improve, create, shape how they move into this new age.
Just one example, is Justgiving. They have leveraged some complex algorithms and machine intelligence to understand the relationship people have between themselves and the causes that are looking for funding. They are doing this across 361 million relationships and presenting in realtime information that has changed their business model and allow them to dramatically improve the level of giving that occurs. That’s the power of machine intelligence.