1. Introduction Approach Case Study Summary Future Work
Understanding Process Behaviours in a Large
Insurance Company in Australia: A Case Study1
S. Suriadi, M. Wynn, C. Ouyang, A.H.M. ter Hofstede, and
N. van Dijk
Information Systems School
Queensland University of Technology
Brisbane, Australia
s.suriadi@qut.edu.au
June 21, 2013
1
This work was supported by the Australian Research Council Discovery Project grant DP110100091.
S.Suriadi et al. Process Mining Case Study at Suncorp 1/ 23

2. Introduction Approach Case Study Summary Future Work
Outline
1 Introduction
2 Approach
3 Case Study
4 Summary
5 Future Work
S.Suriadi et al. Process Mining Case Study at Suncorp 2/ 23

3. Introduction Approach Case Study Summary Future Work
Motivation
Introduction
Case Study
A 6-month case study with Suncorp, one of the largest
insurance organizations in Australia
Goal: to identify reasons for under-performing claims, leading
to process improvement
Approach: Process Mining and L∗-methodology
Why Process Mining?
Explosion of data for analysis
Extract evidence-based insights from data (>100 orgs.)
limited application in Australia
S.Suriadi et al. Process Mining Case Study at Suncorp 3/ 23

4. Introduction Approach Case Study Summary Future Work
Contributions
Introduction
Contributions
A report on the experiences gained from this case study:
challenges, lessons learned, and recommendations
What’s new?
Validation of existing lessons learned
Detailed (new?) insights for every stage of the case study
Review of process mining-related tools
Novice/beginner’s point of view
S.Suriadi et al. Process Mining Case Study at Suncorp 4/ 23

5. Introduction Approach Case Study Summary Future Work
Methodology for ‘Spaghetti Process’
L∗ Methodology
Initial interview and preliminary data analysis suggest:
unstructured process
Adopt the L∗ methodology by van der Aalst (2011)
for “spaghetti” process
- business
understanding
- data
understanding
- historic data
- handmade
models
- objectives
- questions
- explore
- discover
- check
- compare
- promote
- diagnose
- verification
- validation
- accreditation
- redesign
- adjust
- intervene
- support
- feedback to
objectives
S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23

6. Introduction Approach Case Study Summary Future Work
Methodology for ‘Spaghetti Process’
L∗ Methodology
Initial interview and preliminary data analysis suggest:
unstructured process
Adopt the L∗ methodology by van der Aalst (2011)
for “spaghetti” process
- business
understanding
- data
understanding
- historic data
- handmade
models
- objectives
- questions
- explore
- discover
- check
- compare
- promote
- diagnose
- verification
- validation
- accreditation
- redesign
- adjust
- intervene
- support
- feedback to
objectives
Stage 0:
Plan/Justify
S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23

7. Introduction Approach Case Study Summary Future Work
Methodology for ‘Spaghetti Process’
L∗ Methodology
Initial interview and preliminary data analysis suggest:
unstructured process
Adopt the L∗ methodology by van der Aalst (2011)
for “spaghetti” process
- business
understanding
- data
understanding
- historic data
- handmade
models
- objectives
- questions
- explore
- discover
- check
- compare
- promote
- diagnose
- verification
- validation
- accreditation
- redesign
- adjust
- intervene
- support
- feedback to
objectives
Stage 0:
Plan/Justify
Stage 1:
Extract
S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23

8. Introduction Approach Case Study Summary Future Work
Methodology for ‘Spaghetti Process’
L∗ Methodology
Initial interview and preliminary data analysis suggest:
unstructured process
Adopt the L∗ methodology by van der Aalst (2011)
for “spaghetti” process
- business
understanding
- data
understanding
- historic data
- handmade
models
- objectives
- questions
- explore
- discover
- check
- compare
- promote
- diagnose
- verification
- validation
- accreditation
- redesign
- adjust
- intervene
- support
- feedback to
objectives
Stage 0:
Plan/Justify
Stage 1:
Extract
Stage 2: Control Flow and Connect
S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23

9. Introduction Approach Case Study Summary Future Work
Stage 1
Stage 1: Planning
Activities:
Presentation and discussions
Extract question(s) and assess data availability
Mutually-beneficial engagement model
Close engagement
Main Question
Why did the processing of certain ‘simple’ claim take such a long
time to complete?
Case study was conducted in two phases:
First phase: “data-driven” (quite ‘explorative’)
Second phase: “question-driven”
S.Suriadi et al. Process Mining Case Study at Suncorp 6/ 23

10. Introduction Approach Case Study Summary Future Work
Stage 1
Stage 1: Planning - Challenges and Lessons Learned
Challenges
Defining the concept of a ‘simple’ claim
No corresponding business rules
‘commonly-held’ belief, differing opinions
Took up to 5 interview sessions spanning up to 6 weeks in
total
Lessons Learned
Composition of team is crucial for effective communication
Use “question-driven” approach
S.Suriadi et al. Process Mining Case Study at Suncorp 7/ 23

11. Introduction Approach Case Study Summary Future Work
Stage 1
Stage 1: Planning - Results
Results
A clear definition of ‘simple’ claim
A claim with less than x-amount of claim value and should be
completed in no later than y-number of days
Derivation of process mining questions:
Q1: What is the performance distribution of ‘simple’ vs
‘non-simple’ claims?
Q2: What do the corresponding process models look like?
Q3: What are the key differences in the processing of “simple
quick” vs. “simple slow” claims that lead to performance
differences?
S.Suriadi et al. Process Mining Case Study at Suncorp 8/ 23

12. Introduction Approach Case Study Summary Future Work
Stage 2
Stage 2: Extract
Two phases:
First round: data quality issue, omission of important
information, poorly-populated fields
Second round: cleaner data with more accurate information
All finalized claims (no incomplete cases)
>32,000 claims
over 1 million unique events
Data cleaning, filtering, and conversion to XES/MXML
One of the most challenging stages in the case study
S.Suriadi et al. Process Mining Case Study at Suncorp 9/ 23

13. Introduction Approach Case Study Summary Future Work
Stage 2
Stage 2: Extract - Challenges
Challenges
Data interpretation
Complex process (over 20 top-level activities, > 100
second-level activities)
‘High process variant’ is the norm!
flexibility and/or ‘sloppy’ logging practice?
Inconsistent terminology (nat. hazard vs. storm vs. flood)
Data Filtering
Necessary to ensure scalability of analysis and interesting
comparative analysis
Would like to apply hierarchical filtering, but
which criteria to use?
Related to poor definition of ‘simple’ claims
S.Suriadi et al. Process Mining Case Study at Suncorp 10/ 23

14. Introduction Approach Case Study Summary Future Work
Stage 2
Stage 2: Extract - Challenges
Challenges
Data manipulation
Many tools involved, each with its own
strengths and weaknesses, and
‘quirkiness’ w.r.t data format (input and output)
Need to use them all
Highly tedious!
Spreadsheet Disco
Database
(e.g. MySQL)
Text/XML Editor
ProM Tool
CSV XES
XESAME
S.Suriadi et al. Process Mining Case Study at Suncorp 11/ 23

15. Introduction Approach Case Study Summary Future Work
Stage 2
Stage 2: Extract - Lessons Learned
Lessons Learned
Naive interpretation of data –> meaningless findings, e.g.
‘complete’ timestamp interpretation?
activity duration (from [scheduled, assign, start] to complete) -
big differences, often.
schedule
(recorded)
assign
(recorded)
actual completion time
(not recorded)
recorded as complete
(recorded)
start
(not recorded)
{actual duration
activity duration estimate_1
activity duration estimate 2
S.Suriadi et al. Process Mining Case Study at Suncorp 12/ 23

16. Introduction Approach Case Study Summary Future Work
Stage 2
Stage 2: Extract - Lessons Learned
Lessons Learned
Order of data filter matters!
Case-level vs. event-level filtering, e.g.
Rule A: remove all events not done by R
Rule B: remove all cases longer than 7 days
DISCO vs. XESAME
Rule A
event resource
activityA (start) resourceR
activityB resourceC
activityC resourceC
activityD resourceR
activityF (end) resourceE
{10 days
{7 days
The case does not satisfy
Rule B - thus whole
case is removed.
Rule A
Rule B
Rule B
1st
Filter
2nd
Filter
..........
Result
caseID:123ABC event resource
<event removed>
activityB resourceC
activityC resourceC
<event removed>
activityF resourceE
caseID:123ABC
event resource
activityB resourceC
activityC resourceC
activityF resource E
caseID:123ABC
S.Suriadi et al. Process Mining Case Study at Suncorp 13/ 23

17. Introduction Approach Case Study Summary Future Work
Stage 2
Stage 2: Extract - Result
Result
Clear identification of data slices to be used for analysis and
comparison to address the 3 process mining questions defined
earlier.
CaseDuration(days)
$x
Payout amount
Simple Slow (SS):
<=$x payout
> y days
Simple Quick (SQ):
<=$x payout
<= y days
Complex Slow (CS):
>$x payout
> y days
Complex Quick (CQ):
>$x payout
<=y days
"in-between"
cases
y+1
y
y-1
S.Suriadi et al. Process Mining Case Study at Suncorp 14/ 23

18. Introduction Approach Case Study Summary Future Work
Stage 3
Stage 3: Analysis - Challenges
Control-flow analysis, using Disco and ProM Tool.
Challenges
Discovering meaningful process models
Dealing with complex log
‘Inconsistency in behaviour’ is the norm!
Practicality: time consuming process (especially if data had to
be further filtered/cleaned)
genetic miner, ILP miner (sometimes)
S.Suriadi et al. Process Mining Case Study at Suncorp 15/ 23

19. Introduction Approach Case Study Summary Future Work
Stage 3
Stage 3: Analysis - Lessons Learned
Lessons Learned
No ‘clean’ structure or we have not done proper data
pre-processing?
Clues for the non-existence of structured process: the
existence of
high process variants,
low fitness value of Heuristic nets, and
flower-like model in simplified Petri Nets,
despite the application of hierarchical filtering and other
cleaning activities.
Useful algorithms: Heuristics Miner, ILP, Uma, as well as
Fuzzy miner (in Disco).
Not so useful: Genetic miner
S.Suriadi et al. Process Mining Case Study at Suncorp 16/ 23

20. Introduction Approach Case Study Summary Future Work
Stage 3
Stage 3: Analysis - Results
Results
Q1: a significant number of under-performing cases
Q2: unstructured process indeed!
S.Suriadi et al. Process Mining Case Study at Suncorp 17/ 23

21. Introduction Approach Case Study Summary Future Work
Stage 4
Stage 4: Interpretation
Proper identification of differences via classical data mining
techniques
Compare Simple Quick vs. Simple Slow claims
Challenges
Finding a good set of predictor variables (too many attributes)
S.Suriadi et al. Process Mining Case Study at Suncorp 18/ 23

22. Introduction Approach Case Study Summary Future Work
Stage 4
Stage 4: Interpretation - Lessons Learned and Results
Lessons Learned
Two useful process-related metrics:
average execution of an activity-X per case
distribution of an activity-X over all cases
Results
Q3 is addressed
Key differences between ‘Simple Quick’ and ‘Simple Slow’
classes were identified.
S.Suriadi et al. Process Mining Case Study at Suncorp 19/ 23

23. Introduction Approach Case Study Summary Future Work
Stage 4
Stage 4: Interpretation - Result
Activity Simple Quick Simple Slow
actFreq actDist actFreq actDist
Follow Up Requested 1.86 74.4% 5.79 92.3%
Incoming Correspondence 1.75 81.6% 4.27 90.1%
Contact Customer 0.66 46.8% 1.29 63.3%
Contact Assessor 0.11 4.9% 1.36 21.5%
Conduct File Review 2.03 89.8% 6.11 96.9%
S.Suriadi et al. Process Mining Case Study at Suncorp 20/ 23

24. Introduction Approach Case Study Summary Future Work
Conclusion
Conclusion of the Case Study - Stage 5
Useful findings:
Surprising number of under-performing claims
Highlighted non-uniformity and the need for standardization in
the claims processing
Triggered some changes in their claims processing system
“...by mining and analysing our claims ... our business has been
able to make cost saving adjustments to the existing process.”
a new system was trialled
Another subsequent impact (not involved, but informed
verbally)
improved the proportion of claims finalized on-time
S.Suriadi et al. Process Mining Case Study at Suncorp 21/ 23

25. Introduction Approach Case Study Summary Future Work
Future Work
Future Work
Our case study: too afraid to manipulate data too much
(limited data removal, conservative filtering)
But, fine-line between abstraction and over-simplification
When do we stop simplifying/cleaning data?
Data quality - event log quality
Claim: objective insights
Pitfall: low data quality (incomplete/incorrect/noisy) and/or
improper cleaning, abstraction and manipulation
Verification of results is needed!
Establish links between original data and results
Assert the correctness of results
Improved accountability in making important decisions
S.Suriadi et al. Process Mining Case Study at Suncorp 22/ 23

26. Introduction Approach Case Study Summary Future Work
Future Work
Understanding Process Behaviours in a Large
Insurance Company in Australia: A Case Study2
S. Suriadi, M. Wynn, C. Ouyang, A.H.M. ter Hofstede, and
N. van Dijk
Information Systems School
Queensland University of Technology
Brisbane, Australia
s.suriadi@qut.edu.au
June 21, 2013
2
This work was supported by the Australian Research Council Discovery Project grant DP110100091.
S.Suriadi et al. Process Mining Case Study at Suncorp 23/ 23