SlideShare ist ein Scribd-Unternehmen logo

Scoring Metrics for Classification Models

•
•
KNIMESlides
KNIMESlides

You have trained a classification model with a highly sophisticated Machine Learning algorithm. Right. It is now time to evaluate its performance on test data, i.e. to score it. A number of scoring metrics have been proposed over the years in different domains: sensitivity and specificity, precision and recall, accuracy, area under the curve, Cohen’s Kappa, and many more. Generally, they are based on values reported in a confusion matrix. These slides are from a webinar we presented where we explore the concept of confusion matrix, true/false positives/negatives, and the related, most commonly used scoring metrics for classification models. We also demonstrate how to calculate all those metrics within KNIME Analytics Platform. https://www.knime.com/knime-software/knime-analytics-platform View the webinar here: https://youtu.be/dOqRjeOv1VA

Technologie
1 von 28
Downloaden Sie, um offline zu lesen
© 2019 KNIME AG. All rights reserved. Scoring Metrics for Classification Models KNIME: Maarit.Widmann@knime.com @KNIME What to do after training a machine learning algorithm
© 2019 KNIME AG. All rights reserved. Introduction There are many scoring metrics for a classification model. Which of them fits depends on your classification problem. 2
© 2019 KNIME AG. All rights reserved. Different Scoring Metrics 3 1. Confusion Matrix • True positives • False negatives • False positives • True negatives 2. Sensitivity and Specificity 3. Precision and Recall 4. F-measure 5. Overall Accuracy and Cohen‘s kappa
© 2019 KNIME AG. All rights reserved. Why different scoring metrics? 4 1. What is your objective? 2. What is the target class distribution? 3. Is the target binomial or multinomial?
© 2019 KNIME AG. All rights reserved. Introduction Doctor‘s diagnosis as an example of classification 5 Sample of patients Disease carrier Healthy Diagnosis Classification results Positive class Negative class
© 2019 KNIME AG. All rights reserved. Confusion Matrix 6
© 2019 KNIME AG. All rights reserved. Introduction Disease detection using a machine learning algorithm Sample of patients split into training (80 %) and test (20 %) set Model training and prediction Evaluation of classification results
© 2019 KNIME AG. All rights reserved. Scoring Metrics for Classification Models 9
© 2019 KNIME AG. All rights reserved. Sensitivity • Sensitivity= 𝑻𝑷 𝑻𝑷+𝑭𝑵 = 𝟑 𝟑+𝟏 = 𝟎. 𝟕𝟓 Are ALL positive class events found by the model?
© 2019 KNIME AG. All rights reserved. Specificity • Specificity= 𝑻𝑵 𝑻𝑵+𝑭𝑷 = 𝟐𝟎 𝟐𝟎+𝟒 = 𝟎. 𝟖𝟑 Are ALL negative class events found by the model? Sensitivity: Is the model sensitive to detecting disease? Specificity: Is the disease diagnosis specific?
© 2019 KNIME AG. All rights reserved. Recall • Recall= 𝑻𝑷 𝑻𝑷+𝑭𝑵 = 𝟑 𝟑+𝟏 = 𝟎. 𝟕𝟓 Are ALL positive class events found by the model?
© 2019 KNIME AG. All rights reserved. Precision • Precision= 𝑻𝑷 𝑻𝑷+𝑭𝑷 = 𝟑 𝟑+𝟒 = 𝟎. 𝟒𝟑 Are ONLY positive class events found by the model? Recall: Detect the most disease carriers Precision: Make precise disease prediction
© 2019 KNIME AG. All rights reserved. Defining the Classification Threshold Machine learning model predicts each patient Score(Diagnosis=disease carrier). The class assignment is based on the set threshold for this score. True positives False positives False negatives True negatives Score(Diagnosis= disease carrier)0 0.5 1 Recall ↓↑ Precision
© 2019 KNIME AG. All rights reserved. F-measure • F-measure= 2 ∗ 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛∗𝑟𝑒𝑐𝑎𝑙𝑙 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛+𝑟𝑒𝑐𝑎𝑙𝑙 = 2 ∗ 0.43∗0.75 0.43+0.75 ≈ 0.55 Harmonic mean of precision and recall. Are ALL and ONLY positive class events found by the model?
© 2019 KNIME AG. All rights reserved. Overall Accuracy • Overall Accuracy= 𝑻𝑷+𝑻𝑵 𝑻𝑷+𝑭𝑷+𝑭𝑵+𝑻𝑵 = 𝟑+𝟐𝟎 𝟑+𝟏+𝟒+𝟐𝟎 = 𝟎. 𝟖𝟐 Target class distribution must be balanced! Probability of classifying a positive OR negative class event correctly.
© 2019 KNIME AG. All rights reserved. Cohen‘s kappa (𝜿) • 𝜅 = 𝑝0−𝑝 𝑒 1−𝑝 𝑒 , where 𝑝0 is the overall accuracy by the model 𝑝 𝑒 = 𝑝 𝑒1 + 𝑝 𝑒2 𝑝 𝑒1 = 𝑝 𝑝𝑟𝑒𝑑="𝑑𝑖𝑠𝑒𝑎𝑠𝑒 𝑐𝑎𝑟𝑟𝑖𝑒𝑟" × 𝑝 𝑎𝑐𝑡="𝑑𝑖𝑠𝑒𝑎𝑠𝑒 𝑐𝑎𝑟𝑟𝑖𝑒𝑟" 𝑝 𝑒2 = 𝑝 𝑝𝑟𝑒𝑑="ℎ𝑒𝑎𝑙𝑡ℎ𝑦" × 𝑝 𝑎𝑐𝑡="ℎ𝑒𝑎𝑙𝑡ℎ𝑦" The overall accuracy using a random classifier.
© 2019 KNIME AG. All rights reserved. 𝑝 𝑒1 = 7 28 × 4 28 𝑝 𝑒2 = 21 28 × 24 28 𝑝 𝑒 = 𝑝 𝑒1 + 𝑝 𝑒2 = 0.68 𝑝0 = 23 28 = 0.82 𝜅 = 𝑝0−𝑝 𝑒 1−𝑝 𝑒 = 0.14 0.32 ≈ 0.44 Cohen‘s kappa (𝜿) vs. Overall accuracy Overall accuracy 𝑝 𝑒1 = 5 28 × 4 28 𝑝 𝑒2 = 23 28 × 24 28 𝑝 𝑒 = 𝑝 𝑒1 + 𝑝 𝑒2 = 0.73 𝑝0 = 21 28 = 0.75 𝜅 = 𝑝0−𝑝 𝑒 1−𝑝 𝑒 = 0.02 0.27 = 0.07 New model 𝜅 = 1: perfect model performance 𝜅 = 0: performance of a random classifier
© 2019 KNIME AG. All rights reserved. Scoring Metrics for a Multivariate Classification Model 22 Sample of patients Disease carrier Healthy Diagnosis Classification results Recessive disease carrier Positive class Negative class
© 2019 KNIME AG. All rights reserved. Confusion Matrix 23 True positives False positives False negatives True negatives
© 2019 KNIME AG. All rights reserved. Classification Model Evaluation in KNIME 26
© 2019 KNIME AG. All rights reserved. Scorer (JavaScript) node 27
© 2019 KNIME AG. All rights reserved. Interactive View: Confusion Matrix 28
© 2019 KNIME AG. All rights reserved. Interactive View: Confusion Matrix 29
© 2019 KNIME AG. All rights reserved. Workflow for Classification 30 • On KNIME Workflow Hub: Evaluating Classification Model Performance • On EXAMPLES Server: EXAMPLES/04_Analytics/10_Scoring/01_Eval uating_Classification_Model_Performance
© 2019 KNIME AG. All rights reserved. Summary • After training a classification model, the model performance is reported using scoring metrics • Scoring metrics describe and compare the model performance • Confusion matrix shows the numbers of correct and incorrect predictions • Class statistics and overall accuracy statistics are based on the values in the confusion matrix 32
© 2019 KNIME AG. All rights reserved. KNIME Fall Summit 2019 November 5 – 8 at AT&T Executive Education and Conference Center, Austin, Texas • Tuesday & Wednesday: One-day courses • Thursday & Friday: Summit sessions Register by October 1 for Early Bird Discount! Register at knime.com/summits
© 2019 KNIME AG. All rights reserved. KNIME Beginner’s Luck Course Book downloadable from KNIME Press https://www.knime.com/knimepress with code: SCORING-METRICS-0519
© 2019 KNIME AG. All rights reserved. 36 The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME.com AG under license from KNIME GmbH, and are registered in the United States. KNIME® is also registered in Germany. Thank You!

Empfohlen

Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringHJ van Veen
 
Decision tree and random forest
Decision tree and random forestDecision tree and random forest
Decision tree and random forestLippo Group Digital
 
Linear Regression and Logistic Regression in ML
Linear Regression and Logistic Regression in MLLinear Regression and Logistic Regression in ML
Linear Regression and Logistic Regression in MLKumud Arora
 
Decision Tree - C4.5&CART
Decision Tree - C4.5&CARTDecision Tree - C4.5&CART
Decision Tree - C4.5&CARTXueping Peng
 
Methods of Optimization in Machine Learning
Methods of Optimization in Machine LearningMethods of Optimization in Machine Learning
Methods of Optimization in Machine LearningKnoldus Inc.
 
Introduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersFunctional Imperative
 
Random forest
Random forestRandom forest
Random forestUjjawal
 
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Simplilearn
 

Empfohlen

Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringHJ van Veen
 
Decision tree and random forest
Decision tree and random forestDecision tree and random forest
Decision tree and random forestLippo Group Digital
 
Linear Regression and Logistic Regression in ML
Linear Regression and Logistic Regression in MLLinear Regression and Logistic Regression in ML
Linear Regression and Logistic Regression in MLKumud Arora
 
Decision Tree - C4.5&CART
Decision Tree - C4.5&CARTDecision Tree - C4.5&CART
Decision Tree - C4.5&CARTXueping Peng
 
Methods of Optimization in Machine Learning
Methods of Optimization in Machine LearningMethods of Optimization in Machine Learning
Methods of Optimization in Machine LearningKnoldus Inc.
 
Introduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersFunctional Imperative
 
Random forest
Random forestRandom forest
Random forestUjjawal
 
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Simplilearn
 
Module 4: Model Selection and Evaluation
Module 4: Model Selection and EvaluationModule 4: Model Selection and Evaluation
Module 4: Model Selection and EvaluationSara Hooker
 
Bias and variance trade off
Bias and variance trade offBias and variance trade off
Bias and variance trade offVARUN KUMAR
 
Bayesian statistics
Bayesian statisticsBayesian statistics
Bayesian statisticsSagar Kamble
 
Introduction to XGBoost
Introduction to XGBoostIntroduction to XGBoost
Introduction to XGBoostJoonyoung Yi
 
From decision trees to random forests
From decision trees to random forestsFrom decision trees to random forests
From decision trees to random forestsViet-Trung TRAN
 
Machine Learning and Data Mining: 15 Data Exploration and Preparation
Machine Learning and Data Mining: 15 Data Exploration and PreparationMachine Learning and Data Mining: 15 Data Exploration and Preparation
Machine Learning and Data Mining: 15 Data Exploration and PreparationPier Luca Lanzi
 
Classification Algorithm.
Classification Algorithm.Classification Algorithm.
Classification Algorithm.Megha Sharma
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality ReductionSaad Elbeleidy
 
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Universitat Politècnica de Catalunya
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringSri Ambati
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Gaurav Mittal
 
Machine Learning With Logistic Regression
Machine Learning With Logistic RegressionMachine Learning With Logistic Regression
Machine Learning With Logistic RegressionKnoldus Inc.
 
k medoid clustering.pptx
k medoid clustering.pptxk medoid clustering.pptx
k medoid clustering.pptxRoshan86572
 
K Nearest Neighbors
K Nearest NeighborsK Nearest Neighbors
K Nearest NeighborsTilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
 
Lecture-12Evaluation Measures-ML.pptx
Lecture-12Evaluation Measures-ML.pptxLecture-12Evaluation Measures-ML.pptx
Lecture-12Evaluation Measures-ML.pptxGauravSonawane51
 
Random forest
Random forestRandom forest
Random forestMusa Hawamdah
 
Temporal difference learning
Temporal difference learningTemporal difference learning
Temporal difference learningJie-Han Chen
 
K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )Mohammad Junaid Khan
 
DoWhy Python library for causal inference: An End-to-End tool
DoWhy Python library for causal inference: An End-to-End toolDoWhy Python library for causal inference: An End-to-End tool
DoWhy Python library for causal inference: An End-to-End toolAmit Sharma
 
Shap
ShapShap
ShapGiovanni Bruner
 
BigML Education - Evaluations
BigML Education - EvaluationsBigML Education - Evaluations
BigML Education - EvaluationsBigML, Inc
 
CreditRisk+ Model Tutorial 3.0
CreditRisk+ Model Tutorial 3.0CreditRisk+ Model Tutorial 3.0
CreditRisk+ Model Tutorial 3.0mrmelchi
 

Weitere ähnliche Inhalte

Was ist angesagt?

Module 4: Model Selection and Evaluation
Module 4: Model Selection and EvaluationModule 4: Model Selection and Evaluation
Module 4: Model Selection and EvaluationSara Hooker
 
Bias and variance trade off
Bias and variance trade offBias and variance trade off
Bias and variance trade offVARUN KUMAR
 
Bayesian statistics
Bayesian statisticsBayesian statistics
Bayesian statisticsSagar Kamble
 
Introduction to XGBoost
Introduction to XGBoostIntroduction to XGBoost
Introduction to XGBoostJoonyoung Yi
 
From decision trees to random forests
From decision trees to random forestsFrom decision trees to random forests
From decision trees to random forestsViet-Trung TRAN
 
Machine Learning and Data Mining: 15 Data Exploration and Preparation
Machine Learning and Data Mining: 15 Data Exploration and PreparationMachine Learning and Data Mining: 15 Data Exploration and Preparation
Machine Learning and Data Mining: 15 Data Exploration and PreparationPier Luca Lanzi
 
Classification Algorithm.
Classification Algorithm.Classification Algorithm.
Classification Algorithm.Megha Sharma
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality ReductionSaad Elbeleidy
 
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Universitat Politècnica de Catalunya
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringSri Ambati
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Gaurav Mittal
 
Machine Learning With Logistic Regression
Machine Learning With Logistic RegressionMachine Learning With Logistic Regression
Machine Learning With Logistic RegressionKnoldus Inc.
 
k medoid clustering.pptx
k medoid clustering.pptxk medoid clustering.pptx
k medoid clustering.pptxRoshan86572
 
Lecture-12Evaluation Measures-ML.pptx
Lecture-12Evaluation Measures-ML.pptxLecture-12Evaluation Measures-ML.pptx
Lecture-12Evaluation Measures-ML.pptxGauravSonawane51
 
Temporal difference learning
Temporal difference learningTemporal difference learning
Temporal difference learningJie-Han Chen
 
DoWhy Python library for causal inference: An End-to-End tool
DoWhy Python library for causal inference: An End-to-End toolDoWhy Python library for causal inference: An End-to-End tool
DoWhy Python library for causal inference: An End-to-End toolAmit Sharma
 

Was ist angesagt? (20)

Module 4: Model Selection and Evaluation
Module 4: Model Selection and EvaluationModule 4: Model Selection and Evaluation
Module 4: Model Selection and Evaluation
 
Bias and variance trade off
Bias and variance trade offBias and variance trade off
Bias and variance trade off
 
Bayesian statistics
Bayesian statisticsBayesian statistics
Bayesian statistics
 
Introduction to XGBoost
Introduction to XGBoostIntroduction to XGBoost
Introduction to XGBoost
 
From decision trees to random forests
From decision trees to random forestsFrom decision trees to random forests
From decision trees to random forests
 
Machine Learning and Data Mining: 15 Data Exploration and Preparation
Machine Learning and Data Mining: 15 Data Exploration and PreparationMachine Learning and Data Mining: 15 Data Exploration and Preparation
Machine Learning and Data Mining: 15 Data Exploration and Preparation
 
Classification Algorithm.
Classification Algorithm.Classification Algorithm.
Classification Algorithm.
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
 
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature Engineering
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 
Machine Learning With Logistic Regression
Machine Learning With Logistic RegressionMachine Learning With Logistic Regression
Machine Learning With Logistic Regression
 
k medoid clustering.pptx
k medoid clustering.pptxk medoid clustering.pptx
k medoid clustering.pptx
 
K Nearest Neighbors
K Nearest NeighborsK Nearest Neighbors
K Nearest Neighbors
 
Lecture-12Evaluation Measures-ML.pptx
Lecture-12Evaluation Measures-ML.pptxLecture-12Evaluation Measures-ML.pptx
Lecture-12Evaluation Measures-ML.pptx
 
Random forest
Random forestRandom forest
Random forest
 
Temporal difference learning
Temporal difference learningTemporal difference learning
Temporal difference learning
 
K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )
 
DoWhy Python library for causal inference: An End-to-End tool
DoWhy Python library for causal inference: An End-to-End toolDoWhy Python library for causal inference: An End-to-End tool
DoWhy Python library for causal inference: An End-to-End tool
 
Shap
ShapShap
Shap
 

Ähnlich wie Scoring Metrics for Classification Models

BigML Education - Evaluations
BigML Education - EvaluationsBigML Education - Evaluations
BigML Education - EvaluationsBigML, Inc
 
CreditRisk+ Model Tutorial 3.0
CreditRisk+ Model Tutorial 3.0CreditRisk+ Model Tutorial 3.0
CreditRisk+ Model Tutorial 3.0mrmelchi
 
Price workshop day one
Price workshop day one Price workshop day one
Price workshop day one DayOne
 
Insurance Ratemaking and premium data analysis
Insurance Ratemaking and premium data analysis Insurance Ratemaking and premium data analysis
Insurance Ratemaking and premium data analysis Mohsen Gharakhani
 
Automobile Insurance Claim Fraud Detection using Random Forest and ADASYN
Automobile Insurance Claim Fraud Detection using Random Forest and ADASYNAutomobile Insurance Claim Fraud Detection using Random Forest and ADASYN
Automobile Insurance Claim Fraud Detection using Random Forest and ADASYNIRJET Journal
 
Using Minitab for Superior Quality in Medical Device Manufacturing
Using Minitab for Superior Quality in Medical Device ManufacturingUsing Minitab for Superior Quality in Medical Device Manufacturing
Using Minitab for Superior Quality in Medical Device ManufacturingMinitab, LLC
 
Xavier Conort, DataScience SG Meetup - Challenges in insurance pricing
Xavier Conort, DataScience SG Meetup - Challenges in insurance pricingXavier Conort, DataScience SG Meetup - Challenges in insurance pricing
Xavier Conort, DataScience SG Meetup - Challenges in insurance pricingKai Xin Thia
 
Rapid Optimization Application Development Using Excel and Solver
Rapid Optimization Application Development Using Excel and SolverRapid Optimization Application Development Using Excel and Solver
Rapid Optimization Application Development Using Excel and SolverMichael Mina
 
Whiz calculator mcs ch.4_case
Whiz calculator mcs ch.4_caseWhiz calculator mcs ch.4_case
Whiz calculator mcs ch.4_caseCitra Dewi
 
Bank Customer Segmentation & Insurance Claim Prediction
Bank Customer Segmentation & Insurance Claim PredictionBank Customer Segmentation & Insurance Claim Prediction
Bank Customer Segmentation & Insurance Claim PredictionIRJET Journal
 
Workers Compensation Program Overview
Workers Compensation Program OverviewWorkers Compensation Program Overview
Workers Compensation Program OverviewLuke Slabaugh
 
Causality without headaches
Causality without headachesCausality without headaches
Causality without headachesBenoît Rostykus
 
ERM Risk Regime and IFRS17
ERM Risk Regime and IFRS17ERM Risk Regime and IFRS17
ERM Risk Regime and IFRS17Syed Danish Ali
 
AI Class Topic 2: Step-by-step Process for AI development
AI Class Topic 2: Step-by-step Process for AI developmentAI Class Topic 2: Step-by-step Process for AI development
AI Class Topic 2: Step-by-step Process for AI developmentValue Amplify Consulting
 
Satisfaction and loyalty
Satisfaction and loyaltySatisfaction and loyalty
Satisfaction and loyaltyTheDataNation
 
Section 79 powerpoint v 97
Section 79 powerpoint v 97Section 79 powerpoint v 97
Section 79 powerpoint v 97allancorby
 
chapter 1. The Fundamentals of Managerial Economics
chapter 1. The Fundamentals of Managerial Economicschapter 1. The Fundamentals of Managerial Economics
chapter 1. The Fundamentals of Managerial EconomicsKasimSadibek1
 
Automobile Insurance Claim Fraud Detection
Automobile Insurance Claim Fraud DetectionAutomobile Insurance Claim Fraud Detection
Automobile Insurance Claim Fraud DetectionIRJET Journal
 
2022 QPP MIPS Final Rule
2022 QPP MIPS Final Rule2022 QPP MIPS Final Rule
2022 QPP MIPS Final RuleMedisolv, Inc.
 

Ähnlich wie Scoring Metrics for Classification Models (20)

BigML Education - Evaluations
BigML Education - EvaluationsBigML Education - Evaluations
BigML Education - Evaluations
 
CreditRisk+ Model Tutorial 3.0
CreditRisk+ Model Tutorial 3.0CreditRisk+ Model Tutorial 3.0
CreditRisk+ Model Tutorial 3.0
 
Price workshop day one
Price workshop day one Price workshop day one
Price workshop day one
 
Insurance Ratemaking and premium data analysis
Insurance Ratemaking and premium data analysis Insurance Ratemaking and premium data analysis
Insurance Ratemaking and premium data analysis
 
Automobile Insurance Claim Fraud Detection using Random Forest and ADASYN
Automobile Insurance Claim Fraud Detection using Random Forest and ADASYNAutomobile Insurance Claim Fraud Detection using Random Forest and ADASYN
Automobile Insurance Claim Fraud Detection using Random Forest and ADASYN
 
Using Minitab for Superior Quality in Medical Device Manufacturing
Using Minitab for Superior Quality in Medical Device ManufacturingUsing Minitab for Superior Quality in Medical Device Manufacturing
Using Minitab for Superior Quality in Medical Device Manufacturing
 
Xavier Conort, DataScience SG Meetup - Challenges in insurance pricing
Xavier Conort, DataScience SG Meetup - Challenges in insurance pricingXavier Conort, DataScience SG Meetup - Challenges in insurance pricing
Xavier Conort, DataScience SG Meetup - Challenges in insurance pricing
 
Rapid Optimization Application Development Using Excel and Solver
Rapid Optimization Application Development Using Excel and SolverRapid Optimization Application Development Using Excel and Solver
Rapid Optimization Application Development Using Excel and Solver
 
Whiz calculator mcs ch.4_case
Whiz calculator mcs ch.4_caseWhiz calculator mcs ch.4_case
Whiz calculator mcs ch.4_case
 
Bank Customer Segmentation & Insurance Claim Prediction
Bank Customer Segmentation & Insurance Claim PredictionBank Customer Segmentation & Insurance Claim Prediction
Bank Customer Segmentation & Insurance Claim Prediction
 
Workers Compensation Program Overview
Workers Compensation Program OverviewWorkers Compensation Program Overview
Workers Compensation Program Overview
 
Causality without headaches
Causality without headachesCausality without headaches
Causality without headaches
 
ERM Risk Regime and IFRS17
ERM Risk Regime and IFRS17ERM Risk Regime and IFRS17
ERM Risk Regime and IFRS17
 
Hazard identification and control
Hazard identification and controlHazard identification and control
Hazard identification and control
 
AI Class Topic 2: Step-by-step Process for AI development
AI Class Topic 2: Step-by-step Process for AI developmentAI Class Topic 2: Step-by-step Process for AI development
AI Class Topic 2: Step-by-step Process for AI development
 
Satisfaction and loyalty
Satisfaction and loyaltySatisfaction and loyalty
Satisfaction and loyalty
 
Section 79 powerpoint v 97
Section 79 powerpoint v 97Section 79 powerpoint v 97
Section 79 powerpoint v 97
 
chapter 1. The Fundamentals of Managerial Economics
chapter 1. The Fundamentals of Managerial Economicschapter 1. The Fundamentals of Managerial Economics
chapter 1. The Fundamentals of Managerial Economics
 
Automobile Insurance Claim Fraud Detection
Automobile Insurance Claim Fraud DetectionAutomobile Insurance Claim Fraud Detection
Automobile Insurance Claim Fraud Detection
 
2022 QPP MIPS Final Rule
2022 QPP MIPS Final Rule2022 QPP MIPS Final Rule
2022 QPP MIPS Final Rule
 

Mehr von KNIMESlides

What's New in KNIME Analytics Platform 4.1
What's New in KNIME Analytics Platform 4.1What's New in KNIME Analytics Platform 4.1
What's New in KNIME Analytics Platform 4.1KNIMESlides
 
Codeless Deep Learning for Language Modeling and Image Classification
Codeless Deep Learning for Language Modeling and Image ClassificationCodeless Deep Learning for Language Modeling and Image Classification
Codeless Deep Learning for Language Modeling and Image ClassificationKNIMESlides
 
Automating Inferences out of Financial Data
Automating Inferences out of Financial DataAutomating Inferences out of Financial Data
Automating Inferences out of Financial DataKNIMESlides
 
Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020
Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020
Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020KNIMESlides
 
Credit Card Fraud Detection Tutorial
Credit Card Fraud Detection TutorialCredit Card Fraud Detection Tutorial
Credit Card Fraud Detection TutorialKNIMESlides
 
Practicing Data Science: A Collection of Case Studies
Practicing Data Science: A Collection of Case StudiesPracticing Data Science: A Collection of Case Studies
Practicing Data Science: A Collection of Case StudiesKNIMESlides
 
What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9
What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9
What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9KNIMESlides
 
Webinar: Behind the Scenes on Guided Analytics
Webinar: Behind the Scenes on Guided AnalyticsWebinar: Behind the Scenes on Guided Analytics
Webinar: Behind the Scenes on Guided AnalyticsKNIMESlides
 
KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019
KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019
KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019KNIMESlides
 
Open Source Story and what’s new in KNIME Software
Open Source Story and what’s new in KNIME SoftwareOpen Source Story and what’s new in KNIME Software
Open Source Story and what’s new in KNIME SoftwareKNIMESlides
 
Anomaly Detection - Discover unknown Frauds and Anomalies using Machine Learning
Anomaly Detection - Discover unknown Frauds and Anomalies using Machine LearningAnomaly Detection - Discover unknown Frauds and Anomalies using Machine Learning
Anomaly Detection - Discover unknown Frauds and Anomalies using Machine LearningKNIMESlides
 
Sharing and Deploying Data Science with KNIME Server
Sharing and Deploying Data Science with KNIME ServerSharing and Deploying Data Science with KNIME Server
Sharing and Deploying Data Science with KNIME ServerKNIMESlides
 
Guided Automation- A Blueprint for Interactive Automated Machine Learning
Guided Automation- A Blueprint for Interactive Automated Machine LearningGuided Automation- A Blueprint for Interactive Automated Machine Learning
Guided Automation- A Blueprint for Interactive Automated Machine LearningKNIMESlides
 
KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...
KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...
KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...KNIMESlides
 
Sentiment Analysis with KNIME Analytics Platform
Sentiment Analysis with KNIME Analytics PlatformSentiment Analysis with KNIME Analytics Platform
Sentiment Analysis with KNIME Analytics PlatformKNIMESlides
 
Chemistry Data Basics with KNIME Analytics Platform
Chemistry Data Basics with KNIME Analytics PlatformChemistry Data Basics with KNIME Analytics Platform
Chemistry Data Basics with KNIME Analytics PlatformKNIMESlides
 
Sentiment Analysis with Deep Learning, Machine Learning or Lexicon based
Sentiment Analysis with Deep Learning, Machine Learning or Lexicon basedSentiment Analysis with Deep Learning, Machine Learning or Lexicon based
Sentiment Analysis with Deep Learning, Machine Learning or Lexicon basedKNIMESlides
 
KNIME Data Science Learnathon: From Raw Data To Deployment
KNIME Data Science Learnathon: From Raw Data To DeploymentKNIME Data Science Learnathon: From Raw Data To Deployment
KNIME Data Science Learnathon: From Raw Data To DeploymentKNIMESlides
 
KNIME Software Overview
KNIME Software OverviewKNIME Software Overview
KNIME Software OverviewKNIMESlides
 
From Raw Data to Deployment
From Raw Data to DeploymentFrom Raw Data to Deployment
From Raw Data to DeploymentKNIMESlides
 

Mehr von KNIMESlides (20)

What's New in KNIME Analytics Platform 4.1
What's New in KNIME Analytics Platform 4.1What's New in KNIME Analytics Platform 4.1
What's New in KNIME Analytics Platform 4.1
 
Codeless Deep Learning for Language Modeling and Image Classification
Codeless Deep Learning for Language Modeling and Image ClassificationCodeless Deep Learning for Language Modeling and Image Classification
Codeless Deep Learning for Language Modeling and Image Classification
 
Automating Inferences out of Financial Data
Automating Inferences out of Financial DataAutomating Inferences out of Financial Data
Automating Inferences out of Financial Data
 
Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020
Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020
Credit Card Fraud Detection Tutorial - KNIME Meetup Berlin 2020
 
Credit Card Fraud Detection Tutorial
Credit Card Fraud Detection TutorialCredit Card Fraud Detection Tutorial
Credit Card Fraud Detection Tutorial
 
Practicing Data Science: A Collection of Case Studies
Practicing Data Science: A Collection of Case StudiesPracticing Data Science: A Collection of Case Studies
Practicing Data Science: A Collection of Case Studies
 
What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9
What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9
What's New in KNIME Analytics Platform 4.0 and KNIME Server 4.9
 
Webinar: Behind the Scenes on Guided Analytics
Webinar: Behind the Scenes on Guided AnalyticsWebinar: Behind the Scenes on Guided Analytics
Webinar: Behind the Scenes on Guided Analytics
 
KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019
KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019
KNIME Data Science Learnathon: From Raw Data To Deployment - Dublin - June 2019
 
Open Source Story and what’s new in KNIME Software
Open Source Story and what’s new in KNIME SoftwareOpen Source Story and what’s new in KNIME Software
Open Source Story and what’s new in KNIME Software
 
Anomaly Detection - Discover unknown Frauds and Anomalies using Machine Learning
Anomaly Detection - Discover unknown Frauds and Anomalies using Machine LearningAnomaly Detection - Discover unknown Frauds and Anomalies using Machine Learning
Anomaly Detection - Discover unknown Frauds and Anomalies using Machine Learning
 
Sharing and Deploying Data Science with KNIME Server
Sharing and Deploying Data Science with KNIME ServerSharing and Deploying Data Science with KNIME Server
Sharing and Deploying Data Science with KNIME Server
 
Guided Automation- A Blueprint for Interactive Automated Machine Learning
Guided Automation- A Blueprint for Interactive Automated Machine LearningGuided Automation- A Blueprint for Interactive Automated Machine Learning
Guided Automation- A Blueprint for Interactive Automated Machine Learning
 
KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...
KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...
KNIME Data Science Learnathon: From Raw Data To Deployment - Paris - November...
 
Sentiment Analysis with KNIME Analytics Platform
Sentiment Analysis with KNIME Analytics PlatformSentiment Analysis with KNIME Analytics Platform
Sentiment Analysis with KNIME Analytics Platform
 
Chemistry Data Basics with KNIME Analytics Platform
Chemistry Data Basics with KNIME Analytics PlatformChemistry Data Basics with KNIME Analytics Platform
Chemistry Data Basics with KNIME Analytics Platform
 
Sentiment Analysis with Deep Learning, Machine Learning or Lexicon based
Sentiment Analysis with Deep Learning, Machine Learning or Lexicon basedSentiment Analysis with Deep Learning, Machine Learning or Lexicon based
Sentiment Analysis with Deep Learning, Machine Learning or Lexicon based
 
KNIME Data Science Learnathon: From Raw Data To Deployment
KNIME Data Science Learnathon: From Raw Data To DeploymentKNIME Data Science Learnathon: From Raw Data To Deployment
KNIME Data Science Learnathon: From Raw Data To Deployment
 
KNIME Software Overview
KNIME Software OverviewKNIME Software Overview
KNIME Software Overview
 
From Raw Data to Deployment
From Raw Data to DeploymentFrom Raw Data to Deployment
From Raw Data to Deployment
 

Kürzlich hochgeladen

unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embeddingZilliz
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 

Kürzlich hochgeladen (20)

unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 

Scoring Metrics for Classification Models

  • 1. © 2019 KNIME AG. All rights reserved. Scoring Metrics for Classification Models KNIME: Maarit.Widmann@knime.com @KNIME What to do after training a machine learning algorithm
  • 2. © 2019 KNIME AG. All rights reserved. Introduction There are many scoring metrics for a classification model. Which of them fits depends on your classification problem. 2
  • 3. © 2019 KNIME AG. All rights reserved. Different Scoring Metrics 3 1. Confusion Matrix • True positives • False negatives • False positives • True negatives 2. Sensitivity and Specificity 3. Precision and Recall 4. F-measure 5. Overall Accuracy and Cohen‘s kappa
  • 4. © 2019 KNIME AG. All rights reserved. Why different scoring metrics? 4 1. What is your objective? 2. What is the target class distribution? 3. Is the target binomial or multinomial?
  • 5. © 2019 KNIME AG. All rights reserved. Introduction Doctor‘s diagnosis as an example of classification 5 Sample of patients Disease carrier Healthy Diagnosis Classification results Positive class Negative class
  • 6. © 2019 KNIME AG. All rights reserved. Confusion Matrix 6
  • 7. © 2019 KNIME AG. All rights reserved. Introduction Disease detection using a machine learning algorithm Sample of patients split into training (80 %) and test (20 %) set Model training and prediction Evaluation of classification results
  • 8. © 2019 KNIME AG. All rights reserved. Scoring Metrics for Classification Models 9
  • 9. © 2019 KNIME AG. All rights reserved. Sensitivity • Sensitivity= 𝑻𝑷 𝑻𝑷+𝑭𝑵 = 𝟑 𝟑+𝟏 = 𝟎. 𝟕𝟓 Are ALL positive class events found by the model?
  • 10. © 2019 KNIME AG. All rights reserved. Specificity • Specificity= 𝑻𝑵 𝑻𝑵+𝑭𝑷 = 𝟐𝟎 𝟐𝟎+𝟒 = 𝟎. 𝟖𝟑 Are ALL negative class events found by the model? Sensitivity: Is the model sensitive to detecting disease? Specificity: Is the disease diagnosis specific?
  • 11. © 2019 KNIME AG. All rights reserved. Recall • Recall= 𝑻𝑷 𝑻𝑷+𝑭𝑵 = 𝟑 𝟑+𝟏 = 𝟎. 𝟕𝟓 Are ALL positive class events found by the model?
  • 12. © 2019 KNIME AG. All rights reserved. Precision • Precision= 𝑻𝑷 𝑻𝑷+𝑭𝑷 = 𝟑 𝟑+𝟒 = 𝟎. 𝟒𝟑 Are ONLY positive class events found by the model? Recall: Detect the most disease carriers Precision: Make precise disease prediction
  • 13. © 2019 KNIME AG. All rights reserved. Defining the Classification Threshold Machine learning model predicts each patient Score(Diagnosis=disease carrier). The class assignment is based on the set threshold for this score. True positives False positives False negatives True negatives Score(Diagnosis= disease carrier)0 0.5 1 Recall ↓↑ Precision
  • 14. © 2019 KNIME AG. All rights reserved. F-measure • F-measure= 2 ∗ 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛∗𝑟𝑒𝑐𝑎𝑙𝑙 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛+𝑟𝑒𝑐𝑎𝑙𝑙 = 2 ∗ 0.43∗0.75 0.43+0.75 ≈ 0.55 Harmonic mean of precision and recall. Are ALL and ONLY positive class events found by the model?
  • 15. © 2019 KNIME AG. All rights reserved. Overall Accuracy • Overall Accuracy= 𝑻𝑷+𝑻𝑵 𝑻𝑷+𝑭𝑷+𝑭𝑵+𝑻𝑵 = 𝟑+𝟐𝟎 𝟑+𝟏+𝟒+𝟐𝟎 = 𝟎. 𝟖𝟐 Target class distribution must be balanced! Probability of classifying a positive OR negative class event correctly.
  • 16. © 2019 KNIME AG. All rights reserved. Cohen‘s kappa (𝜿) • 𝜅 = 𝑝0−𝑝 𝑒 1−𝑝 𝑒 , where 𝑝0 is the overall accuracy by the model 𝑝 𝑒 = 𝑝 𝑒1 + 𝑝 𝑒2 𝑝 𝑒1 = 𝑝 𝑝𝑟𝑒𝑑="𝑑𝑖𝑠𝑒𝑎𝑠𝑒 𝑐𝑎𝑟𝑟𝑖𝑒𝑟" × 𝑝 𝑎𝑐𝑡="𝑑𝑖𝑠𝑒𝑎𝑠𝑒 𝑐𝑎𝑟𝑟𝑖𝑒𝑟" 𝑝 𝑒2 = 𝑝 𝑝𝑟𝑒𝑑="ℎ𝑒𝑎𝑙𝑡ℎ𝑦" × 𝑝 𝑎𝑐𝑡="ℎ𝑒𝑎𝑙𝑡ℎ𝑦" The overall accuracy using a random classifier.
  • 17. © 2019 KNIME AG. All rights reserved. 𝑝 𝑒1 = 7 28 × 4 28 𝑝 𝑒2 = 21 28 × 24 28 𝑝 𝑒 = 𝑝 𝑒1 + 𝑝 𝑒2 = 0.68 𝑝0 = 23 28 = 0.82 𝜅 = 𝑝0−𝑝 𝑒 1−𝑝 𝑒 = 0.14 0.32 ≈ 0.44 Cohen‘s kappa (𝜿) vs. Overall accuracy Overall accuracy 𝑝 𝑒1 = 5 28 × 4 28 𝑝 𝑒2 = 23 28 × 24 28 𝑝 𝑒 = 𝑝 𝑒1 + 𝑝 𝑒2 = 0.73 𝑝0 = 21 28 = 0.75 𝜅 = 𝑝0−𝑝 𝑒 1−𝑝 𝑒 = 0.02 0.27 = 0.07 New model 𝜅 = 1: perfect model performance 𝜅 = 0: performance of a random classifier
  • 18. © 2019 KNIME AG. All rights reserved. Scoring Metrics for a Multivariate Classification Model 22 Sample of patients Disease carrier Healthy Diagnosis Classification results Recessive disease carrier Positive class Negative class
  • 19. © 2019 KNIME AG. All rights reserved. Confusion Matrix 23 True positives False positives False negatives True negatives
  • 20. © 2019 KNIME AG. All rights reserved. Classification Model Evaluation in KNIME 26
  • 21. © 2019 KNIME AG. All rights reserved. Scorer (JavaScript) node 27
  • 22. © 2019 KNIME AG. All rights reserved. Interactive View: Confusion Matrix 28
  • 23. © 2019 KNIME AG. All rights reserved. Interactive View: Confusion Matrix 29
  • 24. © 2019 KNIME AG. All rights reserved. Workflow for Classification 30 • On KNIME Workflow Hub: Evaluating Classification Model Performance • On EXAMPLES Server: EXAMPLES/04_Analytics/10_Scoring/01_Eval uating_Classification_Model_Performance
  • 25. © 2019 KNIME AG. All rights reserved. Summary • After training a classification model, the model performance is reported using scoring metrics • Scoring metrics describe and compare the model performance • Confusion matrix shows the numbers of correct and incorrect predictions • Class statistics and overall accuracy statistics are based on the values in the confusion matrix 32
  • 26. © 2019 KNIME AG. All rights reserved. KNIME Fall Summit 2019 November 5 – 8 at AT&T Executive Education and Conference Center, Austin, Texas • Tuesday & Wednesday: One-day courses • Thursday & Friday: Summit sessions Register by October 1 for Early Bird Discount! Register at knime.com/summits
  • 27. © 2019 KNIME AG. All rights reserved. KNIME Beginner’s Luck Course Book downloadable from KNIME Press https://www.knime.com/knimepress with code: SCORING-METRICS-0519
  • 28. © 2019 KNIME AG. All rights reserved. 36 The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME.com AG under license from KNIME GmbH, and are registered in the United States. KNIME® is also registered in Germany. Thank You!