The tao of knowledge, revisited

Valentina Tamma
University of Liverpool
The tao of knowledge, revisited: 
the journey vs the goal
Picture by J. A. Alba, on Pixabay.com

V.TammaISWC 2019 Doctoral Consortium
Pearls of wisdom: PhD Comics
2
Piled Higher and Deeper by Jorge Cham www.phdcomics.com
title: "Frozen" - originally published 5/7/2014

This talk
3
Disney
Disney

This talk
4
Disney
Disney

An apprenticeship in knowledge creation
• A PhD is an (individual) research
project involving advanced
scholarship, that makes an original
contribution to knowledge
• PhD:
• Philosophiae: from the Greek, meaning “love of knowledge”,
“pursuit of wisdom”, “systematic investigation”
• Doctor: from the classical Latin “Teacher” (to show, teach,
cause to know)
5
Picture by fancycrave, on Pixabay.com

Scientific method
“Research is systematic investigation to establish the facts.”
6
Creswell, J. W.: Educational Research, 2008  
• A research methodology defines what the activity of
research is, how to proceed, how to measure progress,
and what constitutes success.
• but the methodology depends on the scientific field of enquiry;
• Formal sciences: mathematics, logic, statistics, …
• Natural sciences: chemistry, biology, physics, …
• Social Sciences: psychology, linguistics, anthropology, …

• Some argue that CS, and its
disciplines are not a science
• “CS it is the science of information processes
and their interactions with the world.” (P. Denning
2005).
7
CS
AI
SW
Where is the science in CS?

• Some argue that CS, and its
disciplines are not a science
• “But the claim that artificial objects […] do not
lend themselves to natural-science methods of
research is fallacious. An artificial object is as fully
bound by the laws of nature as any natural
object. […] Scientific laws limit the set of possible
objects, natural or artificial.” (H. Simon 1993)
8
CS
AI
Is this science?
CS
AI
SW
8

What makes the SW?
9
Semantic Web

V.TammaISWC 2019 Doctoral Consortium 10
Adapted from G Dogdig-Crnkovic
State research
problem
Review existing
theories and
observations
Formulate
hypothesis
Deduce
consequence
and make
predictions
Evaluate the
hypothesis
Hypothesis must be redefined
Theory
confirmed or
proposed
Consistency
achieved
Selection
amongst
competing
theories
Hypothesis must be adjusted
Adapted by G. Dogdig-Crnkovic
Scientific
method - the
process

The research problem
• The objective of the investigation.
• It identifies a problem / difficulty that needs solving;
• There must be some value attached to it and the
beneficiaries could be clearly identified;
• There might be alternative means to reach the
objectives;
• It should be feasible, not too generic or narrow
focussed;
• It should have some level of novelty;
• Can often be subdivided in and bounded by a
number of sub-questions.
11
Picture by G. Altmann, on Pixabay.com

In defence of good hypotheses
• Stating your hypotheses clearly is half
of the job done:
• Often there are many hypotheses
• that might be decomposed in a set of subsidiary hypotheses;
• Ambiguous hypotheses cause major
misunderstandings in the reader (reviewer!)
• Vague hypotheses lead to poor methodological
consequences:
• Inconclusive evidence;
• Research direction lacks focus.
12
Picture by qimono, on Pixabay.com

The hypothesis
• A conjectural answer to a
research question
• it is framed and scoped within the
context of existing knowledge;
• it is clearly formulated, with a
measurable / veriﬁable objective;
• should be refutable
• Popper's test for what constitutes science
13
Which one?
1. Our alignment approach is better than
the ones presented in the state of the
art.
2. Our alignment approach improves
precision wrt current systems on X
ontologies in the benchmark / on
ontologies with expressivity of type Y.
3. Our alignment approach significantly
increases both precision and recall
wrt all of the systems included in the
evaluation challenge, for the track Z.

The hypothesis
• A conjectural answer to a
research question
• it is driven by a scientiﬁc problem;
• it states the why, how and possibly
the who;
• it is framed by a task;
• which is particularly important when some
artefact is produced as part of the research;
14
Which one?
1. We designed an ontology that
effectively models domain X.
2. We designed an ontology that
models task Y in domain X, and
aims to answer the following
competency questions.
3. We developed a new ontology editor
and users who attended the tutorial
like it;
4. Our ontology editor facilitates the
editing of large ontologies by
domain experts;

Dimensions of investigation
• Properties can be investigated across different levels:
• properties of a technique vs those of its parameters,
• inherent properties of a task vs a complete system;
• relationship between tasks, parameters, systems;
• Different dimensions for the comparison:
• scientiﬁc;
• engineering;
• cognitive science;
• Different means of investigation:
• Theoretical;
• Experimental;
15

Scientific Engineering Cognitive
Behaviour: The effect or result of the
method. The absolute or comparative
assessment wrt an external "gold standard"
Dependability: The reliability, security and
safeness of the system implementing the
method
External: The model exhibits the
appropriate external behaviours (similar to
scientific behaviour, but the baseline is
different)
Coverage: The range of application of the
method. It identifies a set of situations to
which its application is relevant
Usability: The ease of use of the system
from the perspective of the end user
Internal: The model works in the same
way as the phenomenon / observation that
it models
Efficiency: The resources consumed by
the method. The resources measured are
usually time or space.
Maintainability: The ability of the system
to evolve in order to meet changes in the
user's requirements.
Adaptability: The model accounts for a
wide range of occurring behaviours
Scalability: The potential for the system to
continue to work within realistic resource
limits on the most complex examples
Evolvability: The model truthfully
represents the evolution or learning of the
ability it models
Cost: Resources (developer time,
money…) needed to build and/or maintain
the system
Fitness: The extent to which the system
adheres to the user’s requirements
Adapted from A. Bundy: The need for hypotheses in informatics
Dimensions of
investigation

Predictions
• From hypotheses we can derive
predictions:
• Hypothesis: theory explaining why a
phenomenon occurs
• testable hypotheses
• Prediction: using the hypothesis, scientists
calculate the measurable data points they
believe will result in a given experiment
• often involves different properties of the model being
developed
17

Evaluation
• Demonstrate / show that the
hypothesis holds:
• Theoretical proof
• checks properties such as correctness, completeness,
termination and complexity
• Experimental evaluation:
• Quantitative research method
• Qualitative research method
• Can sometimes be used together to evaluate
different aspects of a proposed method
18

Inclusive models of scientific research
• Scientific research is concerned with
stating knowledge claims
• These claims need to be evaluated and
validated in some way
• Depending on the specific area of interest
(and on the aspect we are evaluating) given
evaluation methods are employed, e.g.:
• Case study, Experiment, Survey, Proof
19
Semantic Web

Qualitative and quantitative methods
• Quantitative:
• Methods associated with
measurements (on numeric
scales)
• Prevalent in natural sciences
• Used to test hypotheses or create
a set of observations for inductive
reasoning
• Accuracy and repeatability are
imperative
• Qualitative
• Methods involving case studies
and surveys
• Prevalent in social sciences
• Used to generate comprehensive
description of processes,
mechanisms, or settings
• Characterise participant
perspectives and experiences
20

Qualitative studies
• Tell the reader about the design being used
• the use of qualitative research and the intent behind it
• But also involves discussing:
• the sample for the study,
• the data collection process,
• the recording procedure;
• Allows inductive and deductive data analysis;
21

Mixing the two
22
Approach: Inductive
Goal: Depth, local meaning,
generate hypotheses
Setting: Natural
Sampling: Purposeful
Data: Words, Images; Narrow but rich
Data analysis: Iterative interpretation
Values: Personal involvement and partiality  
(subjectivity, reﬂexivity)
Approach: Deductive
Goal: Breadth, generalisation,
test hypotheses
Setting: Experimental
Sampling: Probabilistic
Data: Numbers; Shallow but broad
Data analysis: Statistical tests, models
Values: Detachment and impartiality  
(objectivity)
MIXED
Adapted from B. Young and D. Hren: Introduction to qualitative research methods
Qualitatitve Quantitatitve

ISWC 2017, Resources track
23
Criteria How to
Is the ontology logically correct? Proofs (reasoners integrated in ontology editors), quantitative,
Is the chosen design suitable for the intended
purpose?   Qualitative
Is the chosen design of high quality? (e.g., no
hacks and workarounds, no redundancy)
Quantitative
Have other resources been reused? E.g., upper
level ontologies, design patterns
Qualitative (reﬂective)
Is the documentation of good quality? Are the
core ideas of the ontology described?
Quantitative & Qualitative

Word of caution
• Designing qualitative experiments requires
care:
• resource intensive
• signiﬁcant sample of domain experts / users
• to support stratiﬁcation
• control groups
• ordering of questions (how) and groups answering
questions (who) is important
• types and wording of questions, and question format
• support for different types of research (basic, applied
and summative)
• C. Pesquita, V. Ivanova, S. Lohmann and P. Lambrix : A Framework to
Conduct and Report on Empirical User Studies in Semantic Web. in
EKAW 2018
24
Picture by Clker-Free-Vector-Images, on Pixabay.com

Adapted from G Dogdig-Crnkovic
State research
problem
Review existing
theories and
observations
Formulate
hypothesis
Deduce
consequence
and make
predictions
Evaluate the
hypothesis
Hypothesis must be redefined
Theory
confirmed or
proposed
Consistency
achieved
Selection
amongst
competing
theories
Hypothesis must be adjusted
Adapted by G. Dogdig-Crnkovic
Scientific
method - the
process

When art blends with science
• A rigorous approach to research does not
guarantee that we are making science;
• “Science in the making”: The processes by which
scientific facts are proposed, argued, & accepted
(Latour 1987):
• A new model appears as art whilst it is in the making;
• It becomes a “fact” only after it gains consensus.
• It is a messy, political, human process, fraught with emotion and occasional
polemics. (Denning 1995)`
• After sufficient time and validation, a model
becomes part of the scientific body of knowledge.
26
Picture by E. Riva, on Pixabay.com

Making science, communicating science
• Making science has a persuasive element:
• transition from “art” to “fact”
• Often we focus on executing the process, but we neglect the persuasion
• Good science starts with communication
• Training for a PhD should also raise the awareness on communicating science
• With a focus on the construction of the persuasive arguments
• “the formal structures around scientists today require them [the scientists] that they have impact journeys for
their research, from the point of inception” (A. Miah, THE, 2019)
• Using a multitude of media, from social media to rewriting Wikipedia pages or even creating novel platforms
for dissemination
• Taking a stance as public intellectual, increasingly more important in this age of fake and post truth
• e.g. T. Farrell, M. Fernandez, J. Novotny and H. Alani: “Exploring Misogyny Across the Manosphere in Reddit”. Web Sci 2019
27

Some advice
• Read, read, read… and read
• Read anything that captures your imagination
• Read with questions in mind:
• “How can I use this?”
• “Does this really do what the authors claim?”
• “Do I understand the results in the paper?”
• Talk about your research
• To your supervisor(s), to your colleagues, to
students in other departments
• It will help you hone and shape the arguments
28
Picture by Free-Photos, on Pixabay.com

Some advice
• Divide your time between activities
• Proving your hypotheses, writing about your
research, etc
• Document your experiments:
• Make an experimental plan, describe in detail
materials, methods and participants
• There is always light at the end of the
tunnel!
29
Picture by Free-Photos, on Pixabay.com

Conclusion: part 1
• SW ⊑ CS, and so is a science:
• And borrows methodological aspects from other disciplines;
• And all these are needed:
• The synergy gives further strengths and novel insights
• because they complement each other’s limitations.
• We should exploit these synergies in our
research by using the appropriate research
methodologies
• we should become familiar with other research methods and
be prepared to use and adapt evaluation methods from other
disciplines
30
Semantic Web

Conclusion: part 2
• There is value in well designed
qualitative methodologies
• They might be necessary to evaluate some
unique aspects of our research
• And there is an opportunity for
creating new methods by creatively
combining quantitative and qualitative
• whilst striving for the rigour and precision of
these methods
31
Picture by mohamed Hassan , on Pixabay.com

V.TammaEKAW 2018 PhD Symposium
See you at the next…
• Conference…
• ISWC, K-Cap, EKAW, ESWC, WWW, ECAI
IJCAI, AAAI
• … or Journal
• Journal of Web Semantics, Semantic Web
Journal, Transactions on Knowledge and Data
Engineering, Data Semantics, Applied Ontology,
Knowledge Engineering Review, Artiﬁcial
Intelligence Journal, Journal of AI Research
32
Disney

Some useful resources
• H. Akkermans and J. Gordijn1
: Ontology Engineering, Scientific
Method, and the Research Agenda. In Proceedings of EKAW 2006
• A. Bernstein and N. Noy: Is This Really Science? The Semantic
Webber’s Guide to Evaluating Research Contributions. Technical
report:  
https://www.merlin.uzh.ch/publication/show/9417
• A. Bundy: The need for hypotheses in informatics. Technical report: 
http://www.inf.ed.ac.uk/teaching/courses/irm/notes/
hypotheses.html
• D. Chapman (Ed): How to do Research At the MIT AI Lab. Technical
report:  
https://dspace.mit.edu/handle/1721.1/41487
• P.R. Cohen: Empirical methods for Artificial Intelligence. 1995
• J.W. Creswell. Educational Research: Planning, Conducting, and
Evaluating Quantitative and Qualitative Research. 2008
• P. Denning: Is Computer Science Science. Comms of the ACM, Vol.
48, No. 4, 2005
• G. Dodig-Crnkovic: Scientific Methods in Computer Science. In
Proceedings of the Conference for the Promotion of Research in IT
at New Universities and at University Colleges in Sweden, 2002
• T. Farrell, M. Fernandez, J. Novotny and H. Alani: “Exploring
Misogyny Across the Manosphere in Reddit”. Web Sci 2019
• C. M. Judd, E.R. Smith, L.H. Kidder: Research methods in social
relations, 1986
• B. Latour. Science in action. 1987
• A. Miah: Good Science Begins with Communication. Times Higher
Education, July 2019
• C. Pesquita, V. Ivanova, S. Lohmann and P. Lambrix : A Framework
to Conduct and Report on Empirical User Studies in Semantic Web.
in EKAW 2018
• K. Popper: The Logic of Scientific Discovery, 1934
• H. Simon: Artificial Intelligence: An empirical Science. In Artificial
Intelligence Journal, vol 77, issue 1
• V. Tamma and F. Lecue: ISWC 2017 Resources Track: Instructions
for Authors and Reviewers. Technical report: https://goo.gl/426CEv233

The tao of knowledge, revisited

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie The tao of knowledge, revisited

Ähnlich wie The tao of knowledge, revisited (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

The tao of knowledge, revisited