1.0 - Propositions and Inference

1.1 - What is Logic?

• A formal language to represent sets of states.
• A convenient abstraction for dealing with many states
• Regardless of whether there's a natural notion of "near" or not (i.e. not a metric space), we can use logic to group different states together
• For example:
  • I have a laptop $\Rightarrow$ Includes any brand and model
  • There is a laptop on the table $\Rightarrow$ Can be at any position on the table.

1.1.1 - Proposition Definitions

• An interpretation is an assignment of values to variables (candidate solution)
• A model is an interpretation that satisfies the constraints (as in CSPs)
  • Often we don't want to just find a model, but want to know what is true in all models.
  • Discussed in the section on validity.
• A proposition is a statement that is true or false in each interpretation.

1.1.2 - Why Propositions?

• Specifying logical formulae is often more natural than filling in tables (i.e. large truth tables)
• It is easier to check correctness and debug formulae than tables (think of checking linear algebraic expressions versus matrices containing a set of linear equations)
• We can exploit the Boolean nature for (algorithmic) efficient reasoning (pruning, consistency and the resolution rule)

We need a language for asking queries (of what follows in all models) that may be more complicated than asking for the value of a variable.

• You may only have partial knowledge of some constraints
• It is easy to incrementally add formulae
• It can be extended to infinitely many variables with infinite domains (using logical quantification)

1.1.3 - Formal Language

• The formal language representation and reasoning system is made up of:
  • Syntax Specifies the legal sentences
  • Semantics Specifies the meanings of symbols
  • Atoms are logical formulae with no deeper propositional structure (i.e. the primitive components of a formal language)
• These are usually associated with a reasoning theory or proof procedure: a (non-deterministic specification of how an answer can be produced).
• There are many types of logic:
  • Propositional logic (used in this course)
  • Predicate / first order logic
  • High order logic

1.1.4 - Semantics - Human's View of Semantics

Step 1 - Begin with a task domain (e.g. scheduling trains on the network)

Step 2 - Choose atoms in the computer to denote propositions. These atoms have meaning to the knowledge base designer

Step 3 - Tell the system knowledge about the domain

Step 4 - Ask the system questions - Is this statement true at all times?

• The system can tell you whether the question is a logical consequence (of the knowledge base)
• You can interpret the answer with the meaning associated with the atoms.

1.1.5 - Example of Semantics - Electrical Environment

In Computer

• $\text{light1\_broken} \Leftarrow \text{sw\_on} \wedge \text{power} \wedge \text{unlit\_light\_1}$
• $\text{sw\_on}$
• $\text{power}\Leftarrow \text{lit\_light\_2}$
• $\text{unlit\_light\_1}$
• $\text{lit\_light\_2}$

In User's Mind

• $*\text{light\_1\_broken}*$ Light #1 is broken
• $\text{sw\_on}$ Switch is up
• $\text{power}$ There is power in the building
• $\text{unlit\_light\_1}$ Light 1 isn't lit
• $\text{lit\_light\_2}$ Light 2 is lit

Conclusion → $\text{light\_1\_broken}$

• The computer doesn't know the meaning of the symbols
• The user can interpret the symbols using their own meanings.

1.1.6 - Simple Language: Propositional Logic Definitions

• An atom is a symbol (Starting with a lowercase letter)
• A sentence (or body) is an atom or is of the form $s_1\wedge s_2$ where $s_1$ and $s_2$ are sentences.
• A definite clause is an atom or is a rule of the form $h\Leftarrow s$ where $h$ is an atom and $s$ is a sentence
  • $h \Leftarrow s: \text{h is implied by s}$
• A knowledge base is a set of definite clauses.

1.1.7 - RiPPLE Activity

• An atomic proposition / atom is a symbol.
• We use the convention that propositions consist of letters, digits, and underscores (_) (and start with a lowercase character)
• Atoms are known to be either true or false

Are these propositions?

1. What is the distance between Mars and Earth? Can't be evaluated to either True or False.
2. $x+2=2x$ Not always true or always false - therefore, not a proposition
3. $x+2=2x \text { When }x=1$ Yes, is either true or false.
4. $2x<3x$ No, as the truth value changes depending on whether $x$ is positive or negative.

Atoms are often represented with a symbol called propositional variable, (e.g. p, q)

1.1.8 - Complex Propositions

• Complex propositions (or sentences) can be built from simpler propositions using logical connectives.
  • Brackets, ()
  • Negation, $\neg$
  • And/Conjunction, $\wedge$
  • Or/Disjunction, $\vee$
  • Implication, $\Rightarrow$
  • Biconditional/Equivalence, $\Leftrightarrow$

1.1.9 - Rules for Evaluating Truth

$\neg p$

$p\wedge q$

$p\vee q$

$p \Rightarrow q$

$p\Leftrightarrow q$

is TRUE iff p is FALSE

is TRUE iff p is TRUEand q is TRUE

is TRUEiff p is TRUE or q is TRUE

is TRUEiff p is FALSE or q is TRUE

is FALSE iff p is TRUEand q is FALSE

is TRUEiff $p\Rightarrow q \text { and }q \Rightarrow p$

Semantics

• An interpretation I assigns a truth value to each atom
• A sentence $s_1 \wedge s_2$ is true in I if $s_1$ is true in I and $s_2$ is true in I
• A rule $h\Leftarrow b$ is false in I if b is true in I and h is false in I
  • The rule is otherwise true
• A knowledge base KB is true in I if and only if every clause in KB is true in I
• A model of a set of clauses is an interpretation in which all the clauses are TRUE
• If KB is a set of clauses and g is a conjunction of atoms, g is a logical consequence of KB, written $KB \models g$ if $g$ is TRUE in every model of $KB$
• That is, $KB\models g$ if there is no interpretation in which $KB$ is TRUE and $g$ is false.

RiPPLE Activity - Knowledge Base

A model of a set of clauses is an interpretation which all of the clauses are True;

$KB=\begin{cases}p \Leftarrow q\\ q\\ r \Leftarrow s\end{cases}$

$I_1$ is a model of KB as:

$p \Leftarrow q \ \ \ >> \ \ True \Leftarrow True \ \ \ >> \ \ \ True$

$q\ \ \ \ \ \ \ \ \ \ \ >> \ \ True \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ >> \ \ \ True$

$r \Leftarrow s \ \ \ >> \ \ True \Leftarrow True \ \ \ >> \ \ \ True$

$I_2$ is not a model of KB as:

$p \Leftarrow q \ \ \ >> \ \ False \Leftarrow False \ >> \ \ \ True$

$q\ \ \ \ \ \ \ \ \ \ \ >> \ \ False \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ >> \ \ \ \color{#FF6666}False$

$r \Leftarrow s \ \ \ >> \ \ False \Leftarrow False \ >> \ \ \ True$

$I_3$ is a model of KB as:

$p \Leftarrow q \ \ \ >> \ \ True \Leftarrow True \ \ \ >> \ \ \ True$

$q\ \ \ \ \ \ \ \ \ \ \ >> \ \ True \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ >> \ \ \ True$

$r \Leftarrow s \ \ \ >> \ \ False \Leftarrow False \ >> \ \ True$

$I_4$ is a model of KB as:

$p \Leftarrow q \ \ \ >> \ \ True \Leftarrow True \ \ \ >> \ \ \ True$

$q\ \ \ \ \ \ \ \ \ \ \ >> \ \ True \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ >> \ \ \ True$

$r \Leftarrow s \ \ \ >> \ \ True \Leftarrow False \ >> \ \ \ True$

$I_5$ is not a model of KB as:

$p \Leftarrow q \ \ \ >> \ \ True \Leftarrow True \ \ \ >> \ \ \ True$

$q\ \ \ \ \ \ \ \ \ \ \ >> \ \ True \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ >> \ \ \ True$

$r \Leftarrow s \ \ \ >> \ \ False \Leftarrow True \ >> \ \ \ \color{#FF7777}False$

Which of p, q, r, s logically follow from KB? Suppose p, q, r and s are clauses. Does the knowledge base entail these variables?

From the knowledge base, we know that $q$ is true. In the knowledge base, if $p$ is true, then $q$ must be true.

There is no constraint in the knowledge base that causes $r$ or $s$ to be true.

User's View of Semantics

How logic is used to represent a problem.

Formulate information as propositional logic sentences, to create a knowledge base.

1. Choose a task domain (intended interpretation)
2. Associate an atom with each proposition you want to represent
3. Tell the system clauses that are true in the intended interpretation: axiomatizing the domain
   • An axiom is something that is self-evidently true.
   • Axioms are the starting point for determining what is true in our models.
4. Ask questions about the intended interpretation
5. If $KB \models g$ then $g$ must be true in the intended interpretation
   • i.e. if the questions that you are asking are logical consequences of the knowledge base, they must be true.
6. Users can interpret the answer using their intended interpretation of the symbols.

Computer's View of Semantics

• The computer doesn't have access to the intended interpretation
• All it knows is the knowledge base
• The computer can determine if a formula is a logical consequence of KB
• If $KB \models g$ then g must be true in the intended interpretation
• If $KB \not\models g$ then there is a model of $KB$ in which $g$ is false. This could be the intended interpretation.

Deduction: Sentences entailed by the current KB can be added to the KB.

• Added to the knowledge base for later reference / so that we don't have to compute it later
• Querying the knowledge base is a process that can take a significant amount of time

Representing the Electrical Environment

...

Conclusion - Terminology

• A sentence is valid → Its truth value is TRUE for all possible interpretations (e.g. $p \vee \neg p)$
  • The sentence $p\vee \neg p$ is valid as it is True for all possible truth values that $p$ can take.
• A sentence is satisfiable → Its truth value is TRUE for at least one of the possible interpretations (e.g. $\neg p)$
  • Everything that is valid is also satisfiable.
  • The sentence $\neg p$ is satisfiable as it is True when $p=\text{False}$
• A sentence is unsatisfiable → Its truth value is FALSE for all possible interpretations (e.g. $p \wedge \neg p)$
  • The sentence $p\wedge \neg p$ is unsatisfiable as it is true for all possible values of $p$
• For propositional logic, we can always decide if a sentence is valid / satisfiable / unsatisfiable in finite time (decidable problems).
  • Given a very large amount of computational power, we can always determine whether the problem is either valid, satisfiable or unsatisfiable.

2.0 - Validity

2.1 - Terminology

A model of a sentence: An interpretation that makes the sentence to be true

A sentence A entails another sentence B (denoted as $A \models B$) iff every model of $A$ is also a model of $B$ (i.e. $A \Rightarrow B$ is valid)

Two sentences A and B have semantics which result in interpretations of A and B.

If A entails B, all models of B are models of A

(i.e. $A \Rightarrow B)$

2.2 - Simple Validity / Model Checking

Check if $\neg P_{1,2}$ is entailed by the knowledge base.

Knowledge Base

$R_1: \neg P_{1,1}$

$R_2: B_{1,1} \Leftrightarrow (P_{1,2} \vee P_{2,1})$

$R_3: B_{2,1} \Leftrightarrow (P_{1,1} \vee P_{2,2} \vee P_{3,1})$

$R_4: \neg B_{1,1}$

$R_5: B_{2,1}$

2.2.1 - Simple Model Checking

Enumerate the models

• All true/false values for $P_{1,1}, B_{1,1}, P_{1,2},P_{2,1}, P_{2,2}, P_{3,1}$
• Check if $\neg P_{1,2}$ is true in all models where the knowledge base $(R_1\wedge R_2 \wedge R_3 \wedge R_4 \wedge R_5)$ is true

Sound The result is correct

Complete It always gives an answer

Complexity $n$ is the number of propositional variables.

• $\text{Time: } O(2^n) \Leftarrow$ This is bad!
• $\text{Space: }O(n)$

2.2.2 - Theorem Proving - Check Validity without Checking All Models

• Use logical equivalence:
  • For any two sentences$\alpha, \beta$ sentences (which may be atomic or complex), we can convert them to logically equivalent statements using predicate logic rules.
• Two sentences are logically equivalent iff true in the same models: $\alpha \equiv \beta \text{ iff } \alpha \models \beta \text{ and } \beta \models \alpha$

2.2.3 - Transformations of Logical Expressions

Modus Ponens (Model that affirms)

Moduss Tollens (Model that Denies)

And-elimination (for a conjunction, any of the conjuncts can be inferred)

2.2.4 - Example - Theorem Proving using Natural Deduction / Logic

• Enumerate the models
• All true/false values for $P_{1,1}, B_{1,1}, P_{1,2},P_{2,1}, P_{2,2}, P_{3,1}$
• Check if $\neg P_{1,2}$ is true in all models where the knowledge base $(R_1\wedge R_2 \wedge R_3 \wedge R_4 \wedge R_5)$ IS TRUE

Knowledge Base

$S_1: \neg P_{1,1}$

$S_2: B_{1,1} \Leftrightarrow (P_{1,2} \vee P_{2,1})$

$S_3: B_{2,1} \Leftrightarrow (P_{1,1}\vee P_{2,2}\vee P_{3,1})$

$S_4:\neg B_{1,1}$

$S_5:B_{2,1}$

$\color{#AAA}S_2: B_{1,1} \Leftrightarrow (P_{1,2} \vee P_{2,1})$

$S_6: (B_{1,1} \Rightarrow (P_{1,2} \vee P_{2,1}))\wedge ((P_{1,2} \vee P_{2,1})\Rightarrow B_{1,1})$

Using biconditional elimination on $S_2$

$S_7:(P_{1,2} \vee P_{2,1}) \Rightarrow B_{1,1}$ Using And elimination on $S_6$

$S_8:$ $\neg(P_{1,2}\vee P_{2,1})$ Using Modus tollens on $S_4$ and $S_7$

$S_9: \neg P_{1,2} \wedge \neg P_{2,1}$ Using De Morgan to $S_8$

$S_{10}: \neg P_{1,2}$ Using And elimination to $S_9$

3.0 - Conjunctive Normal Form (CNF)

Given $\alpha, \beta$ and $\gamma$ sentences (atomic or complex):

This single inference rule is sound and complete only when applied to propositional logic sentences written in Conjunctive Normal Form.

To make use of this rule, we need to re-express the sentences in our knowledge base in Conjunctive Normal Form

3.1 - Conjunctive Normal Form

• Conjunctive Normal Form (CNF) - Conjunctions of disjunctions, e.g. $(\neg A \vee B)\wedge (C \vee D)\wedge (E\vee F)$
• Some terminology:
  • Clause A disjunction of atoms, e.g. $(\neg A\vee B)$
  • Literals Variables or the negation of variables, $A, \neg A, B$ (atoms or negation of atoms)
• CNF is quite useful for model checking and for solving satisfiability problems too - in fact, we've seen it before!
  • Constraint networks from the previous lecture!

3.1.1 - Converting to CNF

• Three steps to convert to CNF
  1. Eliminate arrows
  2. Drive in negations using De Morgan
  3. Distribute OR over AND
• Note that disjunctions (OR) are inside the parentheses, and the conjunctions (AND) are outside.

Convert $(A \vee B)\Rightarrow (C\Rightarrow D)$

1. Eliminate Arrows

   $(A \vee B) \Rightarrow (\neg C\vee D)$ Using $(\alpha \Rightarrow \beta)\equiv(\neg\alpha \vee\beta)$

   $\neg(A\vee B) \vee (\neg C \vee D)$ Using $(\alpha \Rightarrow \beta)\equiv(\neg\alpha \vee\beta)$

2. Drive in negations

   $(\neg A\wedge\neg B)\vee(\neg C \vee D)$ Using De Morgan's

3. Distribute OR over AND

   $(\neg A\vee\neg C\vee D)\wedge(\neg B\vee\neg C\vee D)$ Using Distributive

3.1.2 - Resolution Refutation

• There are three steps to perform resolution refutation
  1. Convert all sentences to CNF
  2. Negate the desired conclusion
  3. Apply the resolution rule until we either:
     1. derive false (a contradiction)
     2. can't apply the rule anymore
• Resolution Refutation is sound and complete (for propositional logic)
  • If we derive a contradiction, the conclusion follows from the axioms (proof by contradiction)
  • If we can't apply any more, the conclusion cannot be proved from the axioms. (no entailment; not a valid statement)

3.1.3 - Resolution Refutation Example

Knowledge Base:

$(P \vee Q)\wedge (P\Rightarrow R)\wedge(Q\Rightarrow R)$

1. Elimination of arrows

   $(P \vee Q)\wedge (P\Rightarrow R)\wedge(Q\Rightarrow R)$

   $(P \vee Q)\wedge (\neg P\vee R)\wedge(\neg Q\vee R)$

2. Drive in negations using De Morgan's

   As there are no external negations, we don't have to do anything

3. Distribute OR over AND

   As there are only ORs inside the parentheses and ANDs outside the parenthesis, there is nothing to do.

4. Negate the Desired Conclusion

4.0 - Satisfiability

4.1 - Davis-Putnam-Logemann-Loveland (DPLL) Tree Search Algorithm

• An assign-and-simplify strategy

  • Consider a search tree where at each level, we consider the possible arguments to one variable, say V
  • On one branch, we assume V is FALSE and on the other that it is TRUE
  • Given an assignment for a variable, we can simplify the sentence and then repeat the process for another variable
  • DPLL is a backtracking algorithm for systematically doing this.

• The algorithm is building a solution while trying assignments - you have a partial solution which might prove successful or not as you go

• Provides an ordering of variables to check first - this is the key to its efficiency.

• The algorithm is building a solution while trying assignments - you have a partial solution which might prove successful or not as you go

• Provides an ordering of variables to check first - this is the key to its efficiency

• Definitions:

  • Clause: A disjunction of literals
  • Literals: Variables or the negation of variables
  • Unit clause: Clause which has exactly one literal which is still unassigned and the other (assigned) literals are all assigned to FALSE
  • Pure variable: Appears as positive only or negative only in S (a sentence)

• If the current assignment is valid, you can determine the value of the unassigned pure variable in a unit clause, because the literal must be true

• E.g. if we have $(X1\vee X2\vee X3)\wedge (X1\vee X4\vee X5)\wedge(X1\vee X4)$ and we have already assigned $X1=\text{FALSE}$ then choose $X4$ and set it to $\text{TRUE}$

  • If all variables in the clause are false, then the variable that you are looking at must be true.

• DPLL provides an ordering of which variables to check first:

  DPLL(Sentence S)

  If S is empty, return TRUE

  If S has an empty clause, return False

  If S has a unit clause, return DPLL(S(U)) // Assign the one unassigned variable

  Unit clause: Consists of only 1 unassigned literal

  S(U) means a simplified S after a value is assigned to U

  If S has a pure variable U, return DPLL(S(U))

  Pure variable: Appears as positive only or negative only in S

  Pick a variable v // Pick a new variable v, using a heuristic

  If DPLL(S(v)) then return TRUE

  Else return DPLL(S(v))

• Heuristics to pick the variable: Max # of occurrences, min size clauses (only one has to be true, harder to satisfy with less variables).

  • Basically, pick the most constrained variable first (if it's going to fail, it's better to fail early)

DPLL Algorithm

Sound: The result is correct

Complete: It always gives an answer

Speed and memory consumption depends greatly on:

Which symbol is being assigned first

Which assignment is being followed first

A lot of other methods, heuristics and meta-heuristics have been proposed for SAT problems.

https://www.cs.ubc.ca/labs/beta/Projects/SATzilla/