1.0 - Polynomial Time Algorithms - Tractable

Polynomial-time problems are considered tractable (i.e., feasible to solve)
in practice, an algorithm of complexity $\Theta(n^{100})$ is not tractable
However, if a problem has a polynomial time algorithm, there is usually an efficient polynomial-time algorithm.

1.1 - Polynomial Time Algorithms - Computational Models

The class of problems solvable in polynomial time on a serial random-access machine (as we have been using) is the same as:
- The class problems that can be solve in polynomial time on an abstract Turing machine;
- The class of problems that can be solved in polynomial-time on a parallel compute where the number of processors grows polynomially with input size.
For many reasonable models the class of problems that can be solved in polynomial time is the same.

1.2 - Examples of Polynomial and Non-Polynomial Algorithms

1.2.1 - Polynomial Time - Closure Properties

Polynomials are closed under addition, multiplication and composition
- Addition: Run one polynomial time algorithm, then run a second one.
- Multiplication: Run a polynomial time algorithm a polynomial number of times
- Composition: Run a polynomial time algorithm on the output of another polynomial-time algorithm (or run an poly-time algorithm on an input of polynomial size).
For example, if the output of a polynomial time algorithm $F$ is fed as input to another polynomial time algorithm $B$ , the composite algorithm is polynomial time
Note that the size of the input of $F$ may be polynomial in the size of its input
Which problem have polynomial-time algorithms? Problems that seem similar may not have the same time complexity.
We can find the shortest paths from a single source vertex in a directed graph in $O(|V|\cdot |E|)$ time
Finding a longest simple path between two vertices is difficult.
- Simple path - a path with no repeated edges or vertices
Even determining whether a graph contains a simple path with at least a given number of vertices is hard (it is NP-complete)

1.2.2 - Euler Tour vs Hamiltonian Cycle

An Euler tour in a connected directed graph is cycle in which each edge of the graph is traversed exactly once.
Finding an Euler tour can be solved in $O(|E|)$ time
A Hamiltonian cycle in a graph $G$ is a simple cycle that contains all the vertices $G.V$ in the graph.
Though one is a traversal of vertices (Euler), and once is traversal of edges (Hamiltonian), one is completed in linear time, and the other is exponential-time.
Determining whether a graph has a Hamiltonian cycle is difficult (it is NP-complete)

1.2.3 - 2-CNF vs 3-CNF satisfiability

A Boolean formula, such as

$(x_1\vee\neg x_2)\wedge(\neg x_1\vee x_3)\wedge(\neg x_2\vee \neg x_3)$

has a satisfying assignment $x_1=1, x_2=0, x_3=1$ such that it evaluates to 1.
A Boolean formula is in conjunctive normal form (CNF) if it is a conjunction of clauses, each of which is a disjunction of literals, each of which is either a variable or its negation
- Putting expressions into normal forms can help us to complete proofs - we take many problem instances and put them into a common format.
It is in $k-\text{CNF}$ form if it is in CNF form in which each disjunction has exactly $k$ literals - the example above is in $2-\text{CNF}$ .
Determining whether a $2-\text{CNF}$ formula is satisfiable can be done in polynomial time
Determining whether a $3-\text{CNF}$ formula is satisfiable is difficult (it is NP-complete)

1.3 - Defining Classes of Problems

Which problems have polynomial-time algorithms?
To study this, in complexity theory we study classes of problems (not algorithms)
- We focus on decision problems rather than optimisation problems
- We focus on concrete problems rather than abstract problems

1.3.1 - Decision Problems

Decision problems have a simple 1 or 0 answer as their solution
Optimisation problems usually have closely related decision problems
If the optimisation problem is “easy”, the related decision problem is also “easy”
If the optimisation problem is “hard”, the related decision problem is also “hard”

1.3.2 - Decision Problems: An Example

Finding a shortest path in a graph is an optimisation problem

Deciding if a graph $G$ has a path from u to v with at most $k$ is a related decision problem

solve-path(G, u, v, k)
    p = solve-shortest-path(G, u, v)
    if length(p) <= k
				return 1
    else
				return 0

solve-shortest-path can be solved in polynomial time, and hence solve-path can be solved in polynomial time.
In this example solve-shortest-path is our optimisation problem, and solve-path uses solve-shortest-path.
solve-path is the decision problem.
If we cannot solve solve-path in polynomial time then we can argue that we can’t solve solve-shortest-path in polynomial time

2.0 - Complexity Class P

💡 This slide was from Lecture 11.1

Set of concrete decision problems that are polynomial-time solvable.
Decision problems have a simple yes or no, $\{0,1\}$ answer
A concrete problem is a problem whose instance set is the set of binary strings $\{0,1\}^*$
A concrete problem is polynomial-time solvable if there exists and algorithm to solve it in $O(n^k)$ time for some constant $k$ , where $n$ is the length of the binary input string

2.1 - Abstract Problems

An abstract problem is a binary relation on a set $I$ $I$ of the problem instances and a set $S$ $S$ of problem solutions
- There may be multiple solutions for a given problem instance - this is why it is a binary relation, not a function
- The problem instance (input to the problem) is not constrained to be any format - it could be a graph, list, etc.
An abstract decision problem is a function from an instance set $I$ $I$ to the solution set $\{0, 1\}$ ${0, 1}$
- Unique $0$ or $1$ solution for any given instance
- That is, definitively a yes or no answer.

2.2 - Encodings

We need to represent problem instances in a way that a program can understand
An encoding of a set $S$ of abstract projects is a mapping from $S$ to the set of binary strings $\{0, 1\}^*$
Encode a natural number in the usual way: $0, 1,10, 11, 100, 101, 110, 111, \cdots$
Encode a compound object as a binary string combining the representations of the constituent parts.

2.3 - Concrete Problems

A concrete problem is a problem whose instance set is the set of binary strings $\{0, 1\}^*$ ${0, 1}^{*}$
- The input format is now constrained to binary inputs
An algorithm solves a concrete problem in time $O(T(n))$ $O (T (n))$ if when it is provided a problem instance $i$ $i$ of length $n=|i|$ $n = ∣ i ∣$ , the algorithm produces a solution in $O(T(n))$ $O (T (n))$ time.
- Measuring running time as a function of the number of bits in the input

2.4 - Complexity Class P

A concrete problem is polynomial-time solvable if there exists an algorithm to solve it in $O(n^k)$ time for some constant $k$
The complexity class $P$ is the set of concrete decision problems that are polynomial-time solvable

3.0 - Encoding Abstract Problems as Concrete Problems

A abstract decision problem Q maps a problem instance $i$ to $\{0, 1\}$ .

$Q\in I\rightarrow \{0, 1\}$
Using an encoding $e\in I\rightarrow \{0, 1\}^*$ to map an abstract problem $Q$ to a concrete problem $e(Q)$

$e(Q)\in \{0,1\}^*\rightarrow \{0,1\}$
For any instance $i\in I$ of the abstract problem $Q$ , the solution to the concrete problem $e(Q)$ for $e(i)\in \{0,1\}^*$ is $Q(i)$

$e(Q)(e(i))=Q(i)$
- That is, we can give the concrete problem $e(Q)$ the argument $e(i)$ which is the encoded abstract input
Some binary strings may not correspond to instances in $I$ , we assume that such strings arbitrarily map to 0.

3.1 - Polynomial-Time Solvability to Abstract Problems

Polynomial-time solvability is defined above for concrete problems
We want to extend the definition to abstract problems using encodings
But want the definition to be independent of a particular encoding
Unfortunately, the complexity does depend on the encoding.

3.1.1 - Dependency on Encoding - Example

Suppose an integer $k$ is the sole input to an algorithm
The algorithm has a running time which is $\Theta(k)$
If we use a binary representation of $k$ $k$
- The size of the input $n=\lfloor \lg k\rfloor +1$
- The complexity is $\Theta(k)=\Theta(2^n)$
If we use a unary representation of $k$ $k$
- $k$ is represented by a string of $k$ 1s.
- For example, 4 is represented as 1111
- The size of the input is $n=k$
- The complexity is $\Theta(k)=\Theta(n)$
We want to rule out expensive encodings such as the unary encoding.

3.1.2 - Standard Encoding

We assume a standard encoding in which
- The encoding of an integer is polynomially related to its binary encoding
- The encoding of a finite set is polynomially related to its encoding as a list of its elements enclosed in braces separated by commas (e.g., as in ASCII)
- Use $\langle G\rangle$ to stand for the standard encoding of $G$

A function $f\in \{0, 1\}^* \rightarrow \{0, 1\}^*$ is polynomial-time computable if there exists a polynomial time algorithm $A$ , that given any input $x\in \{0, 1\}^*$ produces output $f(x)$
For a set of problem instances, $I$ , two encodings $e_1, e_2$ are polynomially related if there exists polynomial timefunctions $f_{12}$ and $f_{21}$ such that for any $i\in I$ (these functions translate between encodings)

$f_{12}(e_1(i))=e_2(i)\\ f_{21}(e_2(i))=e_1(i)$
And non-instances are mapped to non-instances by both

3.2.1 - Independence from Encoding

Based on the above results, yields the following lemma
Given an abstract problem $Q$ $Q$ on an instance set $I$ $I$ , if $e_1$ $e_{1}$ and $e_2$ $e_{2}$ are polynomially-related encodings of $I$ $I$ , then the concrete problem $e_1(Q)\in P$ $e_{1} (Q) \in P$ if and only if $e_2(Q)\in P$ $e_{2} (Q) \in P$
- The first encoding of the abstract problem is in complexity class $P$ if and only if the second encoding of the abstract problem (given that the first and second encodings are polynomially related)

3.2.2 - Proof

By symmetry, we can show that $e_1(Q)\in P \Rightarrow e_2(Q)\in P$

If $e_1(Q)\in P$ then there exists polynomial-time algorithm $A_1$ that solves $e_1(Q)$
As $e_1$ and $e_2$ are polynomially-related encodings, then there exists a polynomial-time computable function $f_{21}$ such that $f_{21}(e_2(i))=e_1(i)$ for some problem instance $i$
Hence, assume that $F_{21}$ is some polynomial time algorithm implementing $f_{21}$
Then the following algorithm implements $e_2(Q)$ for instance $i$ assuming $i_2=e_2(i)$
```
A2(i2)
	i1 = F21(i2)
  return A1(i1)
```

We need to show that $A_2$ is polynomial time

********************Proof that $A_2$ is polynomial-time

The running time of $F_{21}(i_2)$ is $O(|i_2|^c)$ for some constant $c$
The size of $i_1$ is bounded by the running time of $F_{21}(i_2)$ and hence $|i_1|\in O(|i_2|^c)$
The running time of $A_1(i_1)$ is $O(|i_1|^k)$ for some constant $k$
Hence the running time of $A_1(i_1)$ is $O(|i_1|^k)=O((|i_2|^c)^k)=O(|i_2|^{ck})$
The running time of $A_2(i_2)$ is $O(|i_2|^{ck})$ and hence is polynomial as $c, k$ are constants.