Motivation The gap between theory and practice There are two main communities of parallel algorithm designers: the theoreticians and the practitioners. Theoreticians have been developing algorithms with narrow theory and little practical importance; in contrast, practitioners have been developing algorithms with little theory and narrow practical importance. Clyde P. Kruskal in Efficient Parallel Algorithms: Theory and Practice, [Krus89].
Motivasjon, forts. Datamaskiners ytelse (regnekraft og lagerplass) øker? vi kan løse større problemer Større problemer øker relevansen til asymptotisk analyse og kompleksitetsteori Bruk av et stort antall prosessorer blir mer vanlig? algoritmer fra teoretisk databehandling (Eng. TCS) (f.eks. PRAM algoritmer) blir mer relevant
Kompleksitetsteori Beregningsmessig kompleksitet (Eng. computational complexity) vs. beskrivelsesmessig kompleksitet (Eng. descriptional complexity) Klassifisering av problemer i kompleksitetsklasser Om behovet: The idea met with much resistance. People argued that faster computers would remove the need for asymptotic efficiency. Just the opposite is true, however, since as computers become faster, the size of the attempted problems becomes larger, thereby making asymptotic efficiency even more important. John E. Hopcroft in his Turing Award Lecture Computer Science: The Emergence of a Discipline [Hopc87].
Complexity theory concepts Milestone book : Garey, Michael R. and Johnson, David S., COMPUTERS AND INTRACTABILITY, A Guide to the Theory of NP- Completeness, W. H. Freeman and Co., New York, 1979. Central concepts Problem parameter description specication of the properties required of the solution Problem instance one set of values given for all the parameters Algorithm An algorithm is said to solve a problem? if that algorithm can be applied to any instance I of? and is guaranteed always to produce a solution for that instance I Complexity functions time complexity space complexity
Complexity theory concepts, cnt d Different problem instances may require different amounts of computing resources Worst-case most frequently used efficiency measure provides a guarantee Average case very usefule in practice difficult to derive Best case only for optimists Order-notation and asymptotic complexity an abstraction of the (infinite number of) exact complexity functions can be discussed at a higher level represents growth rate
Upper and lower bounds An upper bound (on time) U(n) is established by the algorithm that among all known algorithms for the problem can solve it using the least number of steps in the worst case the problem can be solved at least that fast can be found by analysing one algorithm A lower bound (on time) L(n) tells us that no algorithm can solve the problem in fewer than L(n) steps in the worst case requires a proof of great practical importance determines the inherent complexity of a problem
Complexity theory concepts, cnt d Levels of problems a given instance finding algorithm for solving all possible instances studies of the structure of a class of problems complexity classes Deterministic algorithm corresponds to our standard interpretation of what an algorithm is Nondeterministic algorithm contains a so called guessing stage which informally is assumed to guess the correct solution
Definitions of NP Standard formulation: The class NP is the class of problems that can be solved by a nondeterministic polynomial time algorithm Practical formulation: The class NP is the class of problems which has solutions that can be verified to be a solution or not by a deterministic polynomial time algorithm
Tradisjonell kompleksitetsteori P = klassen av problemer som kan løses i polynomisk tid vhja. en deterministisk algoritme NP = klassen av problemer som kan løses av en ikke-deterministisk algoritme i polynomisk tid (deterministisk i eksponensiell tid eller raskere) P NP NPC NPC = NP-komplette problemer, klassen av problemer i NP som er vanskeligst å løse i polynomisk tid * over 1000 problemer er i NPC *hvis ett kan vises å være i P så er alle det, dvs. P = NPC!
NP completeness NPC is the class of NP-complete problems. A problem? is NP-complete if 1.?? NP and 2. For any other problem? in NP there exists a polynomial transformation from? to?. Consequently; if? is in P every other problem? can be solved by using the polynomial tranformation and a solution of? this is completeness
NP-komplette problemer Eksempel, ryggsekkproblemet Uformell definisjon: en mengde ting med kjent størrelse (volum), og en stor sekk av kjent størrelse hver ting har en verdi er det plass til et utvalg av tingene som har en samlet verdi større enn K? Eksponensiell parallellitet NP komplette problemer kan løses i polynomisk tid hvis vi tillater bruk av et eksponensielt antall prosessorer
Ryggsekkproblemet løst i polynomisk tid N ting, X1, X2, osv. Binærtre med 2**N prosessorer Er X1 med?, er X2 med? osv. I parallell nivå for nivå Summerer opp totalt volum og verdi i hver prosessor nedover Løser NPC-problem i O(N) tid!
Parallell kompleksitetsteori P = klassen av problemer som kan løses deterministisk i polynomisk tid NC = Nick s Class, problemer som kan løses i poly-logaritmisk tid ( ultraraskt ) vha. et polynomisk ( rimelig ) # prosessorer NC P PC PC = de vanskeligste problemene i P m.h.p. ultrarask løsning med rimelig # prosessorer ( P-komplette )
Nick's Class Nick's Class NC is the class of problems that can be solved in polylogarithmic time (i.e. time complexity O(log k n) where k is a constant) with a polynomial number of processors ( i.e. bounded by O(f(n)) for some polynomial function f, where n is the problem size). NC is robust to a large degree insensitive to underlying computational model NC 1 and NC 2 etc. not robust
P-complete problems Also called inherently sequential problems PC is the class of P-complete problems. A problem X is P-complete if (a) X is member of P, and (b) For any other problem Y in P there exists a NC-reduction from Y to X. A NC-reduction from A to B (written as A? NC B) is a (deterministic) polylogarithmic time bounded algorithm with polynomially bounded processor requirement for transforming (or reducing) any instance I A of A to a corresponding instance I B of B so that the solution of I B gives the required answer for I A.
P-complete problems, cnt d Most researchers believe that P-complete problems are outside NC A P-completeness result has the same practical effect as a lower bound
P-komplette problemer, eksempler Maximum nettverk flyt Lineær programmering Dybde først søk Unifisering Spill med to spillere Further Reading; Limits to Parallel Computation: P-Completeness Theory, av Raymond Greenlaw, H. James Hoover og Walter L. Ruzzo [GHR95]
P-komplette algoritmer Noen algoritmer er mindre egnet for hurtig parallellisering Grådighets-algoritmer An algorithm A for a search problem is P- complete if the problem of computing the solution found by A is P-complete.
Summary of sequential and parallel complexity theory