Introduction to Parallel Complexity Theory and Computational Models, Lasse Natvig, NTNU, Trondheim, Norway. Page 1 NTNU

Transkript

1 Introduction to Parallel Complexity Theory and Computational Models, Lasse Natvig,, Trondheim, Norway. Page 1 Introduction to Parallel Complexity Theory and Computational Models Basert på kap. 2 i Evaluating Parallel Algorithms: Theoretical and Practical Aspects, Lasse Natvig, Dr.Ing. thesis, 1990, Department of Computer Systems and Telematics, The Norwegian Institute of Technology, The University of Trondheim. (Veileder Arne Halaas). Hovedresultat oppsummert i Logarithmic Time Cost Optimal Parallel Sorting is Not Yet Fast in Practice!, Lasse Natvig, Proceedings of SUPERCOMPUTING-90 (IEEE og ACM), NewYork, November 1990, pages Spin-off om synkron MIMD programmering oppsummert i Compile and Runtime Padding: An Approach to Realising Synchronous MIMD Execution, Lasse Natvig, IFIP World Computer Congress, Track 4: Theoretical Foundations of Computing, IFIP Transactions A-51, Volume I, page , Hamburg, Aug 1994 General-Purpose Parallel Programming on the PRAM Model, Lasse Natvig, Abstract Machine Models for Highly Parallel Computers, Oxford Science Publications, Chap. 4, page Motivation The gap between theory and practice There are two main communities of parallel algorithm designers: the theoreticians and the practitioners. Theoreticians have been developing algorithms with narrow theory and little practical importance; in contrast, practitioners have been developing algorithms with little theory and narrow practical importance. Clyde P. Kruskal in Efficient Parallel Algorithms: Theory and Practice, [Krus89]. Motivasjon, forts. Datamaskiners ytelse (regnekraft og lagerplass) øker vi kan løse større problemer Større problemer øker relevansen til asymptotisk analyse og kompleksitetsteori Bruk av et stort antall prosessorer blir mer vanlig algoritmer fra teoretisk databehandling (Eng. TCS) (f.eks. PRAM algoritmer) blir mer relevant Kompleksitetsteori Beregningsmessig kompleksitet (Eng. computational complexity) vs. beskrivelsesmessig kompleksitet (Eng. descriptional complexity) Klassifisering av problemer i kompleksitetsklasser Om behovet: The idea met with much resistance. People argued that faster computers would remove the need for asymptotic efficiency. Just the opposite is true, however, since as computers become faster, the size of the attempted problems becomes larger, thereby making asymptotic efficiency even more important. John E. Hopcroft in his Turing Award Lecture Computer Science: The Emergence of a Discipline [Hopc87]. Complexity theory concepts Milestone book : Garey, Michael R. and Johnson, David S., COMPUTERS AND INTRACTABILITY, A Guide to the Theory of NP- Completeness, W. H. Freeman and Co., New York, Central concepts Problem parameter description specication of the properties required of the solution Problem instance one set of values given for all the parameters Algorithm An algorithm is said to solve a problem Π if that algorithm can be applied to any instance I of Π and is guaranteed always to produce a solution for that instance I Complexity functions time complexity space complexity Complexity theory concepts, cnt d Different problem instances may require different amounts of computing resources Worst-case most frequently used efficiency measure provides a guarantee Average case very usefule in practice difficult to derive Best case only for optimists Order-notation and asymptotic complexity an abstraction of the (infinite number of) exact complexity functions can be discussed at a higher level represents growth rate

2 Introduction to Parallel Complexity Theory and Computational Models, Lasse Natvig,, Trondheim, Norway. Page 2 Upper and lower bounds An upper bound (on time) U(n) is established by the algorithm that among all known algorithms for the problem can solve it using the least number of steps in the worst case the problem can be solved at least that fast can be found by analysing one algorithm A lower bound (on time) L(n) tells us that no algorithm can solve the problem in fewer than L(n) steps in the worst case requires a proof of great practical importance determines the inherent complexity of a problem Behovet for modeller Kompleksitetsanalyse og teori baserer seg på abstrakte maskiner (modeller) Tradisjonell analyse baserer seg på RAM modellen Detaljerte analyser er i større grad avhengig av underliggende modell er det ønskelig? Modeller for parallelle beregninger ( abstrakte maskiner ) Sekvensielle modeller RAM tilsv. Von Neumann Enerådende for tradisjonelle datamaskiner (uniprosessorer) Parallelle modeller Et stort antall finnes ofte ulike navn på samme modell ofte ulik tolkning/forståelse av samme modell f.eks. flere kjente forskere/forfattere hevdet at PRAM er SIMD, det er feil! Kan bygge bro mellom teoretisk databehandling og praktisk databehandling Ingen etablerte standard-modeller for parallelle datamaskiner (multiprosessorer) Anslagsvis 100 ulike er foreslått og brukt, eksempler; PRAM (Parallel Random Access Machine) BSP (Bulk Synchronous Parallel Machine) ABSTRAKSJON:Husk det essensielle, glem detaljer ( selektiv glemming ) Taksonomier Flynn mest benyttet SISD, SIMD, MISD, MIMD (antas kjent) generelt ansett som for grov Duncan s taksonomi er et svar på det, men er lite benyttet Mange mange flere finnes The PRAM Model PRAM-modellen Program P-0 P-1 Global memory P-i... Fortune and Wyllie 1978 Synchronous processors Shared memory with?r?w-access Unit cost instructions (also glob.mem.access) one program, BUT MIMD = Local memory P-i = Processor i Fortune & Wyllie 1978 dominerende modell for forskning og beskrivelse av parallelle algoritmer generelt kritisert som for urealistisk Arkitektur (Uendelig mange) synkrone prosessorer med hver sin program-teller (dvs. synkron MIMD) Et (uendelig) stort globalt lager: uniform og billig aksess (små lokale lager) Varianter; eksklusiv/samtidig lesing/skriving gir 4 varianter EREW PRAM, den svakeste CREW PRAM, den mest brukte (den sterkeste veldefinerte) ERCW PRAM CRCW PRAM, den sterkeste, finnes i en rekke undervarianter Common / Arbitrary / Minimum / Sum / Priority / Garbage

3 Introduction to Parallel Complexity Theory and Computational Models, Lasse Natvig,, Trondheim, Norway. Page 3 Complexity theory concepts, cnt d Levels of problems a given instance finding algorithm for solving all possible instances studies of the structure of a class of problems complexity classes Deterministic algorithm corresponds to our standard interpretation of what an algorithm is Nondeterministic algorithm contains a so called guessing stage which informally is assumed to guess the correct solution Definitions of NP Standard formulation: The class NP is the class of problems that can be solved by a nondeterministic polynomial time algorithm Practical formulation: The class NP is the class of problems which has solutions that can be verified to be a solution or not by a deterministic polynomial time algorithm Tradisjonell kompleksitetsteori P = klassen av problemer som kan løses i polynomisk tid vhja. en deterministisk algoritme P NP = klassen av problemer som kan løses av en ikke-deterministisk algoritme i polynomisk tid (deterministisk i eksponensiell tid eller raskere) NPC = NP-komplette problemer, klassen av problemer i NP som er vanskeligst å løse i polynomisk tid * over 1000 problemer er i NPC * hvis ett kan vises å være i P så er alle det, dvs. P = NPC! NP NPC NP completeness NPC is the class of NP-complete problems. A problem Π is NP-complete if 1. Π Є NP and 2. For any other problem Π in NP there exists a polynomial transformation from Π to Π. Consequently; if Π is in P every other problem Π can be solved by using the polynomial tranformation and a solution of Π this is completeness NP-komplette problemer Eksempel, ryggsekkproblemet Uformell definisjon: en mengde ting med kjent størrelse (volum), og en stor sekk av kjent størrelse hver ting har en verdi er det plass til et utvalg av tingene som har en samlet verdi større enn K? Eksponensiell parallellitet NP komplette problemer kan løses i polynomisk tid hvis vi tillater bruk av et eksponensielt antall prosessorer Ryggsekkproblemet løst i polynomisk tid N ting, X1, X2, osv. Binærtre med 2**N prosessorer Er X1 med?, er X2 med? osv. I parallell nivå for nivå Summerer opp totalt volum og verdi i hver prosessor nedover Løser NPC-problem i O(N) tid!

4 Introduction to Parallel Complexity Theory and Computational Models, Lasse Natvig,, Trondheim, Norway. Page 4 Parallell kompleksitetsteori P = klassen av problemer som kan løses deterministisk i polynomisk tid NC = Nick s Class, problemer som kan løses i poly-logaritmisk tid ( ultraraskt ) vha. et polynomisk ( rimelig ) # prosessorer PC = de vanskeligste problemene i P m.h.p. ultrarask løsning med rimelig # prosessorer ( P-komplette ) NC P PC Nick's Class Nick's Class NC is the class of problems that can be solved in polylogarithmic time (i.e. time complexity O(log k n) where k is a constant) with a polynomial number of processors ( i.e. bounded by O(f(n)) for some polynomial function f, where n is the problem size). NC is robust to a large degree insensitive to underlying computational model NC 1 and NC 2 etc. not robust P-complete problems Also called inherently sequential problems PC is the class of P-complete problems. A problem X is P-complete if (a) X is member of P, and (b) For any other problem Y in P there exists a NC-reduction from Y to X. A NC-reduction from A to B (written as A NC B) is a (deterministic) polylogarithmic time bounded algorithm with polynomially bounded processor requirement for transforming (or reducing) any instance I A of A to a corresponding instance I B of B so that the solution of I B gives the required answer for I A. P-complete problems, cnt d Most researchers believe that P-complete problems are outside NC A P-completeness result has the same practical effect as a lower bound P-komplette problemer, eksempler P-komplette algoritmer Maximum nettverk flyt Lineær programmering Dybde først søk Unifisering Spill med to spillere Further Reading; Limits to Parallel Computation: P-Completeness Theory, av Raymond Greenlaw, H. James Hoover og Walter L. Ruzzo [GHR95] Noen algoritmer er mindre egnet for hurtig parallellisering Grådighets-algoritmer An algorithm A for a search problem is P- complete if the problem of computing the solution found by A is P-complete.

5 Introduction to Parallel Complexity Theory and Computational Models, Lasse Natvig,, Trondheim, Norway. Page 5 Summary of sequential and parallel complexity theory