Statistische Mechanik und Optimierung auf zufälligen Strukturen

anbieten. Hinter dem Titel verbirgt sich die folgende (normalerweise aus der Praxis stammende) Kritik an der traditionellen Analyse von Optimierungsalgorithmen:

Was schert mich, dass man mit viel Mühe eine Instanz konstruieren kann, bei der mein Algorithmus schlecht funktioniert, wenn man solche Instanzen suchen muss wie die Nadel im Heuhaufen?

In der Vorlesung sollen Ansätze aus der Statistischen Mechanik benutzt werden, um Algorithmen zu konstruieren, die (beweisbar) nur bei “Ausnahmen” schlecht funktionieren.

Statistische Mechanik ist (im Grunde) ein Teilgebiet der Wahrscheinlichkeitstheorie, und zwar netterweise, aufgrund der Ursprünge in der Physik, ein sehr intuitives. Physikkenntnisse brauchen Sie für die Vorlesung aber nicht.

Der Inhalt der Vorlesung zerfällt auf natürliche Art in drei Zeitabschnitte.

• Einführung: einige einfache Algorithmen auf zufälligen Strukturen
• Statistische Mechanik für Nicht-Physiker
• Aus der Statistischen Mechanik abgeleitete Algorithmen

Voraussetzungen

• Einführung in die Wahrscheinlichkeitstheorie und Statistik
• Kombinatorische Optimierung

Weitere organisatorische und inhaltliche Informationen

werde ich als Kommentare zu diesem Blogpost schreiben (s.u.  bei Responses). Interessenten und Teilnehmer sind gleichfalls eingeladen, sich in den Kommentaren mit Fragen und Kritik zu Wort zu melden!!

Parallel zur Vorlesung soll ein ausfürliches Skript entstehen.

Dieser Post ist der Vorlesung “Kombinatorische Optimierung” (WS 2011/12) gewidmet.

In den Responses (unten klicken) finden sich

• die Übungsblätter
• (hoffentlich) Ihre Fragen, Kommentare und Diskussionen zur VL und den Übungsblättern (gerne anonym)
• Ankündigungen zur VL bzw zum Übungsbetrieb

Das jeweils neue Übungsblatt wird im Laufe des Donnerstags hochgeladen, und ein Link als Kommentar gepostet.

Übungen
Nur noch mal ganz kurz:

Scheinkriterien

Für fortgeschrittene VLs wie der Kombinatorischen Optimierung gibt es keine Zettelkorrekturen mehr. Daher fällt das übliche Kriterium “50% der Übungszettelpunkte” für den Schein weg. Um Sie zu motivieren, trotzdem die Übungsaufgaben zu lösen (und damit Ihre Chancen zu verbessern, die Klausur zu bestehen), folgen wir seit einiger Zeit dem folgenden kleveren Ansatz (mit Varianten):

1. Einziges Scheinkriterium ist Bestehen der Klausur mit mind. 50% der Punkte
2. Jeder VL-Teilnehmer wird zur Klausur zugelassen
3. In den Übungen werden die Übungsaufgaben zur Klausur von frewilligen Übungsteilnehmern vorgerechnet
4. Für das Vorrechnen können Bonuspunkte zur Klausur gewonnen werden, und zwar 0%, 5% oder 10% pro Aufgabe, abhängig von der Qualität der dargestellten Lösung (10% gibt’s nur bei korrekter Lösung und perfekter Präsentation)
5. Melden sich mehrere Freiwillige zum Vorrechnen der gleichen Aufgabe, so wird aus denjenigen mit geringster Punktzahl einer zufällig ausgewält
6. Maximal 40% der Klausurpunkte können vor der Klausur erlangt werden

Außerdem soll es ein Programmierprojekt geben, mit dem Sie Klausurprozente schnorren können. Details dazu gibt es im Lauf der VL. Punkt (6) oben gilt inklusive der Projektpunkte.

Zum Schluss
den Hinweis auf meine Sprechzeiten und Kontaktdaten.

Viel Spaß bei der VL!

Some notation & terminology:

	factorization rank over the semiring S
	nondeterministic communication complexity of the support of A
	fooling set bound

Theoretical questions

1. Cohen & Rothblum (1993) asked: If is rational, is its factorization rank over equal to the factorization rank over ? More generally: If has entries in a sub-semiring of a sub-semiring of the nonnegative reals, how large can the gap be between and ? (Beasley & Laffey 2009)
2. Given any sub-semiring of and , is there a matrix such that and ? (Beasley & Laffey 2009) This is open even for . Cf. Specific-Question #1.
3. If is a symmetric nonnegative matrix with real rank , and if has exactly one negative eigenvalue, must have a nonnegative factorization where is with bounded as a function of ? (Beasley & Laffey 2009)

4. What is the smallest such that for all 01-matrices ? This corresponds to Lovász and Saks log-rank conjecture (with if that is false). Proving lower bounds for corresponds to proving lower bounds for the nonnegative rank.
5. What is the largest separation between the logarithm of the nonnegative rank and the deterministic communication complexity for Boolean functions? The gap cannot be more than quadratic, but the best known example has a smaller gap. A related question asks for the largest separation between the nonnegative rank and the biclique covering number for 01-matrices.
6. It is known that . (Dietzfelbinger Hromkovic Juraj Schnitger (1996)), and there are examples where . Which of these two bounds can be improved? (Dietzfelbinger, Hromkovic, Juraj, Schnitger 1996)

Questions about specific families of matrices

1. A -dimensional, Euclidean distance Matrix of size is defined by points in -dimensional Euclidean space. Its entries are . Do “generic”/“random” Euclidean distance Matrices have full nonnegative rank? (Gillis & Glineur 2011; Lin & Chu 2010). A proof of this for by Lin & Chu 2010 is fatally flawed. More spcifically: Do some linear Euclidean distance Matrices have the propery asked for in Theory-Question #2? (Beasley & Laffey 2009) (Not all of them do (Gillis & Glineur 2011).)

   The rank bound and the ndCC bound are both trivial for these matrices. Upper bounds for special cases have been improved. (Gillis & Glineur 2011)

2. Does a random (with a suitable distribution) rank- matrix have full nonnegative rank? A weaker question asked by J. Garcke, M. Klimm, H.-C. Kreusler, and A. Uschmajew is: Has the set of matrices with nonnegative rank- has positive Lebesgue measure within the rank- matrices? If is randomly drawn from the -matrices with nonnegative rank , is with probability one / positive probability? (Dong, Lin, Chu)

   For some of these questions, erroneous proofs can be found on the internet. In particular, a proof by Dong, Lin, and Chu is incorrect. The rank bound and the ndCC bound are both trivial for these matrices.

3. Is the extension complexity of the Matching Polytope of an -vertex graph polynomial in ?

   An infamous, several decades old question in Combinatorial Optimization asks, whether there exist an extended formulation for the matching problem whose coding size is polynomial in the number of vertices of the graph. A slightly weaker question asks whether there exist an extended formulation whose number of inequalities is bounded by a polynomial in the number of vertices of the graph. The best known lower bound is the trivial rank bound . The ndCC bound is The known upper bounds are , for values of which have been improved several times in the last three decades.

4. Is the extension complexity of the Spanning Tree polytope ?.

   Michele Goemans asked this question at the Cargese Workshop on Combinatorial Optimization 2010, which was dedicated to extended formulations. The best known lower bound is the trivial rank bound. It is unknown whether the ndCC bound improves on the rank bound. The known upper bound is , with improvements for certain types of graphs.

5. What is the extension complexity of the Stable Set polytope of a perfect graph?

   The question goes back to Yannakakis (1991), and was recently repeated by Michele Goemans (MIT) at the Cargese Workshop on Combinatorial Optimization 2010, which was dedicated to extended formulations.

6. Is the extension complexity of the Completion Time polytope ? (Goemans 2011 & Cargese)

   If true, this would extend work by Goemans (2011). The best known lower bound is the trivial rank bound. The upper bound is , which has been proved in a couple of ways in the last 20 years.

7. Is the extension complexity of an -vertex convex polygon with vertices in general position ? (Fiorini, Rothvoss, Tiwary) The rank bound is 2, the ndCC bound is . Some regular convex polygons have extension complexity . The best known lower bound for algebraically independent vertices is .

]]> http://dirkolivertheis.wordpress.com/2011/08/16/open-questions-about-nonnegative-rank-and-related-concepts/feed/ 0 dirkolivertheis http://dirkolivertheis.wordpress.com/2011/06/06/1p67/ http://dirkolivertheis.wordpress.com/2011/06/06/1p67/#comments Mon, 06 Jun 2011 16:50:05 +0000 dirkolivertheis http://dirkolivertheis.wordpress.com/?p=868 ]]> This post is mainly for your comments.
Go directly to the comments, or click on “Responses” below.

Here’s also a link to the slides, assignments, and quizzes:

Link!

Signed SAT is a “multi-valued logic” version of the classical Satisfiability (SAT) problem. In both cases, one is given as input a set of variables and a set of clauses, each consisting of “literals”, and one seeks to find an assignment of values to the variables such that in each clause, at least one literal is satisfied. Whereas in classical SAT the variables are boolean and the literals have the form “” or ““, in signed SAT the literals have the form , where is a member of some set which is fixed (i.e., not part of the input).

Signed SAT has been around for a while. The best starting point for references is probably the paper “The Helly property and satisfiability of Boolean formulas defined on set families” by Chepoi, Creignou, Hermann, and Salzer (it simply is the most recent I know of), but the groundbreaking work was laid by Manyà and Hähnle in the 90s.

In our paper, we study a variant where the set comprises all intervals contained in ; we refer to it as iSAT. We study what a proper adaption of the well-known Unit Clause algorithm for classical random SAT does for our interval Satisfiability problem. Since Unit Clause algorithms alone are usually not much good, we enhance it with a “repair” (or “very limited backtracking”) option, and analyze the whole thing using Wormald’s famous ODE method.

Here are some details about the paper.

Algorithms which repair their mistakes

Say you have a smart algorithm, which proceeds through a number, say , of iterations. In each iteration, the probability of it succeeding is quite large. For example, let us say that, in each iteration, the failure probability is for some . Due to the large number of iterations, though, the expected number of failures is — too large even for constant .

The remedy is, of course, to let the algorithm check, in each iteration, whether it has made a mistake, and if that is the case, repair the damage.

There are certain complications with this approach, mainly because the algorithm has already “looked at” some of the random input data, so that, during the repair, we have to assume that the distribution of some of the data is changed by conditioning on the earlier choices of the algorithm.

However, there’s a trick in this situation: While the overall algorithm (likely) chooses is actions fairly intelligently, for the repair part it is ok to proceed in a fairly stupid way. This is so because all the repair part needs to do, is to push its own failure probability (conditioned on the fact that a repair was necessary and on the previous choices of the algorithm) to . For example, for constant , it suffices that the repair part fails with probability tending to zero arbitrarily slowly.

In our paper, the we bound the failure probability of the “smart” algorithm by , and the “stupid” repair part fails with conditional probability , too.

Branching system

As is often the case with SAT, the analysis of our algorithm entails studying properties of a branching system. Of interest here is the total population size, and we need the expectation and a tail probability estimate. It is actually closer to the problem to speak of a “discrete time queue”, but at the heart of it is (as is the case with queues) a branching system. I have written two posts (How to analyze a discrete time queue and Tail probability estimate for the busy period) on discrete-time queues in preparation for the completion of our paper. So there’s not much to add here, except the one thing that needs to be dealt with almost always in the “real world” — random variables are almost never completely independent, and also almost never identically distributed. There’s generally a small amount of dependence and deviation from the “right” distribution, which kills the textbook- (or Wikipedia-)level theorems one would like to apply, and results in several pages of estimates, inequalities, coupling, and conditioning on the parameter regions in which it all works, leading to more estimates and inequalities. Personally, I find that these complications really are a pain: they are usually not complicated, but I never know in how much detail to write them down.

The same problem also occurs at a point where we (would usually) use Wald’s equation, but the summands are neither (completely) independent nor (completely) identically distributed. Fortunately, though, at this point we can revert to Optional Stopping.

Wormald’s ODE-method

A black-box theorem by Wormald allows to control parameters of a random process by writing down conditional expectations of their changes from one iteration to the next in the form of an initial value problem. While I am generally not a big fan of black-box theorems, I quite fancy this one. The basic idea behind the whole method is the simple trick “if you know how the conditional expectations behave, try to find a martingale and use the Azuma-Hoeffding inequality”, but the way it is used to prove the ODE-theorem is quite nice.

A little bit of ODE calculus

Even though this is tremendously simple from an analysis point of view, we do some computations with the system of ODEs which we obtain, and transform it into a single initial value problem by a change of parameters. The resulting IVP is still not solvable analytically (I guess), but it is a lot easier to give estimates on how the solution behaves, and also to solve it numerically, which we do in order to determine quantitative properties of the input data for which our algorithm succeeds.

More specifically, the initial ratio of the number of clauses over the number of variables must be such that the solution to the IVP does not intersect a certain line before it falls below a certain other function. The line describes the point at which the queue length (and thus the length of the busy period) of the discrete-time queue crosses from finite to infinite.

1. Zufallsgraphen. Anhand eines Lehrbuches, für Bachelor-Student(inn)en und Diplom im Modul 12 (Blockseminar)
2. Ausgewählte Methoden für die kombinatorische Optimierung. Forschungsliteratur, für Master-Student(inn)en und Vertiefungsgebiet (Mittwochs)

Bitte beachten Sie die aktuellen Mitteilungen: Unten auf Responses klicken!

Scheinkriterien:

• 80 min Vortrag + 10 min Diskussion
• Faustregel: Beweise an der Tafel!
• Hand-Out nach belieben
• Vorbesprechung mit fertigem Vortrag mindestens zwei Wochen vor dem Vortragstermin
• Einen guten Vortrag halten

Zufallsgraphen

Das Seminar wird als Blockseminar stattfinden.

Wenn man alle Graphen auf Ecken in einen Sack steckt und zufällig einen davon zieht — welche Eigenschaften wird dieser Graph wahrscheinlich haben? Wie groß ist z.B. eine maximale Clique? Eine unabhängige Eckenmenge? Hat dieser Graph ein perfektes Matching? Oder sogar einen Hamilton’schen Kreis?

Ziel sind Fragen wie die folgenden: Kann man Algorithmen angeben, die für einen solchen Graphen mit hoher Wahrscheinlichkeit eine große Clique, ein perfektes Matching, einen Hamilton’schen Kreis liefern? Werden auf einem zufälliger Graphen einfache Optimierungsansätze wie etwa Greedy-Algorithmen eher gute oder eher schlechte Ergebnisse liefern?

Einen zufälligen Graphen wie den gerade beschriebenen erhält man, wenn man für jede potentielle Kante eine faire Münze wirft: Bei “Kopf” soll die Kante hinzugefügt werden, bei “Zahl” nicht. Allgemeiner kann man sich eine Wahrscheinlichkeit vorgeben, und jede potentielle Kante mit Wahrscheinlichkeit hinzufügen und mit Wahrscheinlichkeit nicht. Die Graphen, die man so erhält heißen Erdös-Rényi-Zufallsgraphen. Mit Ihnen werden wir uns in dem Seminar beschäftigen.

Voraussetzungen

• Kombinatorische Optimierung oder Graphentheorie
• Lineare Optimierung (bzw. Einführung in die Mathematische Optimierung)
• Einführung in die Wahrscheinlichkeitstheorie und Statistik
• Algorithmische Mathematik
• Englisch
• Man sollte das Symbol ” ” mögen!

Es sind elementare Kenntnisse in der Stochastik vonnöten, wie sie in der genannten Vorlesung vermittelt werden. Fast alle Zufallsvariablen sind diskret .

Vortragsthemen
Die Vortragsthemen sind fast ausschließlich Abschnitte der Bücher

• Random Graphs von Janson, Łuczak und Ruciński (in der UB vorhanden, Errata hier), und
• Random Graphs von Bollobás.

ist der Erdös-Rényi-Zufallsgraph. Dabei ist die Kantenwahrscheinlichkeit eine Funktion der Anzahl der Ecken .

Mit * gekennzeichnete Themen sind noch nicht vergeben.

1. Unter welchen Bedingungen an hat einen (beliebigen festen) vorgegebenen Teilgraphen ? (Theoreme 3.4 und 3.9 in JLR)
2. Wann kann man durch Kopien eines (beliebigen festen) vorgegebenen Teilgraphen überdecken? (Theorem 3.22(i) in JLR)
3. Wann hat ein perfektes Matching? (Theorem 4.4 in JLR)
4. Der Erdös-Rényi-Phasenübergang (von viele kleine Zusammenhangskomponenten plötzlich zu einer großen) (Theorem 5.4 in JLR)
5. Die Unabhängigkeitszahl von (Theorem 7.4 in JLR; das in manchen Exemplaren fehlende Blatt des Buches gibt es hier)
6. Greedy-Algorithmus für das Färben von Zufallsgraphen (Theorem 7.9 in JLR)
7. Mehr über die chromatische Zahl: Konzentration, Bollobás Methode und Mathulas Technik (Theoreme 7.11 & 7.14 sowie Lemma 7.18 in JLR; schweres Thema)
8. Posás Methode für Hamilton’sche Kreise in Zufallsgraphen (Theorem 8.9 in Bollobás)
9. 0/1-Gesetze (Kap. 10 in JLR)
10. Große Minoren in (Theorem 3 in diesem Artikel; extra schweres Thema)

Daten

• Blockseminar
• Termine werden noch festgelegt.

Ausgewählte Methoden der kombinatorischen Optimierung

In diesem Seminar werden wir uns mit einigen schwierigen Artikeln zu jüngeren Forschungsergebnissen beschäftigen. Im Fordergrund steht dabei immer die Identifikation von leistungsfähigen neuen Beweismethoden. Die Auswahl der Artikel ist noch nicht sicher, aber die unten stehenden kommen in Frage. (Noch verfügbare Themen sind mit * gekennzeichnet.)

1. Stern-Chromatischer Index
2. Dependent Random Choice.
3. (Diese Zeile wurde absichtlich leer gelassen)
4. Entropie zum Zählen und Abschätzen
5. Versteckte Cliquen finden
6. *“Stabiles” Max-Cut
7. Algorithmisches Lovász Local Lemma: Hier und hier loslegen
8. *Lemma von Sauer-Shelah-Vapnik-Chervonenkis, Hauser-Schranke, fast-orthogonale -Vektoren hier loslegen
9. *Entropie, Hypergraphen und die Bonami-Beckner-Ungleichung: Hier bei “Hypergraphs, Entropy, and Inequalities” anfangen, dann gegebenenfalls bei “Proof of a Hypercontractive Estimate via Entropy” weitermachen
10. Überdeckungszeit von Random Walks in Graphen
11. *Weitere Themen auf Anfrage

Neben dem im Vorlesungsverzeichnis angegebenen Termin (Mittwochs 17-19 Uhr) werden einige Sitzungen an Samstagen stattfinden.

In other words, if the length of the queue at the beginning of minute (a nonnegative integer), is denoted by , we have the following

where the are iid random variables distributed as and the are iid random variables distributed as . Moreover, .

As in the other post, we are interested in the busy time after the start, in other words, the time between 0 and the first return of to the state 0. I already argued that, if is the probability generating function of this time, and and , are the probability generating functions of and , respectively, then the following is true

where is the probability generating function of some other random variable, which is no longer of particular interest here.

In this post, I’d like to make some statements about properties of , based on the system of equations for the probability generating function.

The mean

In the other post, we obtained the formula for the mean of as a consequence of the fact that is a stationary Markov chain, for which we could write down the probability that it is in state 0. Here, I’ll use the probability generating function.

We have to compute . By the chain rule, we obtain

and, assuming everything converges, we obtain

where, to obtain the second equation, we solved the first one for . With and , this is just what we obtained from the Markov chain.

At this point, note that the desired convergence of the power series and is equivalent to them having finite means, plus Abel’s theorem, possibly.

Tail inequality

Now I’d like to bound the probability that the server is busy for a very long time. For this, I will focus on specific distributions of and , namely, I’ll assume that both are Poisson distributions. Thus, the probability generating functions of and are the following:

Hence, we obtain

The second equation can be rewritten as

of course, so we have a nice expression for the inverse of .

Given that we already “have” the probability generating functions, we will use the exponential moment method (= exponentiate + Markov + optimize) to bound the probability:

which is valid for every , and is optimized at some point. The expression on the right is equal to

We now write and use (*). Noting that is equivalent to , we compute

Now we optimize (later we’ll have to make sure that for our choice of , there really exists a ). Noting that since for and the expression (**) goes to infinity, we just have to take for the unique point where the derivative in (**) vanishes, which is

.

Summarizing, we obtain the following estimate.

Proposition (Tail estimate for our discrete queue).

Now, with , the term under the logarithm tends to , and because of , so that there exist (depending on ) and (depending on both and ) such that for all . Consequently, the bound in the proposition is essentially linear, i.e., for these we have

Corollary.

.

We still have to verify that our choice for is feasible, i.e., there must exist a with . Let . A swift computation shows that for . From elementary calculus we know that this implies that for these , there exists a with . But our was because (unless the queue were trivial). Moreover, we have

where the last inequality holds whenever (because the two roots of the equation are and ). By increasing if necessary, we can assume that holds.

Deterministic Communication Complexity

As a digression, let us first discuss the classical deterministic communication complexity of a boolean function . The idea is that Alice and Bob want to compute a specific value , but Alice only knows whereas Bob only knows . Now they’ll have to communicate, until one of them, say Bob, knows . Communication is assumed to be the “expensive” thing, so that is what Alice and Bob try to minimize.

The maximum over all inputs of the minimum number of bits Alice and Bob have to exchange before Bob knows the answer, is called the communication complexity of .

It is important to realize that Alice and Bob have unlimited amount of storage and computing power. In particular, both of them may store the complete - matrix (along with all prime numbers and every book ever written), if they think it might help. (In this sense, it is like the opposite of an exam for students: Alice and Bob may use the complete solution to all exam problems, but they have to find out which problem they are supposed to solve). The question is not, how difficult is to compute . The question is, how difficult it is to agree on which region of the matrix the parties “are” in.

Nondeterministic CC

Now let us discuss nondeterministic communication complexity. Here, in addition to their respective input and , Alice and Bob are given a “certificate” for the answer , which both can verify independently. If in their verifications, they both independently conclude that the certificate proves that , then Bob replies ’1′. Otherwise, he says ’0′. Bob may err in his reply, but only in saying ’0′ when the right answer would have been ’1′. If the true answer is ’0′, Bob may never say ’0′. (This is just like in nondeterministic computational complexity classes.) Moreover, if the true answer is ’1′, there must be at least one certificate with which Alice and Bob arrive at the true answer.

Obviously, for Bob to learn the result of Alice’s checking of the certificate, Alice has to communicate one bit to Bob; this bit is generally ignored. The “nondeterministic communication” is then the coding length of the certificate. Since Alice and Bob may store all possible certificates which they might ever occur, this is the same as the coding length of the number of certificates needed for the function .

The only kind of yes/no questions Alice and Bob can answer independently are of the form (for Alice), or , respectively (for Bob). Thus, certificates amount to rectangles.

Suppose that we have a number of submatrices of , such that every entry in every is one. These matrices are supposed to cover all the 1s in in the sense that for every if , then at least one of the contains the -entry. The usual word for such submatrices is 1-rectangle (or just rectangle), and with said properties is called a rectangle covering. when Alice and Bob receive their inputs and , respectively, if the right answer would be ’1′, the certificate could be any of the rectangles containing the entry . If, however , it does not matter which certificate is given — Bob will always reply ’0′.

Thus, determining the finding the nondeterministic communication complexity of amounts to fining a rectangle covering with minimum number of rectangles. The nondeterministic communication complexity is then the coding length of , i.e., .

Note that “rectangle covering” makes sense for all 0/1-matrices — not just square matrices with the number of rows/columns a power of two.

Rectangle covering as a coloring problem

Let us make this even a bit more combinatorial. Let be an -matrix , and denote by its rectangle covering number, i.e., the minimum number of 1-rectangles needed to cover all 1-entries of .

We define the rectangle graph as follows. The vertex set is the set of 1-entries of ; two such entries are adjacent, if the rectangle they span contains a 0. Say, , so we have the two vertices and . They are adjacent, if and only if, .

Denoting by the usual (vertex-) chromatic number, the following is true.

Lemma 1

Proof. For a proof, it suffices to realize that a 1-rectangle in is an independent set in , and that, by the definition of the edges of , for every independent set in , the smallest rectangle which contains is a 1-rectangle.

I have been unable to find this fact elsewhere in published literature. The proof, although quite easy, can be found in a forthcoming paper.

So nondeterministic CC amounts to coloring a certain graph defined by the boolean function.

Connection with nonnegative rank (and extension complexity)

Now that the basics are clear, let us come back to the original question. Let be a nonnegative matrix and ist support-matrix, i.e., is the 0/1-matrix with if and only if . The following fact was already known to Yannakakis (1991).

Lemma 2

Proof. The nonnegative rank of is the minimum number of nonnegative rank-1 matrices whose sum equals . In other words, there exist nonnegative vectors such that

.

The support of is thus the union of the supports of the . But clearly, the support of is a rectangle: it is the product of the support of and the support of .
Thus, we have a rectangle covering using rectangles, which implies the inequality in the lemma.

In the context of the extension complexity of a polytope , the rectangle graph can be defined as follows. The vertices of are the pairs , where is a point in and the inequality is valid and supporting for , such that does not lie on the hyperplane defined by . Two such pairs are adjacent in , if at least one of the two points does lie in one of the two hyperplanes. For this graph we have

Lemma 3

]]> http://dirkolivertheis.wordpress.com/2011/01/08/nonnegative-rank-and-nondeterministic-communication-complexity/feed/ 0 dirkolivertheis http://dirkolivertheis.wordpress.com/2011/01/05/nonnegative-rank-and-projections/ http://dirkolivertheis.wordpress.com/2011/01/05/nonnegative-rank-and-projections/#comments Wed, 05 Jan 2011 17:22:53 +0000 dirkolivertheis http://dirkolivertheis.wordpress.com/?p=512 ]]> For a nonnegative matrix -matrix , its nonnegative rank is the smallest integer for which there exist nonnegative matrices and with columns such that . Expanding the product, this is the same as the minimum number of nonnegative rank-1 matrices which sum up to :

The expression is called a nonnegative factorization of size of .

Recall that the extension complexity of a (convex) polyhedron is the minimum number of facets of a polyhedron for which there exists a projective mapping which maps onto(!) .

In this post I would like to cover the relationship between the nonnegative rank of a nonnegative matrix and extension complexity of a polyhedron . As it turns out, the extension complexity of is just the nonnegative rank of the so-called slack matrix of (or, more accureately, any slack matrix).

This essentially goes back to Yannakakis’ 1991 paper Expressing combinatorial optimization problems by linear programs.

The slack matrix

The central object linking extension complexity of polyhedra and nonnegative rank is the slack matrix. Let the polyhedron be given by a system of linear inequalities

where each is an affine function , and the are in ; and also by a set of points and vectors , i.e.,

where denotes the convex set generated by and denotes the convex cone generated by . Let , and consider the -matrix defined by

This matrix is called a slack matrix of with respect to the systems (2) and (3).

Nonnegative factorizations of the slack matrix and extensions of the polyhedron

The following is the central fact of this post.

Theorem 1

• Every nonnegative factorization of of size gives rise to an extension of with at most facets.
• Every extension of with facets gives rise to a nonnegative factorization of of size at most .

In particular, .

Some remarks about the nonnegative factorization

In the “linear” case, we know that row- or column-operations does not change the rank of a matrix. The same is true for the nonnegative rank — but only “nonnegative” row- and column-operations are allowed. A nonnegative column operation is one of the following:

1. multiplying a row by a strictly positive scalar,
2. adding a nonnegative multiple of one row to another row,
3. adding a new row to the matrix, all of whose entries are 0.

Nonnegative column-operations are defined similarly. The following is an easy exercise.

Lemma 2 If the matrix results from by a series of nonnegative row- and column-operations, then for every nonnegative factorization of of size , there is a nonnegative factorization of of size , and vice versa.

In particular, .

A (projective) extension gives a factorization

We now prove the first part of Theorem 1. We first deal with the basic case when is a polyhedral cone, the are all zero, and the projections are required to be linear. After that, we will reduce the general cases to this one.

Suppose that is a polyhedral cone which is not a point. This implies that must be among the points , and for all . By Lemma 2, we may assume that and . Deleting the corresponding 0-column, we arrive at and we are interested in nonnegative factorizations of the -matrix defined as .

We first prove that an extension with linear projection mapping gives rise to a nonnegative factorization of appropriate size. Suppose that there is a polyhedron and a linear mapping which maps onto . By a translation, we may assume that . Denoting by the adjoint linear mapping, the linear inequalities are valid for . Moreover, since these inequalites are satisfied with equality by the point , for example, by Farkas’ Lemma, is a nonnegative linear combination of linear functions for which the inequality define a facet of :

Choosing for every a with , we conclude

if we let and . Thus we have a nonnegative factorization whose size is no larger than the number of facets of .

Secondly, we need to show that the general case ( a polyhedron and the projection allowed to be projective) is implied by the cone/linear case. This is done easily by homogenization. Saying that is projective with is the same as saying that arises from a linear mapping which maps onto . But then also maps the homogenization of onto the homogenization of . A factorization of a slack matrix of is also a slack matrix of .

A factorization gives an affine extension

Here we prove that every nonnegative factorization of the slack matrix of a polyhedron gives rise to an extension. Indeed, if , then define the polyhedron

and the projection . Clearly, if then and so satisfies for all , which implies that (because the system (2) defines ).

To see that maps onto , denote the rows of by . Then, for we have

whereas for , the equation is

To show that contains , we prove that that for all , and for all and . But for by (8), and follows form the linearity of and by (8) for and (9).

]]> http://dirkolivertheis.wordpress.com/2011/01/05/nonnegative-rank-and-projections/feed/ 1 dirkolivertheis http://dirkolivertheis.wordpress.com/2011/01/04/how-to-analyze-a-discrete-time-queue/ http://dirkolivertheis.wordpress.com/2011/01/04/how-to-analyze-a-discrete-time-queue/#comments Tue, 04 Jan 2011 21:29:20 +0000 dirkolivertheis http://dirkolivertheis.wordpress.com/?p=473 ]]> In this post, I would like to make some comments about a special case of discrete queues.  This is almost surely in some textbook exercise somewhere, but since I have only found continuous time queue stuff, I am going to write it down for fun.  To read this post, no background in queueing theory is required.  Large parts of this post are transcriptions from one of my favourite books: Grimmett & Stirzaker’s Probability and Random Processes, which covers the classical continuous time case with iid interarrival times.

In the classification of queues, we are in the M/D/1 case:  The arrival of customers is Markovian, the service time is deterministic, and there is one server.  More concretely, suppose that a server can finish off precisely one customer per, say, minute.  Suppose that at time — meaning, is an integer indicating how many minutes have passed since “time 0″ —, there are customers waiting in the queue.  In the th minute, the server will deal with one of them, in the case when , or take a break, if .  In any case, customers arrive during the minute that passes.

We will make the assumption that the number of newly arriving customers may depend on  whether the queue is empty or not.  One may interpret this in two ways:  If the queue is empty, then, apparently, it is not a very busy time, so it is likely that the number of arriving customers is smaller than “usual”.  The other direction in which this can be interpreted is this: if the queue is empty, then everybody rushes to this server, in the hope of being done quickly, so the number of arriving customers in such a situation should be larger than “usual”.

Whichever interpretaion you choose, denote by the number of customers arriving in the th minute in the case when the queue is empty at the beginning of this minute, and denote by the number of customers arriving at the queue in minute , if, at the beginning of this minute, the queue is not empty.  Then clearly, we have the following

and .

We make the additional assumptions that

• the are iid, distributed as
• the are iid, distributed as
• the and the are independent from each other and from the , .

We are interested in computing the distribution of the busy time of the server, i.e., the time between two successive “breaks”.  Intuitively, the mean busy time should amout to , if and , because in minute 0, customers arrive, and in one minute customers are serviced.

Let us make this rigorous.

The Markov chain

Under these assumptions, Q(t) is a Markov chain with state space the nonnegative integers.  Moreover, it is time homogeneous, i.e., depends only on the value of , not on .  The busy time is the first return time of the state 0 in our Markov process.
Let us write down the transition probability matrix, which I like to write in the form the probability to go from state to state :



where and .
Now note that the Markov process is irreducible and aperiodic.  By solving the equation , we check whether a stationary distribution exists, which, by the general theory of time-homogeneous Markov processes, is equivalent to a Markov chain being non-null persisten (i.e., the first return time is finite).

From the triangular structure of the matrix we see that, given an arbitrary value for , the , , are determined.  Moreover, it is easy to see that, if , then these are also nonnegative [see Grimmet & Stirzacker, proof of (14.3.5)].  Now let
,
the probability generating function of the stable distribution, should it exist, the probability generating function of the and the probability generating function of the .  Writing the equations , or, row-wise



in terms of , we see that



and solving for , we obtain



For a stationary distribution to exist, we must have and, by Abel’s theorem,



Using l’Hôpital’s rule, this amounts to



Summarizing, the mean first return time to state 0, , is finite if and only if and in this case it is in fact equal to

.

At the same time, we have computed the probability generating function of the stable distribution.

Distribution of the busy time

In the minute that the th customer is serviced, new customers arrive.  Denote by the accumulated service times of these customers and their “descendants” (sons, sons of sons, ….), where a customer Peter is a “son” of the customer Paul, if Peter arrives in the minute in which Paul is serviced.  The service time, , of the th customer and its descendants is now equal to .  Now note that the distribution of is the same as that of the , and that the are independent.  For the probability generating function of , we conclude that

,

in other words, if and , then

.

(Recall these properties of probability generating functions of sums.)
Finally, the length of the total busy time after time 0 satisfies



if is the accumulated service time of the th customer arriving in the zeroeth minute and its descendants — so the are iid and distributed like .  We obtain

.

Put differently, if we denote the probability generating function of the total busy time after time 0 by and the probability generating function of by , then they satisfy the equations


.


]]> http://dirkolivertheis.wordpress.com/2011/01/04/how-to-analyze-a-discrete-time-queue/feed/ 3 dirkolivertheis