Jekyll2019-04-18T14:29:45-04:00https://paulhe.com/feed.xmlPaul HePaul HeCoinduction2019-04-17T00:00:00-04:002019-04-17T00:00:00-04:00https://paulhe.com/2019/04/17/coinduction<p>Coinduction is the mathematical dual to an indispensible mathematical tool: induction. While mathematical induction has been known for thousands of years, coinduction has only been studied for a few decades. It is still primarily used in computer science, from which it originated in the field of concurrency theory. Coinduction allows us to define circular or <em>infinite</em> objects (such as <a href="https://en.wikipedia.org/wiki/Stream_(computer_science)">streams</a>, lists that can be infinitely long), and to prove things about them.</p> <p>It should not be confused with <a href="https://en.wikipedia.org/wiki/Coinduction_(anaesthetics)">this coinduction</a>, which may put you to sleep instead.</p> <h2 id="inductive-definitions">Inductive definitions</h2> <p>Inductive (or recursive) definitions are ubiquitous in mathematics, to the point where they are often implicit. They follow a common pattern to build up a set of objects incrementally. A base case (or multiple) is first established, and then rules for building up objects based on previous levels are defined.</p> <p>The set of finite strings <script type="math/tex">S</script> on an alphabet <script type="math/tex">\Sigma</script> is the set inductively defined by the following rules, in <a href="https://en.wikipedia.org/wiki/Rule_of_inference">inference rule</a> notation:</p> <script type="math/tex; mode=display">\frac{}{\epsilon \in S} \qquad \frac{s \in S \quad \sigma \in \Sigma}{\sigma s \in S}</script> <p>So <script type="math/tex">\epsilon</script> (the empty string) is a string, and for any symbol <script type="math/tex">\sigma</script> in the alphabet, we can prepend that onto another string to yield a string. Only the objects generated from the rules are in <script type="math/tex">S</script>.</p> <p>Inductive definitions can be thought of as an iterative process: we start with the empty set and keep adding objects according to the definition, until in the limit, we reach a <a href="https://en.wikipedia.org/wiki/Fixed_point_(mathematics)">fixed point</a>, when applying the rules no longer adds anything new to the set. We add <script type="math/tex">\epsilon</script>, then the length 1 strings, then the length 2 strings, and so on, until we have the infinite set of strings over <script type="math/tex">\Sigma</script> of any length in <script type="math/tex">\mathbb{N}</script>.</p> <p>An inductive definition is thus the <em>smallest</em> set closed <em>forward</em> under its defining rules. That is, <script type="math/tex">S</script> is the smallest set such that <script type="math/tex">\epsilon \in S</script> and that if <script type="math/tex">s \in S</script>, then <script type="math/tex">\sigma s \in S</script> for any <script type="math/tex">\sigma \in \Sigma</script>. We apply the rules from premises to conclusion.</p> <h2 id="coinductive-definitions">Coinductive definitions</h2> <p>Since coinduction is the dual to induction, let’s try “flipping” the inductive definition. A coinductive definition is the <em>largest</em> set closed <em>backward</em> under its defining rules.</p> <p>What does this mean? For an inductive definition, we can think of the set as starting from <script type="math/tex">\varnothing</script> and iteratively adding elements according to the rules. For a coinductive definition, we can think of the set as starting from the set of all possible objects (even infinite ones), and iterative removing objects that contradict the rules.</p> <p>If we use the same rules that inductive defined <script type="math/tex">S</script> above, the coinductively defined set <script type="math/tex">S'</script> is the largest set such that <script type="math/tex">\epsilon \in S'</script> and that if <script type="math/tex">\sigma s \in S</script>, then <script type="math/tex">s \in S</script> (and <script type="math/tex">\sigma \in \Sigma</script>). Here, the backward closure goes from the conclusion to the premises, the opposite of the forward closure. The set of finite strings, <script type="math/tex">S</script>, is included in <script type="math/tex">S'</script>. But we also have some new strings in <script type="math/tex">S'</script>, the infinitely long strings. Consider the string <script type="math/tex">s = aaaaaa \dots</script>, where <script type="math/tex">a \in \Sigma</script>. We cannot construct it using the base case, but it doesn’t lead to a contradiction either, since if <script type="math/tex">s = aaaaa \dots \in S</script>, taking off the first <script type="math/tex">a</script> results in the same infinite string <script type="math/tex">s</script>, and <script type="math/tex">s \in S</script> as desired.</p> <p>The proof tree for <script type="math/tex">s</script> is infinite, and looks like the following:</p> <script type="math/tex; mode=display">\large \frac{a \in \Sigma \quad \frac{ a \in \Sigma \quad \frac{ \cdots }{ aaa \dots \in S' } }{ aaa \dots \in S' }}{ aaa \dots \in S' }</script> <p>While objects of inductive definitions require finite derivations, objects of coinductive definitions can have infinite derivations.</p> <h2 id="proof-principles">Proof principles</h2> <p>For the following, I will skip over some (many) details.</p> <p>The function <script type="math/tex">F</script> can be thought of as the set of rules for a given (co)inductive definition. <script type="math/tex">F(X)</script> is the set of conclusions obtained after applying the rules using <script type="math/tex">X</script> as the set of premises.</p> <p>Recall that an inductive definition is the least fixed point of a set of rules, and that a coinductive definition is the greatest fixed point. Now here is a specialization of the Knaster–Tarski fixpoint theorem:</p> <p><strong>Theorem:</strong> <br /> The least fixed point of <script type="math/tex">F = \mu F = \bigcap \{ X \mid F(X) \subseteq X \}</script>. <br /> The greatest fixed point of <script type="math/tex">F = \nu F = \bigcup \{ X \mid X \subseteq F(X) \}</script>.</p> <p><script type="math/tex">F(X) \subseteq X</script> captures the meaning of the informal “closed forwards” definition from earlier. Given a set <script type="math/tex">T</script> where the premises <script type="math/tex">X \subseteq T</script>, we can apply <em>all</em> rules in the “forwards” direction, obtaining the set of conclusions <script type="math/tex">F(T)</script> which are also in <script type="math/tex">T</script>: <script type="math/tex">F(X) \subseteq X \subseteq T</script>.</p> <p>Dually, <script type="math/tex">X \subseteq F(X)</script> captures the meaning of “closed backwards”. Given a set <script type="math/tex">T</script> where the conclusions <script type="math/tex">F(X) \subseteq T</script> from some set of premises <script type="math/tex">X</script>, we can apply <em>some</em> rule for each <script type="math/tex">t \in F(X)</script> in the “backwards” direction, obtaining the set of premises <script type="math/tex">X</script> which are also in <script type="math/tex">T</script>: <script type="math/tex">X \subseteq F(X) \subseteq T</script>.</p> <p>Simple corollaries of the fixpoint theorem gives us proof principles for inductive and coinductive definitions:</p> <p><strong>Lemma (Induction Principle):</strong> <br /> If <script type="math/tex">F(X) \subseteq X</script>, then <script type="math/tex">\mu F \subseteq X</script>. <br /> <strong>Lemma (Coinduction Principle):</strong> <br /> If <script type="math/tex">X \subseteq F(X)</script>, then <script type="math/tex">X \subseteq \nu F</script>.</p> <h3 id="proof-by-induction">Proof by induction</h3> <p>Using the induction principle, we can show that every element of a inductively defined set satisfies some condition, by showing that the condition is preserved for each rule of the definition.</p> <p>We can derive the more familiar principle of mathematical induction using this. Let <script type="math/tex">F(X) = \{ 0 \} \cup \{ 1 + x \mid x \in X \}</script>. This is the set of rules for the natural numbers. It may be more familiar if I write it as the following:</p> <script type="math/tex; mode=display">\frac{}{0 \in \mathbb{N}} \qquad \frac{n \in \mathbb{N}}{1 + n \in \mathbb{N}}</script> <p>Then to prove some fact about the natural numbers, we just need to show that it is preserved when applying these rules in the forwards direction. For example, we will show that <script type="math/tex">1 + 2 + \dots + n = \frac{n(n+1)}{2}</script> is true for all natural numbers. Let’s take <script type="math/tex">X = \{ n \in \mathbb{N} \mid 1 + 2 + \dots + n = \frac{n(n+1)}{2} \}</script>. Then we will prove that <script type="math/tex">\mu F = \mathbb{N} \subseteq X</script>. This is exactly the conclusion of the Induction Principle, so we need to show that <script type="math/tex">F(X) \subseteq X</script>.</p> <p>An element of <script type="math/tex">F(X)</script> can either be <script type="math/tex">0</script> (the base case), which we can easily verify is in <script type="math/tex">X</script>, or <script type="math/tex">1 + n</script> (the inductive case) where <script type="math/tex">n \in X</script> (the inductive hypothesis). This should look familiar. Some fiddling will show that the second case is true as well, and we are done! <script type="math/tex">\Box</script></p> <h3 id="proof-by-coinduction">Proof by coinduction</h3> <p>Dually, using the coinduction principle, we can show that an element is in the coinductively defined set.</p> <p>Using just <script type="math/tex">S'</script>, our only coinductively defined set so far, would not be very interesting, since it would involve only the membership proofs we saw earlier. Let’s make another coinductive definition, this time a relation on elements of <script type="math/tex">S'</script>: let <script type="math/tex">F(X) = \{ (\epsilon, \epsilon) \} \cup \{ (\sigma_1 s_1, \sigma_2 s_2) \mid \sigma_1 \le \sigma_2 \land (s_1, s_2) \in X \}</script>, where <script type="math/tex">\le</script> is some ordering on the alphabet (the usual one on the English alphabet, for instance). Can you tell what relation this defines? Let’s write down the inference rules:</p> <script type="math/tex; mode=display">\frac{}{\epsilon \leqslant \epsilon} \qquad \frac{\sigma_1 \le \sigma_2 \qquad s_1 \leqslant s_2}{\sigma_1 s_1 \leqslant \sigma_2 s_2}</script> <p>The notation should help: <script type="math/tex">\nu F</script> is the lexicographic ordering relation on our (possibly) infinite strings, displayed here as <script type="math/tex">\leqslant</script>.</p> <p>Now we can prove that some strings are related by this relation. For an example, we will show <script type="math/tex">aaaa \dots \leqslant baaaa \dots</script>. Note that these are infinitely long strings.</p> <p>Using the coinduction principle, we just need to show that <script type="math/tex">(aaaa \dots, baaaa \dots)</script> is in some set of pairs of strings that is closed backwards under <script type="math/tex">F</script>. Let’s try the singleton set <script type="math/tex">X = \{(aaaa \dots, baaaa \dots)\}</script> first. Then <script type="math/tex">F(X) = \{ (\epsilon, \epsilon) \} \cup \{ (\sigma_1 aaaa \dots, \sigma_2 baaaa \dots) \mid \sigma_1 \le \sigma_2 \}</script>. But then <script type="math/tex">X \not \subseteq F(X)</script>, since the second string of every pair in <script type="math/tex">F(X)</script> has a <script type="math/tex">b</script> as the second symbol.</p> <p><script type="math/tex">X</script> is our “coinductive hypothesis”. Like how during induction we sometimes have to strengthen the inductive hypothesis, here we have to strengthen the coinductive hypothesis by making it bigger.</p> <p>Recall the “backwards closed” intuition. We want to show that by applying <em>some</em> rule “backwards”, we obtain something still in <script type="math/tex">X</script>. If we start with <script type="math/tex">(aaaa \dots, baaaa \dots)</script>, we can only apply the second rule, stripping off the first symbol of each string. <script type="math/tex">a \le b</script>, so that premise is fine, and we just need to show that <script type="math/tex">(aaaa \dots, aaaa \dots) \in X</script> now. It looks like we need to grow <script type="math/tex">X</script> by adding this new pair to it, strengthening the coinductive hypothesis.</p> <p>Now <script type="math/tex">X = \{ (aaaa \dots, baaaa \dots), (aaaa \dots, aaaa \dots) \}</script>, and <script type="math/tex">F(X) = \{ (\epsilon, \epsilon) \} \cup \\ \{ (\sigma_1 aaaa \dots, \sigma_2 baaaa \dots) \mid \sigma_1 \le \sigma_2 \} \cup \\ \{ (\sigma_1 aaaa \dots, \sigma_2 aaaa \dots) \mid \sigma_1 \le \sigma_2 \}</script></p> <p>Let’s check that <script type="math/tex">X \subseteq F(X)</script>. <br /> <script type="math/tex">(aaaa \dots, baaaa \dots) = (a\cdot aaaa \dots, b\cdot aaaa \dots)</script>, and <script type="math/tex">a \le b</script>. <br /> <script type="math/tex">(aaaa \dots, aaaa \dots) = (a\cdot aaaa \dots, a\cdot aaaa \dots)</script>, and <script type="math/tex">a \le a</script>.</p> <p>And since <script type="math/tex">(aaaa \dots, baaaa \dots) \in X</script>, we’re done! <script type="math/tex">\Box</script></p> <h2 id="conclusion">Conclusion</h2> <p>Recently I’ve been working on <a href="https://github.com/DeepSpec/InteractionTrees/">Interaction Trees</a>, a library that provides a coinductive data structure for reasoning about interactive programs in Coq. Coinduction is less convenient than induction in Coq. For example, in the coinductive proof above the “coinductive hypothesis” included exactly the conclusion we were trying to prove. When doing the proof informally, we know we must apply one of the rules backwards and only then can we apply the coinductive hypothesis.</p> <p>Doing this in a proof assistant like Coq is more complex. Using “vanilla” Coq, it will allow you to apply the coinductive hypothesis immediately, and then complain that you got it wrong when you try to finish the proof. The <a href="https://github.com/snu-sf/paco">paco</a> library solves this problem, but more complex reasoning quickly gets complex, which is why I started learning more about the theory behind coinduction.</p> <p>I find it really intriguing how (relatively) new coinduction is and how useful it has become. There’s been a lot of work recently on areas related to coinduction, and I’m excited to do more work in this area.</p> <h2 id="resources">Resources</h2> <p>I first encountered coinduction in <a href="https://www.cis.upenn.edu/~bcpierce/tapl/">Types and Programming Languages</a> by Benjamin C. Pierce, where they are introduced to talk about the metatheory of recursive types. While I wouldn’t recommend reading this if you’re <em>just</em> interested in coinduction, it serves as an excellent introduction to programming languages and type systems.</p> <p><a href="http://www.cs.unibo.it/~sangio/IntroBook.html">Introduction to Bisimulation and Coinduction</a> by Davide Sangiorgi is a very accessible textbook that goes into detail about all of this and more. It cleared up a lot of questions I had about coinduction, and helped me understand it more rigorously.</p>Paul HeCoinduction is the mathematical dual to an indispensible mathematical tool: induction. While mathematical induction has been known for thousands of years, coinduction has only been studied for a few decades. It is still primarily used in computer science, from which it originated in the field of concurrency theory. Coinduction allows us to define circular or infinite objects (such as streams, lists that can be infinitely long), and to prove things about them.Welcome2019-03-06T00:00:00-05:002019-03-06T00:00:00-05:00https://paulhe.com/2019/03/06/first-post<p>Welcome to my blog! This is the first (official) post, hopefully of many.</p> <h2 id="goals">Goals</h2> <p>I decided to start this blog because I rarely write anything longer than a sentence at a time, which seems like a useful thing to practice for a PhD student. I’ve also always felt pretty weak at communicating about research or technical stuff (not to mention just in general). Hopefully this will help me with these things, as well as improve my understanding of the technical material I’ll be writing about.</p> <p>I plan to write about various technical things I encounter during my research work. These will probably be things related to functional programming and programming language theory.</p> <h2 id="website">Website</h2> <p>Let me tell you a bit about the inner workings of the website, which I spent (and will continue to spend) a lot of time on—instead of writing posts. The site is static, hosted on <a href="https://pages.github.com/">Github Pages</a>, and is generated by <a href="https://jekyllrb.com/">Jekyll</a>. I don’t want to handle any complexity related to hosting, so a static website seems fine to me.</p> <p>However if you look at the bottom of the page, you’ll see something less standard: an ugly hacked together comment system. I originally tried using <a href="https://disqus.com/">Disqus</a>, which was really quite nice and easy to use. You can see an example of it on my blog <a href="/2019/02/28/first-post.html">here</a>.</p> <p>I preferred something more lightweight though, and also something I controlled entirely. Here’s an <a href="/2019/02/28/second-post.html">example</a> of what my current solution looks like. You can even embed html (what’s sanitization?)!</p> <p>I’m using <a href="https://staticman.net/">Staticman</a> to display user-generated content. When a comment is submitted, it goes through the Staticman web service, which creates a pull request on my website’s Github repo to add the comment as a text file, to be included in the (updated) static site.</p> <p>I think this is super cool. No databases or anything to deal with! Staticman was pretty nice to use, though the documentation is a little out of date. Originally it was run as a single public instance, and due to the number of users it had it was hitting the rate limit for the Github API (see <a href="https://github.com/eduardoboucas/staticman/issues/243">this Github issue</a> for details). The developer then updated Staticman to be a Github app, so each user would get their own instance and thus their own API quota. However, this was fairly recent (Dec 2018) and the documentation wasn’t updated to reflect this, so it took me a few hours to get it working.</p> <h2 id="to-come">To Come</h2> <p>I’ll be working on updating the site a bit more to add a navigation menu (<strong>Edit</strong> Mar 7: Done!), less ugly comments (<strong>Edit</strong> Mar 8: Arguably done!), and so on. For my research I’ll be spending most of my time in the next few weeks on coinduction, a very cool proof technique, and I hope to write something introductory about it soon.</p>Paul HeWelcome to my blog! This is the first (official) post, hopefully of many.Blogging Like a Hacker2019-02-28T00:00:00-05:002019-02-28T00:00:00-05:00https://paulhe.com/2019/02/28/first-post<h1 id="test">Test</h1> <div id="disqus_thread"></div> <script> /** * RECOMMENDED CONFIGURATION VARIABLES: EDIT AND UNCOMMENT THE SECTION BELOW TO INSERT DYNAMIC VALUES FROM YOUR PLATFORM OR CMS. * LEARN WHY DEFINING THESE VARIABLES IS IMPORTANT: https://disqus.com/admin/universalcode/#configuration-variables */ /* var disqus_config = function () { this.page.url = https://paulhe.com/2019/02/28/first-post.html this.page.identifier = /2019/02/28/first-post }; */ (function() { // DON'T EDIT BELOW THIS LINE var d = document, s = d.createElement('script'); s.src = 'https://paulhe.disqus.com/embed.js'; s.setAttribute('data-timestamp', +new Date()); (d.head || d.body).appendChild(s); })(); </script> <noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript" rel="nofollow">comments powered by Disqus.</a></noscript> <script id="dsq-count-scr" src="//paulhe.disqus.com/count.js" async=""></script>Paul HeTestComments2019-02-28T00:00:00-05:002019-02-28T00:00:00-05:00https://paulhe.com/2019/02/28/second-postPaul He