Elixir, etc. - Dialyzer Talk (2019.1119)

Written by Rich Morin.

Contents: (hide) (show)

Path:  AreasContentOverviews

Precis:  a gentle introduction to Dialyzer and Typespecs

I gave a short, introductory talk (A Gentle Introduction to Dialyzer and Typespecs) at the Erlang & Elixir SF meetup on November 19, 2019. This item contains the slides for the talk. See Elixir, etc. - Typespecs for broader and deeper coverage, links to resources, etc.



(hide) (show)

<h4>Title</h4> <h4>A Gentle Introduction to Dialyzer and Typespecs</h4> <p>by Rich Morin (rdm@cfcl.com)<br> Canta Forda Computer Laboratory</p> <p>Erlang &amp; Elixir SF<br> November 19, 2019</p> <h4>Biography</h4> <ul> <li>started programming in 1969 (on Wylbur, at SF State) </li> <li>wrote lots of asm, AWK, C, Fortran, Perl, sed, sh, … </li> <li>dabbled in Clojure, COBOL, Icon, Prolog, Python, … </li> <li>mostly programs in Ruby and Elixir these days<br><br> </li> <li>columnist for MacTech, SunExpert, and Unix Review </li> <li>edited several collections of open source software </li> <li>consults on design, development, and documentation </li> <li>volunteers on several disability-related projects </li> </ul> <h4>Motivation</h4> <p>Using Dialyzer and Typespecs will let your project:</p> <ul> <li>document how internal interfaces use data types </li> <li>verify this documentation against the code base </li> <li>enhance visibility of data types and structures </li> <li>encourage careful analysis of internal interfaces </li> <li>make the code base more robust and maintainable </li> </ul> <h4>Background</h4> <p>Dialyzer is a workaround, motivated by a conundrum:</p> <ul> <li>Erlang, like Prolog, is dynamically typed. </li> <li>Static type checking might reveal coding errors. </li> <li>Adding static typing could break existing code. </li> </ul> <p>Changing the language definition was not an option:</p> <ul> <li>Many lines of Erlang code were in production. </li> <li>An optional, unintrusive approach was needed. </li> <li>Success typing was the “least worst option”. </li> </ul> <h4>Theory</h4> <p>All BEAM-based languages are strongly typed:</p> <ul> <li>Type errors can be detected in various ways. </li> <li>Some type errors can be detected statically. </li> <li>Others can slip through into running code. </li> </ul> <p>Dialyzer compares function calls to definitions:</p> <ul> <li>Determine the union of its type signatures. </li> <li>Calls which conform to a signature “succeed”. </li> <li>The remainder are reported as possible errors. </li> </ul> <h4>Practice</h4> <p>In summary, Dialyzer:</p> <ul> <li>performs program-wide detection of type conflicts </li> <li>uses both type inference and explicit specifications </li> </ul> <p>The diagnostics are generally reliable, but imperfect:</p> <ul> <li>Dialyzer very seldom produces false positives. </li> <li>Reports can be incomplete, opaque, and voluminous. </li> <li>Interpreting the results can take a bit of practice. </li> <li>Loose or missing specs may prevent error detection. </li> </ul> <h4>Syntax: @spec</h4> <pre><code>@doc &quot;Create a simple Map data structure.&quot; @spec foo(atom, number) :: map def foo(key, val), do: %{ key =&gt; val }</code></pre> <ul> <li>Some types are namespaced (e.g., <code class="inline">Regex.t</code>). </li> <li>Specs can be written using multiple lines. </li> <li>Aliases and guards can shorten type names. </li> <li>Custom types can (and should!) be defined. </li> </ul> <h4>Syntax: @type</h4> <pre><code>@typedoc &quot;Atoms and strings are common map keys.&quot; @type map_key :: atom | st @typep st :: String.t</code></pre> <ul> <li><code class="inline">@type</code> and <code class="inline">@typep</code> define derived types. </li> <li>They are generally placed in <code class="inline">*.ex</code> files. </li> <li>Attribute placement order isn’t critical. </li> <li><code class="inline">@typedoc</code> can be used to document <code class="inline">@type</code>. </li> </ul> <h4>Tooling</h4> <p>Jeremy Huffman’s Dialyxir package:</p> <ul> <li>is the most popular Mix wrapper (~1M downloads) </li> <li>provides “Mix tasks to simplify use of Dialyzer” </li> <li>translates Dialyzer’s reports from Erlang to Elixir </li> </ul> <p>Dialyxir’s default output is rather verbose, so I use it as follows:</p> <pre><code>mix dialyzer --quiet</code></pre> <h4>Usage: Aliases</h4> <p>Long, complicated specs can be hard to read:</p> <pre><code>@spec do_files(String.t, (String.t -&gt; {String.t, InfoToml.Types.schema})) :: [InfoToml.Types.schema]</code></pre> <p>So, use aliases and guards to keep things tidy:</p> <pre><code>alias InfoToml.Types, as: ITT @spec do_files(st, (st -&gt; {st, sc})) :: [sc] when sc: ITT.schema, st: String.t</code></pre> <h4>Usage: Modules</h4> <p>The libraries in Pete’s Alley are structured as follows:</p> <pre><code>.../apps/foo/lib/ | foo/ | | _foo_t.ex # Foo.Types (eg, @type, @typedoc, typep) | | bar.ex # Foo.Bar (eg, @doc, @spec, def, defp) | | ... | foo.ex # Foo (eg, @doc, defdelegate)</code></pre> <ul> <li>app-specific types are defined in <code class="inline">_foo_t.ex</code> </li> <li>external interfaces are defined in <code class="inline">foo.ex</code> </li> <li>implementations are defined in <code class="inline">foo/*.ex</code> </li> </ul> <h4>Advice: Iterate</h4> <p>I recommend using a cautious, iterative approach:</p> <ul> <li>Before writing <em>any</em> specs, run Dialyzer. </li> <li>Clean up problems until it goes quiet. </li> <li>Add (or tighten) a few specs at a time.<br><br> </li> <li>Fix (or work around) reported problems. </li> <li>If need be, loosen or even disable specs. </li> <li>Rinse, repeat… </li> </ul> <h4>Advice: Refine</h4> <p>Start with a working spec, then refine (i.e., tighten) it:</p> <pre><code>@spec get_kv_info(any) :: map @spec get_kv_info(atom) :: map @spec get_kv_info(atom) :: %{ any =&gt; any } @spec get_kv_info(atom) :: %{ atom =&gt; any } @spec get_kv_info(atom) :: %{ atom =&gt; list | map } @spec get_kv_info(atom) :: ITT.kv_info</code></pre> <h4>Advice: Reformat</h4> <p>Dialyzer’s report (and spec) syntax can be challenging to read:</p> <pre><code>Foo.bar( _inp_map :: %{atom() | binary() =&gt; %{atom() | binary() =&gt; binary() | map()}}, ... )</code></pre> <p>It can help a lot to reformat things for readability:</p> <pre><code>Foo.bar(_inp_map :: %{ atom() | binary() =&gt; %{ atom() | binary() =&gt; binary() | map() } }, ... )</code></pre> <h4>Advice: Code Smells</h4> <p>Stay alert for assorted code smells. For example:</p> <ul> <li><code class="inline">any</code> may indicate a sub-optimal interface design </li> <li><code class="inline">list</code>, <code class="inline">map</code>, and <code class="inline">tuple</code> can hide a lot of complexity </li> <li><code class="inline">number</code> should be promoted to <code class="inline">float</code>, <code class="inline">integer</code>, etc. </li> </ul> <p>Keep your code tidy and DRY:</p> <ul> <li>Aliases and guards can make specs easier to read. </li> <li>Complex literals may warrant definition as types. </li> </ul> <h4>Example: map_key</h4> <p>Define a type in <code class="inline">Common.Types</code> (<code class="inline">_common_t.ex</code>):</p> <pre><code>@typedoc &quot;Constrain map keys to atoms and strings.&quot; @type map_key :: atom | st @typep st :: String.t</code></pre> <p>Use the type in <code class="inline">Common.Maps</code> (<code class="inline">maps.ex</code>):</p> <pre><code>alias Common.Types, as: CT @doc &quot;Get the maximum value of a non-empty map.&quot; @spec get_map_max( %{required(CT.map_key) =&gt; any} ) :: any def get_map_max(map), do: map |&gt; Map.values |&gt; Enum.max()</code></pre>