Posted in 2022
Likelihood, greed and temperature in sequence learning
- 28 May 2022
Imagine we have a model \(D(w)\) of a dynamical system with states \(s \in S,\) that is parametrized by some weight \(w \in W\). Each state \(s\) comes with a set \(N(s) \subset S\) of neighbors and an associated energy function \(E(s'|s,w) \in \mathbb{R}\) that assigns an energy to each neighbor \(s' \in N(s)\).
Parametric typeclasses aid generalization in program synthesis
- 22 January 2022
We envision programming being done in top-down fashion. The human describes the goal (e.g. sorting), and the machine reduces it to smaller subgoals based on well-known heuristics (e.g. divide and conquer). The easier subgoals could even be fulfilled automatically. This top-down heuristics approach will be more amenable to machine learning. See my Topos Institute talk for more info.
Information topos theory - motivation
- 22 January 2022
Relative information (also known as the Kullback-Leibler divergence) is an important fundamental concept in statistical learning and information theory.