Starting These Notes

August 15, 2026

For a long time I told myself I’d start writing once I had something finished to write about. That bar turned out to be a moving target — so this is me starting anyway. The plan: short notes on papers I’m reading, longer derivations when a topic deserves it, and occasional blog posts.

This first entry doubles as a rendering test.

Math

Inline math works with single dollars: the TD(0) update is $V(S_t) \leftarrow V(S_t) + \alpha \left[ R_{t+1} + \gamma V(S_{t+1}) - V(S_t) \right]$.

Display math uses double dollars:

\[J(\theta) = \mathbb{E}_{\tau \sim \pi_\theta} \left[ \sum_{t=0}^{T} \gamma^t r(s_t, a_t) \right]\]

Numbered AMS equations work too, and can be referenced:

\[\begin{equation} \nabla_\theta J(\theta) = \mathbb{E}_{\pi_\theta} \left[ \nabla_\theta \log \pi_\theta(a \mid s) \, Q^{\pi_\theta}(s, a) \right] \label{eq:pg} \end{equation}\]

Equation $\eqref{eq:pg}$ is the policy gradient theorem.

Code

def td0_update(V, s, r, s_next, alpha=0.1, gamma=0.99):
    V[s] += alpha * (r + gamma * V[s_next] - V[s])
    return V

How a new note gets added

Drop a Markdown file in _posts/ named YYYY-MM-DD-slug.md, push, done. The sidebar and the index page update themselves.