Introduction E9F5FC

Questions FFFFC0

• View
• Edit
• History
• Print

Sources

• Chemical kinetics

Kinds of Energy

Kinectic energy of the motion of the system as a whole

Potential energy of the position of the system as a whole

Work {$W$} (implies a persistent system, thus an improving system)(learning - the improvement in the explanation of variance - overall, this improvement)(the structure that makes for a good model)(the structure that enables variance to be explained - the channeling of useful work)(structure that channels activity)

• energy transferred to or from an object via the application of force along a displacement
• work done {$P\Delta V $} on a system by its surroundings or {$-P\Delta V $} by a system on it surroundings where {$P$} is pressure and {$V$} is volume change.

Heat {$Q$} (akin to Statistical Variance or Entropy in an informational model)(recurring activity which evokes structure)

• Supplied to a system or withdrawn from a system
• Thermal energy transferred between systems due to a temperature difference
• Transfer is by mechanisms other than thermodynamic work or transfer of matter
• Transfer can include conduction, radiation, and friction
• The kinetic energy of vibrating and colliding atoms in a substance

Internal energy {$U$}

• {$\Delta U=Q-W$}
• a mathematical abstraction
• conserved by the First Law of Thermodynamics
• turning heat (or entropy) to zero gives the maximum likelihood "absolute zero" solution

Enthalpy {$H = U + PV$}

• Sum of a thermodynamic system's internal energy and the product of its pressure and volume.

Gibbs free energy, also known as free enthalpy

• {$G = H − TS$}
• A measure of thermodynamic potential (especially in chemistry) when it is convenient for applications that occur at constant pressure and constant temperature.
• Excludes the {$p\;dV$} work needed to make room for additional molecules produced.
• The free energy relevant for chemistry.

Helmholtz free energy

• A thermodynamic potential that measures the useful work obtainable from a closed thermodynamic system at a constant temperature (isothermal).
• The work content. Its change is equal to the amount of reversible work done on, or obtainable from, a system at constant temperature.
• {$A\equiv U-TS$}
• Includes
• Its decrease is the maximum amount of work which can be done by a system at constant temperature.
• It can increase at most by the amount of work done on a system isothermally.
• Is proportional to the logarithm of the partition function for the canonical ensemble in statistical mechanics.
• The free energy relevant for physics.

Thermodynamic free energy

• Refers to Gibbs free energy or Helmholtz free energy
• The change in the free energy is the maximum amount of work that the system can perform in a process at constant temperature, and its sign indicates whether the process is thermodynamically favorable or forbidden.

Variational free energy (active inference)(equation 2.5)

• Evidence lower bound of (an observation) ELBO
• function of belief distribution and observation, regarding the real world
• but not studying the transfer of work between system and world
• functional f(Q,Y) with regard to data and belief
• evidence lower bound Blbl
• if you try to maximize your reward doesn't work well
• minimizing surprise works better
• maximizing evidence is minimizing surprise
• maximum likelihood model = minimum surprise model
• can bound surprise from the top

Fundamental thermodynamic relation

• {$\textrm{d}U=T\textrm{d}S-P\textrm{d}V$}
• {$\textrm{d}H=T\textrm{d}S+V\textrm{d}P$}
• {$\textrm{d}F=-S\textrm{d}T-P\textrm{d}V$}
• {$\textrm{d}G=-S\textrm{d}T+V\textrm{d}P$}

Kinetic energy within the system

Potential energy within the system

Partition function

Principles

Free Energy Principle

Free energy principle

• The dynamics of physical systems minimise a quantity known as surprisal (which is just the negative log probability of some outcome); or equivalently, its variational upper bound, called (variational) free energy.
• A theoretical framework suggesting that the brain reduces surprise or uncertainty by making predictions based on internal models and updating them using sensory input. It highlights the brain's objective of aligning its internal model with the external world to enhance prediction accuracy. This principle integrates Bayesian inference with active inference, where actions are guided by predictions and sensory feedback refines them.

Evidence lower bound, ELBO, sometimes called variational lower bound

• A useful lower bound on the log-likelihood of some observed data.

Maximum Entropy Principle

Least Action Principle

Inference

Bayesian inference

• {$H$} stands for any hypothesis whose probability may be affected by data (called evidence below). Often there are competing hypotheses, and the task is to determine which is the most probable.
• {$P(H)$}, the prior probability, is the estimate of the probability of the hypothesis {$H$} before the data {$E$}, the current evidence, is observed.
• {$E$}, the evidence, corresponds to new data that were not used in computing the prior probability.
• {$P(H\mid E)$}, the posterior probability, is the probability of {$H$} given {$E$}, i.e., after {$E$} is observed. This is what we want to know: the probability of a hypothesis given the observed evidence.
• {$P(E\mid H)$} is the probability of observing {$E$} given {$H$} and is called the likelihood. As a function of {$E$} with {$H$} fixed, it indicates the compatibility of the evidence with the given hypothesis.
• The likelihood function is a function of the evidence, {$E$}, while the posterior probability is a function of the hypothesis, {$H$}.
• {$P(E)$} is sometimes termed the marginal likelihood or "model evidence". This factor is the same for all possible hypotheses being considered (as is evident from the fact that the hypothesis {$H$} does not appear anywhere in the symbol, unlike for all the other factors) and hence does not factor into determining the relative probabilities of different hypotheses.
• {$P(E)>0$} (Else one has {$0/0$}.)
• Why is asserted by the hypothesis {$H$} and its probability {$P(H)$}, the prior probability
• How is asserted by the posterior probability, the probability of {$H$} given {$E$}
• What is asserted by the likelihood, the probability of {$E$} given {$H$}
• Whether is asserted by the evidence {$E$} and its probability {$P(E)$}, the marginal likelihood

Notes

Energy dissipation is the flow of the arrow of time.

Statistically, we want the model to fit better.

The arrow of time in statistics training is learning.

First mind = thermodynamic embodied free energy

Second mind = predictive anticipatory. (generative model) variational free energy

Third mind = cybernetic. "I3 control what I2 probably do I1". I control the distribution of beliefs that my behavior is sampled from.

Heat is embedded in internal energy, is embedded in enthalpy, is embedded in free energy, yielding a three-cycle.