I’ve been learning more about Information Science lately and I created a fun widget to visualize information content in text.

This was a fun toy to build and is useful for explaining information content to others.

Input

Alphabet Size:

Symbol (x)	P(x)	I(x) bits

Explanation

The outputs include a table showing the number of bits required to encode a character in the alphabet, and a heatmap of the message.

Red characters have the most information content.
Lighter/pale characters have less information content.
White characters have 0 information content; although you will never see a character with 0 bits of information.
You can hover over each character in the heatmap to see its information content.

P(x) is the probability that a token/character occurs in the message. Its value ranges from 0 (exclusive) to 1 (inclusive).

I(x) is the information content/surprise that a token/character carries in the message. Its value is measured in bits and is always greater than zero.

How I Built It

You will notice that the tool is interactive and reactive, but those of you curious enough to peek under the hood will be shocked to see that I don’t use a single line of React or any NPM packages!

I found that programming a small project like this without a large framework is refreshing.

# Entropy Calculator

Input

Explanation

How I Built It

# Web Piano

# Finding How PyTorch Samples from a Multinomial Distribution