Lab 03 - Logits and Softmax¶

Goal: convert logits to probabilities and inspect temperature effects.

Info: Temperature as control

Temperature affects routing reliability. Lower temperature is deterministic; higher introduces variance. Set by use case, not globally.

Info: Confidence margin

Small margin between top-1 and top-2 signals uncertainty. Use margin for escalation, not just top-1 probability.

In [1]:

import math

def softmax(logits, temperature=1.0):
    scaled = [x / temperature for x in logits]
    shift = max(scaled)
    exps = [math.exp(x - shift) for x in scaled]
    total = sum(exps)
    return [v / total for v in exps]

logits = [3.2, 2.8, 0.5]
print("Base probabilities:", [round(p, 4) for p in softmax(logits)])

import math def softmax(logits, temperature=1.0): scaled = [x / temperature for x in logits] shift = max(scaled) exps = [math.exp(x - shift) for x in scaled] total = sum(exps) return [v / total for v in exps] logits = [3.2, 2.8, 0.5] print("Base probabilities:", [round(p, 4) for p in softmax(logits)])

Base probabilities: [0.5755, 0.3858, 0.0387]

In [2]:

for t in [0.7, 1.0, 1.5]:
    probs = softmax(logits, temperature=t)
    print(f"T={t}:", [round(p, 4) for p in probs])

for t in [0.7, 1.0, 1.5]: probs = softmax(logits, temperature=t) print(f"T={t}:", [round(p, 4) for p in probs])

T=0.7: [0.6306, 0.3561, 0.0133]
T=1.0: [0.5755, 0.3858, 0.0387]
T=1.5: [0.5178, 0.3966, 0.0856]

In [3]:

probs = softmax(logits, temperature=1.0)
sorted_probs = sorted(probs, reverse=True)
margin = sorted_probs[0] - sorted_probs[1]
print("Top-1 confidence margin:", round(margin, 4))

probs = softmax(logits, temperature=1.0) sorted_probs = sorted(probs, reverse=True) margin = sorted_probs[0] - sorted_probs[1] print("Top-1 confidence margin:", round(margin, 4))

Top-1 confidence margin: 0.1897

Visualization: temperature comparison¶

This chart compares how class probabilities change as temperature changes.

In [4]:

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import math

def softmax(logits, temperature=1.0):
    scaled = [x / temperature for x in logits]
    shift = max(scaled)
    exps = [math.exp(x - shift) for x in scaled]
    total = sum(exps)
    return [v / total for v in exps]

logits = [3.2, 2.8, 0.5]
temps = [0.5, 0.7, 1.0, 1.3, 1.6, 2.0]
series = [softmax(logits, t) for t in temps]
for idx in range(3):
    plt.plot(temps, [row[idx] for row in series], marker='o', label=f'class_{idx+1}')
plt.title('Softmax probability vs temperature')
plt.xlabel('Temperature')
plt.ylabel('Probability')
plt.ylim(0, 1)
plt.grid(alpha=0.3)
plt.legend()
plt.tight_layout()
plt.show()

import matplotlib matplotlib.use('Agg') import matplotlib.pyplot as plt import math def softmax(logits, temperature=1.0): scaled = [x / temperature for x in logits] shift = max(scaled) exps = [math.exp(x - shift) for x in scaled] total = sum(exps) return [v / total for v in exps] logits = [3.2, 2.8, 0.5] temps = [0.5, 0.7, 1.0, 1.3, 1.6, 2.0] series = [softmax(logits, t) for t in temps] for idx in range(3): plt.plot(temps, [row[idx] for row in series], marker='o', label=f'class_{idx+1}') plt.title('Softmax probability vs temperature') plt.xlabel('Temperature') plt.ylabel('Probability') plt.ylim(0, 1) plt.grid(alpha=0.3) plt.legend() plt.tight_layout() plt.show()

/var/folders/xr/skp57kbd3txdt0r466c2q94h0000gp/T/ipykernel_33900/2072163618.py:25: UserWarning: FigureCanvasAgg is non-interactive, and thus cannot be shown
  plt.show()