, , ,

Storing Complex Objects in localStorage

Posted by

A practical, senior-dev guide to serializing complex data (Dates, Maps/Sets, BigInt, circular refs), compression, versioning/migrations, quotas, and rock-solid utilities for the real world.

A practical, senior-dev guide to serializing complex data (Dates, Maps/Sets, BigInt, circular refs), compression, versioning/migrations, quotas, and rock-solid utilities for the real world.

Introduction

localStorage is deceptively simple:

localStorage.setItem("user", JSON.stringify({ id: 1 }));

…and you’re done, right? Not quite. Real apps need to persist complex objects: Dates, Maps/Sets, BigInts, nested graphs, and sometimes circular references. You’ll hit issues like:

  • JSON.stringify silently drops undefined and functions.
  • BigInt throws.
  • Map/Set become {} or [].
  • Circular graphs crash serialization.
  • Exceeding the ~5MB quota throws a DOMException.
  • Safari Private Mode or disabled storage can make localStorage unusable.
  • You deploy a new version and your old data doesn’t match the new schema.

This post is your end-to-end playbook for storing complex objects in localStorage—what to store, how to serialize safely, compressing to stay under the limit, versioning and migrations, security pitfalls, performance, and production-ready utilities you can paste into your codebase.


1) localStorage 101 (and its constraints)

  • API: Synchronous key/value store of strings only.
  • setItem(key, string), getItem(key) -> string|null, removeItem(key), clear(), key(index).
  • Scope: Per-origin (scheme + host + port). Shared across tabs in the same origin.
  • Capacity: Roughly 5–10 MB per origin (varies by browser; plan for ~5 MB).
  • Sync & blocking: Calls block the main thread. Heavy reads/writes can jank the UI.
  • Persistence: Data survives page reloads, but users can clear it anytime; some contexts (private modes, storage disabled/denied) may throw.
  • Security: Accessible by any script on the origin. Never store secrets (tokens, passwords, PII) unless you’ve rigorously mitigated XSS and understand the risk.

Rule of thumb: localStorage is great for preferences, UI state, small caches, feature flags, last-opened doc metadata. For larger, structured, or frequently updated data, prefer IndexedDB (async, larger, structured) and use localStorage as a tiny fast cache or feature toggle store.


2) Serializing complex objects: baselines & pitfalls

The baseline: JSON

const json = JSON.stringify(value);
// …
const value = JSON.parse(json);

Pitfalls with plain JSON:

  • Drops: undefined, functions, symbols.
  • Converts Date → ISO string.
  • NaN, Infinitynull.
  • Map/Set{}/[] (lose type).
  • BigInt → ❌ throws.
  • Circular refs → ❌ throws.

You’ll need custom serializers or a library.


3) Safer serialization strategies

3.1 JSON + replacer/ reviver (DIY control)

Use a replacer to transform types during stringify, and a reviver to undo on parse.

// serializer.ts
const TYPE = {
DATE: "__DATE__",
MAP: "__MAP__",
SET: "__SET__",
BIGINT: "__BIGINT__",
CIRCULAR: "[Circular]"
};

export function serialize(value) {
const seen = new WeakSet();
return JSON.stringify(value, (_k, v) => {
if (typeof v === "bigint") return { __type: TYPE.BIGINT, value: v.toString() };

if (v instanceof Date) return { __type: TYPE.DATE, value: v.toISOString() };
if (v instanceof Map) return { __type: TYPE.MAP, value: [...v.entries()] };
if (v instanceof Set) return { __type: TYPE.SET, value: [...v.values()] };

if (typeof v === "object" && v !== null) {
if (seen.has(v)) return TYPE.CIRCULAR; // choose to drop/mark cycles
seen.add(v);
}
return v;
});
}

export function deserialize(json) {
return JSON.parse(json, (_k, v) => {
if (v && v.__type === "__DATE__") return new Date(v.value);
if (v && v.__type === "__MAP__") return new Map(v.value);
if (v && v.__type === "__SET__") return new Set(v.value);
if (v && v.__type === "__BIGINT__") return BigInt(v.value);
if (v === TYPE.CIRCULAR) return undefined; // or keep as string marker
return v;
});
}

Notes:

  • Decide how to handle circular refs. Either drop them, mark them, or choose a library that supports them losslessly.
  • This approach gives you explicit control and stays dependency-light.

3.2 Use a robust library

  • superjson – Serializes Dates, Maps/Sets, BigInt, custom classes with minimal effort. Great for app state and API payloads.
  • flatted – JSON-compatible circular reference support.
  • json-stable-stringify – Deterministic key order (useful for hashing, cache keys).

If you don’t need special types, plain JSON is fine; otherwise, reach for superjson for convenience or DIY if you want total control.


4) Compress to stretch the 5MB limit

Text JSON can get chunky. Compression helps — especially for repeated keys and large arrays.

  • lz-string: popular, browser-friendly, UTF-16 safe.
  • compressToUTF16 / decompressFromUTF16
  • compressToBase64 / decompressFromBase64
  • pako (gzip/deflate) is heavier and binary-oriented (needs base64/utf8 wrapping).

Example:

import { compressToUTF16, decompressFromUTF16 } from "lz-string";
import { serialize, deserialize } from "./serializer";

export function saveCompressed(key, value) {
const json = serialize(value);
const payload = compressToUTF16(json);
localStorage.setItem(key, payload);
}

export function loadCompressed(key) {
const payload = localStorage.getItem(key);
if (payload == null) return null;
const json = decompressFromUTF16(payload);
return deserialize(json);
}

Tip: Compression and (de)serialization are CPU work — debounce or offload to a Worker if the payload is big or frequently updated.


5) Namespacing, versioning, and migrations (don’t skip)

You will change schemas. Future-proof it.

5.1 Namespacing keys

Use a prefix to avoid collisions and to scope features:

app:v1:user
app:v1:cart
app:v1:settings

5.2 Version stamp & metadata envelope

Store an envelope with version, ts, and maybe ttl.

type Envelope<T> = {
v: number; // schema version
ts: number; // saved at (ms)
ttl?: number; // optional ms-to-live
data: T;
};

function wrap<T>(v: number, data: T, ttl?: number): Envelope<T> {
return { v, ts: Date.now(), ttl, data };
}

5.3 Migration pipeline

When loading, read version and migrate step-by-step.

type UserV1 = { name: string };
type UserV2 = { firstName: string; lastName: string };

function migrateUser(v: number, data: any): UserV2 {
switch (v) {
case 1: {
const [firstName, ...rest] = data.name.split(" ");
return { firstName, lastName: rest.join(" ") };
}
case 2:
return data as UserV2;
default:
// unknown version: clear or try best-effort upgrade
return { firstName: "", lastName: "" };
}
}

5.4 TTL (time-to-live) and staleness

Implement optional expiry:

function isExpired(envelope: Envelope<any>) {
return envelope.ttl != null && Date.now() - envelope.ts > envelope.ttl;
}

On load, if expired → drop and return null.


6) Production-ready storage wrapper (copy–paste)

A small, safe API that handles detection, errors, compression, versioning, TTL, and fallbacks.

// storage.ts
import { compressToUTF16, decompressFromUTF16 } from "lz-string";
import { serialize, deserialize } from "./serializer";

function hasLocalStorage(): boolean {
try {
const k = "__probe__";
localStorage.setItem(k, "1");
localStorage.removeItem(k);
return true;
} catch {
return false;
}
}

// Fallback in-memory store for environments without localStorage
const memoryStore = new Map<string, string>();

function setRaw(key: string, value: string) {
if (hasLocalStorage()) localStorage.setItem(key, value);
else memoryStore.set(key, value);
}
function getRaw(key: string) {
return hasLocalStorage() ? localStorage.getItem(key) : memoryStore.get(key) ?? null;
}
function removeRaw(key: string) {
if (hasLocalStorage()) localStorage.removeItem(key);
else memoryStore.delete(key);
}

type Envelope<T> = { v: number; ts: number; ttl?: number; data: T };

export function save<T>(key: string, version: number, data: T, ttl?: number) {
try {
const envelope: Envelope<T> = { v: version, ts: Date.now(), ttl, data };
const json = serialize(envelope);
const payload = compressToUTF16(json);
setRaw(key, payload);
return true;
} catch (e) {
// QuotaExceededError or serialization error
console.warn(`[storage] save failed for ${key}`, e);
// Optional: LRU eviction or specific keys to clear
return false;
}
}

export function load<T>(key: string, migrate: (v: number, data: any) => T): T | null {
const payload = getRaw(key);
if (payload == null) return null;

try {
const json = decompressFromUTF16(payload);
const env = deserialize(json) as Envelope<any>;
if (!env || typeof env.v !== "number" || typeof env.ts !== "number") {
// corrupted or old format
removeRaw(key);
return null;
}
if (env.ttl != null && Date.now() - env.ts > env.ttl) {
removeRaw(key);
return null;
}
// migrate if needed
const data = migrate(env.v, env.data);
// Optionally re-save with latest version after migration
return data as T;
} catch (e) {
console.warn(`[storage] load failed for ${key}`, e);
removeRaw(key);
return null;
}
}

export function clearKey(key: string) {
removeRaw(key);
}

What you get out of the box:

  • Works when localStorage is blocked (falls back to memory).
  • Compression + custom serialization (Dates, Map/Set, BigInt support).
  • TTL & corruption handling.
  • Migration hook to evolve schemas safely.

7) React hook: ergonomic useLocalStorage

A tiny hook that debounces writes and handles parse failures.

// useLocalStorage.tsx
import { useEffect, useRef, useState } from "react";
import { save, load } from "./storage";

export function useLocalStorage<T>(
key: string,
initial: T,
version: number,
migrate: (v: number, data: any) => T,
debounceMs = 250
) {
const [state, setState] = useState<T>(() => load<T>(key, migrate) ?? initial);
const timer = useRef<number | null>(null);

useEffect(() => {
if (timer.current) window.clearTimeout(timer.current);
timer.current = window.setTimeout(() => {
save<T>(key, version, state);
}, debounceMs);
return () => {
if (timer.current) window.clearTimeout(timer.current);
};
}, [key, version, state, debounceMs]);

// Sync across tabs
useEffect(() => {
function onStorage(e: StorageEvent) {
if (e.key !== key) return;
const next = load<T>(key, migrate);
if (next !== null) setState(next);
}
window.addEventListener("storage", onStorage);
return () => window.removeEventListener("storage", onStorage);
}, [key]);

return [state, setState] as const;
}

Usage:

const migrateSettings = (v: number, data: any) => {
if (v === 1) return { theme: data.theme ?? "light", lang: "en" };
if (v === 2) return data;
return { theme: "light", lang: "en" };
};

const [settings, setSettings] = useLocalStorage(
"app:v2:settings",
{ theme: "light", lang: "en" },
2,
migrateSettings
);

8) Handling errors and quotas (without crashing UX)

8.1 Detecting support and failures

  • Wrap setItem/getItem in try/catch. Safari Private Mode may throw even for reads/writes.
  • On failure, fall back (memory store) and show a graceful message (e.g., “Your changes will only persist for this session”).

8.2 Quota management

When setItem throws QuotaExceededError (DOMException code 22):

  • Compress bigger payloads.
  • Evict least-important keys first (keep a small LRU list or evict known cache keys).
  • Split large payloads across multiple keys (chunking — see §9.3).
  • Consider IndexedDB for large datasets.

9) Patterns for large or complex payloads

9.1 Deterministic order for stable diffs/hashes

If you use JSON as a cache key, ensure stable ordering:

function stableStringify(obj) {
const keys = Object.keys(obj).sort();
return JSON.stringify(obj, keys);
}

9.2 Change detection before write

Avoid unnecessary writes (and quota churn):

function shallowEqual(a, b) {
if (a === b) return true;
const ka = Object.keys(a), kb = Object.keys(b);
if (ka.length !== kb.length) return false;
for (const k of ka) if (a[k] !== b[k]) return false;
return true;
}

Only save when changed.

9.3 Chunking large values

If you must store >5MB (not recommended), split:

function setChunked(key, str, chunkSize = 1024 * 1000) { // ~1MB per chunk
const chunks = Math.ceil(str.length / chunkSize);
localStorage.setItem(`${key}:chunks`, String(chunks));
for (let i = 0; i < chunks; i++) {
localStorage.setItem(`${key}:${i}`, str.slice(i * chunkSize, (i + 1) * chunkSize));
}
}
function getChunked(key) {
const chunks = Number(localStorage.getItem(`${key}:chunks`) || 0);
if (!chunks) return null;
let out = "";
for (let i = 0; i < chunks; i++) out += localStorage.getItem(`${key}:${i}`) ?? "";
return out;
}

Caution: This increases the chance of partial writes and fragmentation. Prefer IndexedDB for large records.


10) Security considerations (seriously important)

  • Never store secrets (auth tokens, refresh tokens, API keys, sensitive PII). If an attacker injects JS (XSS), localStorage is fully exposed.
  • Encrypting before storage helps only if the encryption key isn’t in JS (which it usually is). Treat encryption more as obfuscation client-side.
  • Sanitize untrusted input before saving to avoid prototype pollution if you later merge it into objects.
  • If you must cache server content, consider adding a domain-scoped allowlist of keys and size caps to prevent storage abuse.

11) Performance & UX

  • localStorage is sync: large reads/writes block. Use sparingly in render paths.
  • Debounce saves; batch on exit events (beforeunload, visibilitychange) if needed.
  • For heavy serialization/compression, use a Web Worker to keep the UI responsive.
  • Keep values small and focused. Persist IDs or compact shapes, not giant graphs.

12) Testing & observability

  • Unit test serialization: Dates round-trip, Maps/Sets preserved, BigInt stringified and rehydrated.
  • Corruption tests: Truncate payloads, feed invalid JSON, ensure the loader recovers (clears and returns defaults).
  • Quota tests: Mock setItem to throw, assert fallback path is used.
  • E2E: Verify cross-tab sync via the storage event.

13) Real-world examples you’ll reuse

13.1 Persisting a cache with TTL and migration

type CacheV2 = { items: Array<{ id: string; ts: number }> };

const KEY = "app:v2:cache";

const VERSION = 2;
function migrateCache(v: number, data: any): CacheV2 {
if (v === 1) return { items: data.items.map((x: any) => ({ id: x.id, ts: Date.now() })) };
if (v === 2) return data as CacheV2;
return { items: [] };
}

// save:
save<CacheV2>(KEY, VERSION, { items: [{ id: "a", ts: Date.now() }] }, /*ttl*/ 1000 * 60 * 60);

// load:
const cache = load<CacheV2>(KEY, migrateCache); // null if expired/corrupted

13.2 Map/Set round-trip

const settings = new Map([["theme", "dark"], ["lang", "en"]]);
save("app:v1:settings", 1, { settings });
const restored = load("app:v1:settings", (v, d) => d)?.settings; // Map

13.3 BigInt counters

const counters = { total: 12345678901234567890n };
save("app:v1:stats", 1, counters);
const stats = load("app:v1:stats", (_v, d) => d); // BigInt restored

14) When to use IndexedDB instead

Choose IndexedDB if:

  • You need >5MB or many large items.
  • You need async access to avoid blocking.
  • You need queries, indexes, partial updates, or transactional semantics.
  • You want to store blobs/files.

Great wrappers: idb, Dexie, RxDB. Use localStorage as a fast flag store or for a small boot cache, and keep heavy lifting in IndexedDB.


15) Common mistakes (and quick fixes)

  1. Blind JSON everywhere → Dates become strings, Map/Set lost.
     Fix: Use a serializer (DIY or superjson).
  2. Assuming 5MB is “plenty” → Quota errors in production.
     Fix: Compress, prune, or move to IndexedDB; catch and handle quota errors.
  3. Writing on every keystroke → Jank & quota churn.
     Fix: Debounce; only save on meaningful checkpoints.
  4. No versioning/migrations → Deployed schema mismatch breaks users.
     Fix: Envelope with v, ts, optional ttl, and a migration function.
  5. Assuming localStorage is always available → Safari Private Mode crash.
     Fix: Feature detect; fall back to in-memory and inform the user.
  6. Storing secrets → XSS exfiltration.
     Fix: Don’t. Keep short-lived tokens in memory; use httpOnly cookies server-side.
  7. Monolithic key → Big, fragile payloads.
     Fix: Split by feature; avoid megavalues.

Conclusion

Persisting to localStorage can be reliable and ergonomic if you treat it like a serialization contract:

  • Use a robust serializer (DIY replacer/reviver or superjson) to handle Dates, Map/Set, BigInt, and circular graphs.
  • Compress to stretch the quota; debounce to avoid blocking.
  • Build a versioned envelope with TTL and migrations so data survives refactors.
  • Guard against missing/blocked storage and recover from corruption.
  • Keep security in mind: no secrets in localStorage.

Pro Tip: Keep localStorage small and strategic. For anything bigger or frequently changing, put it in IndexedDB and treat localStorage as your fast boot cache + feature flags.


Call to Action (CTA)

What’s the trickiest localStorage bug you’ve had—quota explosions, Safari surprises, or mangled Dates?
Drop your story (and fix) in the comments. If this helped, bookmark it and share with a teammate before they JSON.stringify a 20MB object.

Leave a Reply

Your email address will not be published. Required fields are marked *