Your JavaScript runs on one thread, but the waiting doesn’t. Node hands network I/O to the operating system (epoll, kqueue, IOCP) and file-system work to libuv’s thread pool, then runs your callback on the main thread when the result is ready. The main thread never waits; it only runs callbacks.

A loop inside libuv that cycles through phases: timers, pending callbacks, poll (I/O), check (setImmediate) and close callbacks. Each phase has a queue, and each callback runs to completion on the call stack before the next one starts.

Synchronous code first. Then the whole nextTick queue, then the whole promise microtask queue. Only then does the loop move on to setTimeout callbacks in the timers phase and setImmediate in the check phase. Both microtask queues are drained again after every single callback.

Node starts an ES module from inside a promise job, so when its top-level code finishes, V8 is still draining the microtask queue and runs the promise callbacks you queued before Node gets to the nextTick queue. In CommonJS the nextTick queue goes first. The examples on this line are CommonJS: run scenario 1 as an .mjs file and C and D swap places.

From the main module it isn’t guaranteed; it depends on how quickly the loop starts. Inside an I/O callback, setImmediate always wins, because the check phase comes straight after the poll phase.

It runs work that has no non-blocking OS API: most fs calls, dns.lookup, and CPU-heavy crypto and zlib functions. It has 4 threads by default, set by UV_THREADPOOL_SIZE. Network sockets don’t use it.

Any long synchronous work: a huge JSON.parse, sync fs or crypto calls, heavy loops, a regex with catastrophic backtracking. While it runs, no request, timer or promise can move. Split the work into chunks, stream the data, or move it to worker_threads.

A timer’s delay is a minimum, not a promise. The callback runs in the first timers phase after the delay has passed, so a long callback ahead of it delays it, exactly like the loop done at 100 ms scenario on this line.

Yes. The nextTick queue is drained completely before the loop continues, so a callback that keeps scheduling another nextTick stops all I/O. setImmediate yields to I/O instead.

worker_threads run JavaScript in parallel inside one process and can share memory with SharedArrayBuffer; use them for CPU-heavy tasks. cluster, or several processes behind a load balancer, runs separate Node processes so a server can use every core.

Macrotasks are the callbacks the loop picks up phase by phase: timers, I/O callbacks, setImmediate, close events. Microtasks are promise reactions and queueMicrotask callbacks, plus Node’s own nextTick queue. After every single macrotask callback, Node drains both microtask queues completely before it runs the next one.

queueMicrotask puts a callback on V8’s microtask queue, the same queue promise callbacks use. process.nextTick uses a separate Node queue that, in CommonJS, runs before it. Prefer queueMicrotask: it works the same in browsers and Node, and it can’t jump ahead of promises you already queued.

The loop asks the OS for finished I/O and runs those callbacks. If there is nothing to run, it waits there for new I/O, but only until the nearest timer is due. If a setImmediate is queued, it doesn’t wait at all and moves straight on to the check phase.

An async function runs synchronously up to its first await. There it returns a pending promise to its caller, and the rest of the function is resumed later as a microtask once the awaited value settles. Nothing runs in parallel: await just splits your function into callbacks.

Since Node 15, an unhandled rejection emits unhandledRejection, and if nothing handles it, it is thrown as an uncaught exception and the process exits with an error. In an uncaughtException handler, log and exit: the process is in an unknown state, so let a supervisor restart it instead of carrying on.

No. The promise, callback and sync versions of fs do the same work; the async ones run it on libuv’s thread pool and only differ in how you get the result. Network I/O, including fetch and http, is the part that goes to the OS without the pool.

dns.lookup, which http and net use by default, calls getaddrinfo on the thread pool. With only 4 threads, a few slow lookups can occupy the whole pool and fs, crypto and zlib work queues up behind them. Raise UV_THREADPOOL_SIZE, cache lookups, or use dns.resolve, which queries the network directly.

Streams move data in chunks instead of loading it all at once, so a 10 GB file uses the same memory as a 10 KB one, and processing starts with the first chunk. There are four kinds: Readable, Writable, Duplex (both, like a socket) and Transform (a Duplex that changes the data, like gzip).

It is what happens when the producer is faster than the consumer. write() returns false once the internal buffer passes highWaterMark, and you should stop writing until 'drain'. Ignore it and the buffer grows until the process runs out of memory. pipe() and pipeline() handle it for you.

.pipe() doesn’t forward errors: if one stream fails, the others stay open and leak file descriptors or sockets. pipeline() destroys every stream when any of them fails and gives you one callback, or one promise from stream/promises, for success or error.

No. emit() calls every listener synchronously, in the order they were added, on the current call stack, and returns only when all of them have finished. A slow listener blocks the emitter. And an 'error' event with no listener throws.

Measure the delay: perf_hooks.monitorEventLoopDelay() gives a histogram of how late the loop is, and performance.eventLoopUtilization() shows how busy it is. To find the cause, take a CPU profile with --cpu-prof or the inspector and look for long synchronous frames in the flame graph.

setInterval fires on schedule whether or not the previous async run has finished, so slow requests overlap and pile up. Scheduling the next setTimeout at the end of each run guarantees a gap between runs and makes back-off easy.

Each worker has its own V8 isolate and event loop, so nothing is shared by default. postMessage copies data with the structured clone algorithm; listing an ArrayBuffer as transferable moves it instead of copying. Only a SharedArrayBuffer is truly shared, and you coordinate access to it with Atomics.

spawn starts a program and streams its output. exec runs a command through a shell and buffers the output, so it’s open to shell injection and limited by maxBuffer. execFile is exec without the shell. fork starts another Node process with an IPC channel for send() and 'message'.

On SIGTERM, call server.close() to stop accepting connections and let in-flight requests finish; close idle keep-alive connections (closeIdleConnections()); then close database pools and queues, and exit. Add a hard timeout so a stuck request can’t hold the process forever.

It carries context, such as a request ID or the current user, through every callback and await that follows from a request, without passing it as an argument. It is the async equivalent of thread-local storage, and it is how loggers and tracing tools tag everything done for one request.

With an AbortController. Pass its signal to fetch, fs.readFile, the promise versions of the timers, events.once or your own functions, then call abort(). AbortSignal.timeout(ms) gives you a signal that aborts on its own.

The loop keeps running while anything still holds it open: a timer, an open socket or server, a database pool, a child process. Close what you opened. For a handle that shouldn’t keep the process alive on its own, such as a background interval, call .unref().

It runs nothing: the operations started when you created the promises, and Promise.all only waits for them. They wait concurrently, and it rejects as soon as one rejects, without cancelling the others. Use allSettled to get every result, any for the first success, race for the first to settle, and a limiter such as p-limit when you have thousands of tasks.

One list object is created on the heap and both names refer to it. Assignment never copies: it binds a name to an object, and the list’s reference count goes to 2.

Neither. It passes references to objects by value, often called call by sharing. A function can mutate an object you pass in and you’ll see the change, but rebinding the parameter inside the function doesn’t touch your variable.

Mainly by reference counting: when an object’s count drops to zero it’s freed immediately. A generational cyclic garbage collector finds groups of objects that only reference each other, which refcounting alone can never free.

is checks identity: are these the same object? == checks equality through __eq__. Use is for singletons like None, and == for everything else.

CPython caches the integers from -5 to 256, so those are always the same objects. Whether other equal numbers or strings share an object depends on interning and constant folding, an implementation detail you should never rely on.

Not directly. del removes a name (or a container item) and decrements the reference count. The object is freed only when its count reaches zero, or later by the garbage collector if it’s part of a cycle.

Small objects come from pymalloc, which takes memory from the OS in large arenas and can only give an arena back when every block in it is free. A few surviving objects keep whole arenas alive, as the pymalloc demo on this line shows. Python reuses that memory; it just doesn’t hand it back.

Look for references that live too long: module-level caches, an unbounded functools.lru_cache, closures and callbacks that capture big objects, and globals. tracemalloc shows which lines allocated the memory that stays, and gc.get_referrers() shows who is holding an object.

For caches and back-references that shouldn’t keep an object alive. A weak reference doesn’t increase the reference count, so the object can still be freed, after which the weak reference returns None.

Every CPython object has a counter of how many references point at it: names, list slots, dict values, attributes. Binding a name adds one, dropping it subtracts one, and at zero the object is freed immediately. sys.getrefcount(x) shows it, one higher than you expect because the call itself holds a reference.

Since Python 3.12 (PEP 683), objects such as None, True, False, small ints and many interned strings have a fixed reference count that never changes and are never freed. Skipping those counter writes saves work, keeps memory pages shared after fork, and makes sharing them between threads and interpreters safe.

Reference counting can’t free cycles. If a list contains itself, or two objects point at each other, their counts never reach zero even when nothing else can reach them. The cycle collector finds such groups by subtracting the references they hold to each other and frees the ones left with no outside references.

The collector sorts container objects by age. New objects start in the youngest generation, which is scanned often because most objects die young; survivors are promoted to older generations that are scanned less and less often. Thresholds from gc.get_threshold() decide when each one runs.

Rarely. Reference counting frees almost everything, and the collector already runs on its own. Disabling it is safe only if your code never creates cycles, otherwise those leak. The one common tuning is gc.freeze() before forking workers, so the collector doesn’t touch, and copy, pages the children share.

It’s a finalizer that runs when an object is about to be freed. When that happens depends on reference counts, cycles and the Python implementation, so it’s a bad place for closing files or connections, and exceptions raised in it are only printed. Use a context manager (with) or weakref.finalize instead.

CPython’s allocator for small objects, 512 bytes or less. It takes big arenas from the OS, splits them into pools, and each pool hands out blocks of one size class. Creating and freeing small objects then costs almost nothing, because most requests never reach the system allocator.

It only measures the object itself. For a list that is the header plus the array of pointers, not the objects they point at. A list of a thousand strings reports a few kilobytes however long the strings are. To measure the whole structure, walk it, or compare tracemalloc snapshots before and after.

It replaces the per-instance __dict__ with fixed slots in the object itself. Each instance gets smaller and attribute access slightly faster, which matters when you create millions of them. The cost: you can’t add attributes that aren’t listed, and you need a __weakref__ slot for weak references.

Keeping a single copy of equal strings. CPython interns identifiers and many short literals automatically, and you can intern others with sys.intern(), which saves memory and makes dict lookups on repeated keys faster. It’s an optimisation, not a rule, so always compare strings with ==, never is.

After fork, parent and child share memory pages until one of them writes to a page. In CPython, simply reading an object changes its reference count, which is a write, so shared pages are copied one by one. Immortal objects and gc.freeze() reduce this.

A list is a resizable array of pointers to objects stored elsewhere. When it’s full, CPython allocates a bigger array with spare room, so most appends just fill a free slot and the occasional copy averages out: amortised O(1). Inserting or deleting at the front is O(n), because every pointer after it moves.

Each element is a pointer (8 bytes) to a full int object (28 bytes or more) with its own header. array.array('q') or a NumPy array stores the raw 8-byte values next to each other instead, several times smaller and much faster to loop over in C.

A generator produces values one at a time and keeps only its paused frame between them. sum(x * x for x in range(10**8)) needs constant memory, while the list comprehension version builds a list of a hundred million ints first. The catch: a generator can only be consumed once.

A view on the memory of another object that supports the buffer protocol, such as bytes, bytearray or array. Slicing a memoryview doesn’t copy, so parsing a large binary buffer piece by piece stays fast and uses no extra memory.

A nested function keeps every variable it uses from the enclosing scope in a cell. As long as the function exists, for example stored as a callback, those objects can’t be freed, even if you only needed one small value from a huge one. Copy out what you need before defining the function.

A traceback references every frame it passed through, and every frame references its local variables. Storing an exception, for example in a list of errors, keeps all of those locals alive. That’s why Python deletes the name after except E as e:; store str(e) if you only need the message.

It keeps a reference to every argument and result it stores. With maxsize=None, or @cache, it grows forever. On a method it also stores self, which keeps every instance alive. Give it a bounded size, and cache functions rather than methods.

An integer that is unique to an object for as long as it’s alive. In CPython it’s the memory address. Once an object is freed, a new object can get the same address, so two different objects created one after the other can report the same id.

Without the GIL, counter updates from different threads would race. Free-threaded builds use biased reference counting: the thread that owns an object updates a local count without atomics, and other threads update a shared count atomically. Some objects use deferred counting, and memory comes from mimalloc instead of pymalloc.

The Global Interpreter Lock is a mutex in CPython that lets only one thread run Python bytecode at a time within an interpreter. It keeps the interpreter’s internals, such as reference counts, safe without fine-grained locking.

Because a thread releases the GIL while it waits: blocking I/O on sockets and files, and time.sleep, let other threads run. Many C extensions, such as NumPy and hashlib on large inputs, release it during heavy computation too.

Use processes (multiprocessing or concurrent.futures.ProcessPoolExecutor) so each worker has its own interpreter and GIL, move the hot loop into native code that releases the GIL, or run on the free-threaded build.

No. Threads can switch between any two bytecodes, so count += 1 (load, add, store) can interleave and lose updates, which is the race you can step through on this line. Protect shared state with threading.Lock, or pass data between threads with queue.Queue.

A thread waiting for the GIL asks for it after the switch interval, 5 ms by default (sys.getswitchinterval()), and the holder releases it at the next safe point. A thread that starts blocking I/O releases it straight away.

A separate CPython build without the GIL (PEP 703), added experimentally in 3.13 as python3.13t and officially supported from 3.14 (PEP 779). Threads can then run Python code on several cores at once, at some cost to single-threaded speed. sys._is_gil_enabled() tells you which mode you’re in.

asyncio for many concurrent I/O tasks with async libraries on a single thread. Threads for I/O with blocking libraries, or a handful of background tasks. Processes for CPU-bound pure Python, paying for start-up time and for pickling data between them.

A CPU-bound thread keeps the GIL for a whole switch interval at a time, so I/O threads that only need it for a moment after each read queue up behind it. Their latency grows even though the total work barely changes, as the Mixed workload on this line shows.

No, it’s part of CPython, the reference implementation. Jython and IronPython never had one, PyPy has its own, and since 3.13 CPython has an optional free-threaded build without it. The language only says what your program means, not how threads are scheduled.

It’s a simple way to keep the interpreter’s internals safe: reference counts, the allocator and built-in objects can be updated without a lock of their own. That keeps single-threaded code fast and makes C extensions easy to write, at the price of only one thread running Python bytecode at a time.

During blocking I/O (sockets, files, time.sleep), and inside C code that chooses to release it, such as many NumPy operations, hashlib on large inputs and zlib. While executing Python bytecode, the running thread also gives it up when another thread has waited for the switch interval, 5 ms by default.

Without it, every reference count change would need an atomic operation or a lock, which slowed single-threaded code in earlier attempts, and many C extensions quietly relied on the GIL for safety. PEP 703 solved this with biased reference counting, per-object locks and immortal objects, and ships as an opt-in build.

No. It compiles to several steps: load the value, add one, store it back, and a thread switch between the load and the store loses an update. Single operations on built-ins, such as list.append, happen to be atomic in CPython, but don’t rely on it; protect shared state with a Lock.

A Lock deadlocks if the thread holding it tries to acquire it again. An RLock (reentrant lock) lets the owning thread acquire it several times, and it’s released after the same number of releases. Use RLock when a locked method calls another locked method of the same object.

Always acquire locks in the same global order, hold them for as short a time as possible, use with so they’re always released, and never call unknown code, such as callbacks, while holding one. A timeout on acquire turns a silent hang into an error you can see.

Through queue.Queue. It does its own locking, blocks consumers until items arrive and producers when it’s full, and turns shared state into messages. For signalling, use threading.Event; to limit how many threads use a resource, Semaphore.

They share the concurrent.futures API, so the choice is about the work. Threads for I/O-bound tasks: cheap to start, share memory. Processes for CPU-bound tasks: they get around the GIL, but the function and its arguments must be picklable, and each call pays to send data to another process.

Starting processes, pickling every argument and result to move it between them, and a separate copy of the interpreter and data in each one. How processes start depends on the platform: spawn on Windows and macOS, forkserver on Linux since 3.14 (before that, fork).

The child gets a copy of memory, but only the thread that called fork. A lock held by any other thread at that moment stays locked forever in the child, so the child can hang on its first log call or allocation. Python 3.12 and later warn about it; prefer spawn or forkserver.

Send messages through a multiprocessing.Queue or Pipe; the data is pickled and copied. For large arrays, use multiprocessing.shared_memory, which gives every process the same bytes without copying. A Manager shares Python objects through a server process, which is convenient but slow.

No. It runs one thread, and tasks take turns at each await. That handles thousands of waiting connections cheaply, but a single blocking call such as time.sleep or requests.get freezes every task. Move blocking work off the loop with asyncio.to_thread or run_in_executor.

Concurrency is dealing with many things at once: tasks make progress by taking turns. Parallelism is doing many things at once: tasks run at the same moment on different cores. Threads under the GIL and asyncio give you concurrency; processes, and free-threaded Python, give you parallelism.

Often, yes. NumPy releases the GIL inside many operations on large arrays, so several threads can run them on different cores. The Python lines between those calls still take turns, so the win depends on how much of the time is spent inside NumPy.

Several isolated Python interpreters in one process. Since 3.12 each can have its own GIL (PEP 684), and 3.14 adds the concurrent.interpreters module and InterpreterPoolExecutor. You get parallelism with less overhead than processes, but objects aren’t shared; data is passed between interpreters.

Call sys._is_gil_enabled() (3.13 and later). A free-threaded build, usually named python3.13t or python3.14t, runs without it, but it turns the GIL back on when it imports a C extension that isn’t marked safe, unless you force it off with PYTHON_GIL=0 or -X gil=0.

No. Built-in objects lock themselves internally, so a list won’t be corrupted, but counter += 1 and check-then-act code race exactly as before, only more often, because threads now really run at the same time. You still need locks, and single-threaded code runs somewhat slower in that build.

threading.local() gives every thread its own copy of some attributes, such as a database connection. contextvars does the same for asyncio tasks, which share one thread, and also works with threads. For request-scoped data in async code, use contextvars.

You can’t kill one from outside. Give it a threading.Event and have it check the event regularly and return. A daemon thread (daemon=True) doesn’t stop the interpreter from exiting, but it’s killed without cleanup, so open files and transactions may be left half done.

Seven primitives (undefined, null, boolean, number, bigint, string and symbol) plus objects. Arrays, functions, dates, maps and class instances are all objects.

Eight strings: "undefined", "boolean", "number", "bigint", "string", "symbol", "object" and "function". typeof null is "object" because of a historical bug, so test for null with === null and for arrays with Array.isArray().

Default to ===, which never converts types. == applies conversion rules: null and undefined only equal each other, booleans become numbers, a string compared with a number becomes a number, and objects are turned into primitives. The one common use of == is x == null, which checks for null or undefined.

No. Everything is passed by value, but the value of an object variable is a reference. A function can mutate an object you pass in, but reassigning the parameter doesn’t change your variable. Python works the same way.

Spread, Object.assign and slice() copy one level, so nested objects stay shared. structuredClone copies deeply, including Map, Set, Date and circular references, but it throws on functions, and class instances come back as plain objects.

Numbers are 64-bit IEEE 754 doubles, and 0.1, 0.2 and 0.3 have no exact binary form, so each is stored as the nearest double and the rounding errors don’t cancel out. Compare with a tolerance, or keep money in integer cents.

false, 0, -0, 0n, "", null, undefined and NaN (plus the legacy document.all in browsers). Everything else is truthy, including "0", "false", [] and {}.

2⁵³ − 1. Above it, not every integer can be represented, so a 64-bit ID from a database can silently change when parsed as a number. Keep such IDs as strings or BigInt.

undefined means a value was never set: a missing property, an unassigned variable, a function with no return. null is an explicit “no value” that you assign. JSON has null but no undefined, and JSON.stringify drops properties whose value is undefined.

A bug from the first JavaScript engine: values carried a type tag, objects had tag 0, and null was the null pointer, which also read as 0. Changing it now would break the web, so it stays. Test for null with x === null.

Array.isArray(x). typeof says "object", and instanceof Array fails for arrays from another realm, such as an iframe or a vm context, because each realm has its own Array constructor.

The IEEE 754 value for an undefined result, like 0 / 0 or Number("abc"). Its type is number, and it isn’t equal to anything, itself included, so x === NaN is always false. Use Number.isNaN(x); the global isNaN converts first, so isNaN("abc") is true.

In exactly two cases: Object.is(NaN, NaN) is true, and Object.is(0, -0) is false. Everything else matches ===. React uses it to decide whether state changed, and Map, Set and includes use a variant that treats 0 and -0 as equal.

When you write "abc".toUpperCase(), the engine uses a temporary String wrapper to find the method on String.prototype; the primitive itself doesn’t change. Never create wrappers yourself with new String or new Boolean: new Boolean(false) is an object, so it’s truthy.

It calls obj[Symbol.toPrimitive](hint) if it exists. Otherwise it tries valueOf() then toString() for a number hint, and the other way round for a string hint. That’s why [] + {} is "[object Object]": [] becomes "", {} becomes "[object Object]", and + joins strings.

An integer type of any size, written 123n, with typeof "bigint". Use it for IDs, money in minor units or anything beyond 2⁵³. You can’t mix it with numbers in arithmetic (1n + 1 throws a TypeError), it has no fractions, and JSON.stringify throws on it.

A unique value, mostly used as a property key that can’t clash with any other key, even one with the same description. for...in, Object.keys and JSON.stringify skip symbol keys. Built-in ones like Symbol.iterator let your objects plug into language features such as for...of.

No. const only stops you from rebinding the name; the object it points at can still change. Object.freeze stops changes to the object’s own properties, but it’s shallow: nested objects stay mutable unless you freeze them too.

Prefer structuredClone: it copies Date, Map, Set, typed arrays and circular references. The JSON round trip drops undefined and functions, turns dates into strings and NaN into null, and throws on cycles and BigInt. Neither copies functions or keeps class prototypes.

A small integer that V8 stores inside the pointer itself, tagged by its lowest bit, so it needs no allocation. Numbers that don’t fit, such as fractions, -0 or large integers, become a HeapNumber: a 64-bit float in a separate heap object. That’s why integer-heavy code is often faster.

V8 gives every object a hidden class, or shape, that records its properties and where they are stored. Objects built with the same properties in the same order share a shape, so property access can be cached and becomes very fast. Adding properties in different orders, or using delete, creates new shapes and slows that code down.

V8 tracks what an array holds: only small integers, only doubles, or anything, and whether it has holes. Packed small-integer arrays are the fastest. Transitions only go one way: after arr.push(1.5) or a hole like arr[100] = 1, that array never becomes fast again. new Array(n) starts out holey.

IEEE 754 floats keep a sign bit even for zero. 0 === -0 is true, but 1 / -0 is -Infinity and Object.is(0, -0) is false. It shows up when a small negative value rounds to zero, and it can tell you which direction a value approached zero from.

Store integer minor units such as cents, with BigInt if they can get large, or use a decimal library, and format with Intl.NumberFormat. toFixed is not a fix: it rounds the binary value, so (1.005).toFixed(2) is "1.00", because 1.005 is really 1.00499999...

parseInt reads digits until it hits something else, so parseInt("12px") is 12. Number converts the whole string or returns NaN, so Number("12px") is NaN, but Number("") is 0. Always give parseInt a radix, and never pass it numbers: parseInt(0.0000005) is 5.

For x == null, which is true only for null and undefined, so it catches both in one check. Many style guides allow exactly that one case and use === everywhere else.

a || b falls back to b for every falsy value, including 0, "" and false. a ?? b falls back only for null and undefined. For defaults such as count ?? 10 you want ??, so that a real 0 is kept.

Use a Map for dictionaries with dynamic keys: keys can be any value, including objects, it has size, iterates in insertion order and has no inherited keys like __proto__ to worry about. Object keys are always strings or symbols, and integer-like keys are listed first, in numeric order. Use objects for records with fixed fields.

Attaching data to objects you don’t own without keeping them alive. Keys must be objects, and once a key is unreachable anywhere else, the entry can be garbage collected. Because entries can disappear at any time, a WeakMap can’t be iterated and has no size. Typical uses: caches and private data per DOM node.

list, dict, set and bytearray, plus most classes you write. int, float, complex, bool, str, bytes, tuple, frozenset, range and None are immutable: every “change” builds a new object.

It needs a __hash__ that never changes during its lifetime, and objects that compare equal must hash equally. Only hashable objects can be dict keys or set members, which is why a list can’t be a key but a tuple of hashable items can.

They compare equal and have the same hash, so the dict treats them as one key. The first key object is kept and only the value is replaced, as the hashing stepper on this line shows.

It calls a.__add__(b). If that returns NotImplemented, it tries b.__radd__(a), and raises TypeError if both give up. If b’s type is a subclass of a’s type that overrides __radd__, b goes first.

Not for mutable types. += calls __iadd__ first, which extends a list in place, so every name for that list sees the change. Immutable types fall back to x = x + y and rebind only x. That’s also why pair[0] += [2] on a tuple both mutates the list and raises.

Default values are evaluated once, when the function is defined, so every call shares the same list. Use items=None and create a new list inside the function.

None is a singleton, so identity is the precise test. It’s also faster, and it can’t be fooled by a class whose __eq__ says yes to everything.

A list for an ordered collection you change, a tuple for a fixed record or a hashable key, a set for uniqueness and fast membership tests, and a dict for lookup by key. Reach for a frozenset when you need a set that is hashable.

list(x), x.copy(), x[:] and copy.copy() are shallow: the new container holds the same inner objects. copy.deepcopy() copies everything recursively.

+= first calls list.__iadd__, which extends the list inside the tuple in place, then tries to store the result back into t[0], which raises TypeError. The tuple’s slots can’t change, but the objects in them can. For the same reason, a tuple that contains a list isn’t hashable.

Objects that compare equal must have equal hashes, and the hash must not change while the object is in a set or dict. That’s why defining __eq__ sets __hash__ to None: Python makes the class unhashable until you define a hash based on the same fields. @dataclass(frozen=True) does both for you.

It’s a hash table. The key’s hash picks a slot in a small index array, collisions are resolved by probing other slots, and the index points into a compact array of entries kept in insertion order. Lookups, inserts and deletes are O(1) on average, and a hit is confirmed with is or == on the key.

Yes, insertion order is guaranteed since Python 3.7. OrderedDict is still useful: it has move_to_end, its equality checks take order into account, and it’s handy for LRU caches. Sets have no order at all.

Python cares what an object can do, not what class it is: anything with __iter__ can be looped over, anything with read() can act as a file. Code usually just tries the operation and handles the exception (EAFP). For type checkers, typing.Protocol describes such a shape without inheritance.

NotImplemented is a value you return from a binary method such as __add__ or __eq__ to say “I don’t know this type”, so Python tries the other operand’s method. NotImplementedError is an exception you raise in a method that subclasses must override. Raising the first or returning the second are both bugs.

For a + b, when a has no __add__ or it returns NotImplemented, Python tries b.__radd__(a). One exception: if b’s type is a subclass of a’s type and overrides __radd__, it goes first, so subclasses can take control. It’s how sum() works on custom types starting from 0.

No. Python ignores them when it runs your code; tools such as mypy and pyright check them before it runs. They are available at runtime through typing.get_type_hints() and annotationlib, which is how libraries like Pydantic and FastAPI validate data. Since 3.14 annotations are evaluated lazily, only when something asks for them.

No, ints have arbitrary precision and grow as needed; sys.maxsize is the largest container size, not the largest int. Big ints just get slower. Since 3.11, converting an int with more than 4300 digits to or from a string raises ValueError by default, to prevent denial-of-service attacks.

// rounds down, towards minus infinity, so -7 // 2 is -4, and % takes the sign of the divisor: -7 % 2 is 1. JavaScript, C and Java truncate towards zero instead. int(-3.5) truncates too, giving -3.

Python rounds halves to the nearest even number (banker’s rounding), so round(2.5) is 2 and round(3.5) is 4; that avoids a bias when you add many rounded values. round(2.675, 2) gives 2.67 because 2.675 can’t be stored exactly. For money, use decimal.Decimal.

str is text, a sequence of Unicode code points. bytes is raw 8-bit data from files, sockets or hashes. Python 3 never converts between them implicitly: encode() text into bytes and decode() bytes into text at the edges of your program, naming the encoding, usually UTF-8.

len counts code points, not what you see. “é” can be one code point (U+00E9) or two: “e” plus a combining accent. Normalise with unicodedata.normalize("NFC", s) before comparing or counting. Internally, CPython stores each string with 1, 2 or 4 bytes per character, depending on its widest one.

Strings are immutable, so each += can create a new string and copy everything so far, which is O(n²) overall. CPython sometimes extends in place, but other implementations don’t, and you can’t rely on it. Collect the parts in a list and call "".join(parts) once, or use io.StringIO.

isinstance accepts subclasses and abstract base classes such as collections.abc.Mapping, which is almost always what you want. It has one classic surprise: bool is a subclass of int, so isinstance(True, int) is true and True + True is 2. Use type(x) is T only when subclasses must be rejected.

A list checks every element in turn: O(n). Sets and dicts hash x and look in one slot: O(1) on average. Turning a list into a set before checking thousands of values in a loop is one of the easiest big speed-ups. x in range(...) is O(1) too: it’s just arithmetic.

namedtuple: an immutable, lightweight tuple with field names. dataclass: a normal class with generated __init__, __repr__ and __eq__, mutable or frozen, with defaults and methods. TypedDict: type hints for dicts that stay plain dicts, such as JSON payloads. A plain dict: truly dynamic keys.

Yes. Python’s sort is stable, so items with equal keys keep their order, and you can sort by several keys in several passes, the least important first. It’s O(n log n) and very fast on data that is already partly sorted. It only needs <; mixing types that can’t be compared, like None and int, raises TypeError.

It calls __bool__ if the class defines it, otherwise __len__, and treats anything else as true. So None, False, zero of every numeric type and empty containers are falsy. Watch out for if not x: when 0 or an empty list is a valid value; write if x is None: instead.

An iterable, like a list, can give you a fresh iterator with iter(), as often as you like. An iterator remembers its position, returns the next item from __next__ and is used up after one pass. Generators and file objects are iterators, which is why looping over one a second time yields nothing.

It runs a loop. The text so far is tokenised, passed through the network, and the network outputs a score (logit) for every token in its vocabulary. Softmax turns the scores into probabilities, a decoding rule picks one token, and that token is appended to the input for the next pass. It stops at a stop token or at the token limit.

A token is a unit from a fixed vocabulary learned by the tokenizer, usually with byte-pair encoding: common words become one token, rarer words split into pieces, and anything can fall back to bytes. Characters would make sequences very long; whole words would need a huge vocabulary and still miss new words. Subwords balance the two. In English a token is roughly three quarters of a word, but numbers, code and many other languages need more tokens.

A number that divides the logits before the softmax: softmax(logits / T). Below 1 it sharpens the distribution so likely tokens dominate; above 1 it flattens it so unlikely tokens get drawn more often. T = 0 is treated as greedy decoding. It changes how the model picks, not what it knows.

A fixed-length list of numbers that represents a piece of text (or an image, or code) so that similar meanings get vectors that point in similar directions. You get it from an embedding model, and compare embeddings with cosine similarity. It powers semantic search, clustering, deduplication and the retrieval step of RAG.

Fluent output that isn’t supported by facts or by the provided sources. The model is trained to produce likely text, not true text, so when it lacks a fact it produces a plausible one. Training data rarely says “I don’t know”, many evaluations reward guessing, knowledge has gaps and a cut-off, and sampling can pick a wrong token that later tokens then build on.

Ground the model with retrieved sources and require citations, tell it that “I don’t know” is acceptable, and lower the temperature for factual tasks. Use tools for facts and arithmetic, ask for structured output and validate it in code, and measure with evals on known answers. Where errors are expensive, add a checking step or a human review.

Temperature reshapes the whole distribution. Top-k keeps only the k most likely tokens. Top-p (nucleus sampling) keeps the smallest set of most likely tokens whose probabilities add up to at least p, so the number kept adapts: few when the model is confident, many when it isn’t. After a cut, the survivors are renormalised and one is sampled.

Greedy (or a low temperature) for extraction, classification, code and anything with one right answer, because you want the most likely output and repeatability. Sampling with a moderate temperature for creative writing, brainstorming or several varied candidates. Greedy decoding can get stuck in repetitive loops on long open-ended text.

Greedy decoding has no randomness, but the arithmetic isn’t perfectly repeatable. GPU floating-point addition isn’t associative, and batching with other requests changes the order of operations, so two nearly tied logits can swap. One different token changes everything after it. Providers treat a seed as best effort, so design for small variations.

The maximum number of tokens the model can process in one call, counting the system prompt, the conversation, retrieved documents, tool results and the generated answer. Anything beyond it must be truncated, summarised or retrieved when needed. Bigger windows cost more and models don’t use every part of a long context equally well.

APIs bill per input and output token, output usually at a higher price. Input is processed in parallel (prefill), which sets the time to first token; output is generated one token at a time, so total latency grows with answer length. Shorter prompts, prompt caching for repeated prefixes, smaller models and capped output lengths all help.

For each position, attention computes how relevant every earlier position is (by comparing a query vector with key vectors), then takes a weighted mix of their value vectors. That is how the word “is” can pull in “France” and “capital”. Many attention heads run in parallel in every layer, and a causal mask stops positions from seeing the future.

During generation, each layer’s keys and values for all earlier tokens are stored instead of recomputed, so each new token only computes its own position. It makes decoding much faster but uses GPU memory that grows with context length and batch size, which is a main limit on long contexts and on how many requests a server can handle at once.

The dot product of two vectors divided by the product of their lengths: the cosine of the angle between them, from −1 to 1. It compares direction and ignores length, which matches how embedding models encode meaning. If vectors are normalised to length 1, cosine equals the dot product, which is cheaper to compute.

Use the metric the embedding model was trained for. Dot product is affected by vector length, Euclidean distance by both length and angle, cosine by angle only. On normalised vectors they all give the same ranking, since the squared distance is 2 − 2 cos.

A store that indexes vectors with metadata and answers “which stored vectors are nearest to this one?”. Exact search compares against every vector, which is too slow at scale, so it uses approximate nearest neighbour indexes. HNSW builds a layered graph of neighbours and walks greedily towards the query from the sparse top layer down; IVF clusters vectors and searches only the closest clusters. You trade a little recall for a large speed-up.

Offline: split documents into chunks, embed each chunk and store the vectors with metadata. Online: embed the question, retrieve the top-k chunks (often combined with keyword search), optionally rerank them, then put the best ones in the prompt with instructions to answer only from them and cite them. Evaluate retrieval (did we fetch the right chunk?) separately from generation (did we use it faithfully?).

Chunks must be small enough to be specific and to fit several in the prompt, and big enough to carry complete ideas. Split on structure (headings, paragraphs) rather than fixed character counts where you can, and add some overlap so a sentence cut at a boundary still appears whole somewhere. Then tune by measuring retrieval quality on real questions.

A second stage that rescores the retrieved candidates with a stronger, slower model, usually a cross-encoder that reads the query and each passage together. Embedding search is fast but coarse; reranking the top 50 or so and keeping the best few improves precision, and its scores make a better threshold for dropping irrelevant passages.

Retrieval can return the wrong passage, because nearest isn’t necessarily relevant, or miss the right one. The model can ignore the context in favour of what it learned, mix sources, or answer beyond them. Mitigate with a relevance threshold or reranker, explicit permission to abstain, required citations, and a check that each claim is supported by a cited passage.

Grounding means the answer is derived from sources you supplied. Asking for citations makes each claim traceable to a passage id, which lets users verify the answer and lets your code check that cited passages exist and actually contain the claim. It also discourages answers that have no support in the context.

Start with prompting. Use RAG when answers depend on private or frequently changing knowledge, or need citations. Use fine-tuning to change behaviour: a consistent format or tone, or a narrow task handled by a smaller, cheaper model. Fine-tuning is a poor way to add facts. They combine well, for example a fine-tuned model inside a RAG pipeline.

Pretraining on a huge text corpus with next-token prediction gives a base model. Supervised fine-tuning on example conversations teaches it to follow instructions. Preference tuning, such as RLHF or DPO, aligns it with which answers people prefer. Reasoning models add reinforcement learning on tasks whose answers can be checked, such as maths and code.

Both learn from pairs of answers where one was preferred. RLHF first trains a separate reward model on those preferences and then optimises the LLM against it with reinforcement learning, classically PPO, with a penalty for drifting too far from the original model. DPO skips the reward model and the RL loop, and optimises the LLM directly on the preferred and rejected pairs with a simple classification-style loss. DPO is simpler and more stable to run.

Low-rank adaptation: instead of updating all the weights, you freeze the model and train small low-rank matrices added to some layers. It needs far less memory and storage, and you can keep several adapters for one base model. QLoRA does the same on a quantised base model to save even more memory.

Instructions hidden in content the model processes, such as a web page, an email or a retrieved document, that try to override your instructions, for example to leak data or call a tool. The model can’t reliably tell data from instructions, so you can’t fully prevent it with prompting. Limit what the model can do: least-privilege tools, confirmation for side effects, no secrets in the context, and output validation.

You give the model a JSON schema, either for its answer or for each tool it may call. The model emits JSON that matches it, and many providers can enforce the schema with constrained decoding, which only allows tokens that keep the output valid. For a tool call your code runs the function and sends the result back as a new message. Validate the arguments anyway; matching a schema doesn’t make the values correct.

Build a test set of real inputs with expected answers or grading criteria, and run it on every change of prompt, model or retrieval. Use exact checks where possible (valid JSON, correct field, cited passage contains the claim), and an LLM judge with a clear rubric where you can’t, checked against human labels. Track retrieval metrics, faithfulness, latency and cost, and monitor production traces.

They see tokens, not characters: strawberry is three tokens in o200k_base, so the letters aren’t directly visible. Numbers are split into chunks of up to three digits, which don’t line up with place value. Give the model a tool such as a code interpreter or a calculator instead.

Use the smallest model that passes your evals, and route only hard requests to a larger one. Shorten prompts, cache repeated prefixes, retrieve fewer but better chunks, cap output length, stream responses, and cache whole answers for repeated questions. Batch offline work, and measure time to first token and tokens per second separately.

LangChain gives you standard, provider-independent parts for LLM apps: chat models, messages, prompts, tools, retrievers and output parsing, all composable as Runnables, plus create_agent for tool-calling agents. It pays off when you compose steps, swap providers, stream, trace or build agents. For a single model call with no tools or retrieval, the provider’s SDK is simpler, and you shouldn’t add a layer you can’t explain.

LangChain is the set of building blocks and the high-level create_agent. LangGraph is the low-level runtime for stateful workflows: graphs with state, cycles, checkpoints, interrupts and streaming; create_agent runs on it. LangSmith is the observability and evaluation platform, and it works with or without the other two.

A Runnable is anything with the standard interface: invoke, batch, stream and their async versions. Prompts, models, parsers, retrievers, tools and compiled graphs are all Runnables. LCEL, the LangChain Expression Language, composes them with |, which builds a RunnableSequence where each step’s output is the next step’s input. The composed chain is itself a Runnable, so it gets streaming, batching, async and tracing for free.

invoke takes one input and returns one output. batch takes a list and runs the inputs concurrently on a thread pool, limited by max_concurrency. stream yields the output in chunks, such as tokens, as soon as they are ready. astream and the other a methods are the async versions for asyncio servers, so a slow model call doesn’t block other requests.

RunnableParallel runs several Runnables on the same input and returns a dict of their results; a plain dict inside a chain is turned into one automatically. RunnablePassthrough passes its input through unchanged, and RunnablePassthrough.assign adds computed keys to an input dict. RunnableLambda wraps a plain Python function so it can sit in a chain. The classic RAG chain uses all three ideas: {"context": retriever | format, "question": RunnablePassthrough()} | prompt | model.

SystemMessage carries instructions, HumanMessage the user’s input, and AIMessage the model’s reply, which can include tool_calls and usage_metadata. ToolMessage carries a tool’s result and must have the tool_call_id of the call it answers. A chat model takes a list of messages and returns an AIMessage; in 1.x, .text gives the text and .content_blocks gives typed blocks in the same shape for every provider.

You bind tools with model.bind_tools(tools), which sends each tool’s name, description and JSON schema with the request. The model doesn’t run anything: it returns an AIMessage with tool_calls, each with a name, args and an id. Your code runs each tool and appends a ToolMessage with the matching tool_call_id, then calls the model again with the whole history. When the model replies without tool calls, you’re done.

Use bind_tools when the model should decide whether to act and which tools to call, as in an agent. Use with_structured_output(Schema) when you always want data in a fixed shape, such as extraction or classification: it forces the format with the provider’s JSON mode or a tool call, then parses the result into your Pydantic model, TypedDict or dict. include_raw=True also returns the raw message and any parsing error.

The model gets the conversation and the tool schemas. If it returns tool calls, the tools run and their results are appended as ToolMessages, then the model is called again. The loop ends when the model answers without tool calls, or when a limit stops it. In LangGraph that’s two nodes, model and tools, a conditional edge such as tools_condition, and an edge from tools back to the model.

create_agent(model, tools, ...) gives you the standard tool-calling loop on LangGraph, with checkpointing, streaming and middleware for things like human approval, summarisation, retries and call limits. Use it whenever the model choosing tools in a loop is the right shape. Build your own StateGraph when you need custom control flow: fixed steps mixed with agent steps, several agents, parallel fan-out, or approval at one specific step.

Hooks that run around the agent’s model and tool calls, so you change behaviour without rewriting the loop. Built-in ones include HumanInTheLoopMiddleware, SummarizationMiddleware, ModelRetryMiddleware, ModelFallbackMiddleware, ModelCallLimitMiddleware, ToolCallLimitMiddleware and PIIMiddleware. You write your own with decorators such as @before_model, @after_model, @wrap_model_call and @dynamic_prompt.

A StateGraph is built from a state schema, usually a TypedDict. Nodes return partial updates, and each key’s reducer decides how an update is merged: no reducer means overwrite, operator.add means concatenate. add_messages appends new messages and replaces any with the same id, so nodes can return just the new message and parallel nodes don’t overwrite each other. MessagesState is the ready-made state with a messages key using it.

add_conditional_edges(node, router) calls router(state) after the node and goes to the node it returns, or to END. An edge back to an earlier node creates a cycle, which is how agents loop. A node can also return Command(goto=..., update=...) to route and update in one step. Every loop needs an exit condition, and the recursion limit is the safety net.

It caps the number of super-steps in one run. When it’s reached, LangGraph raises GraphRecursionError instead of looping forever. Set it per call with config={"recursion_limit": n}. Older versions defaulted to 25; langgraph 1.2 defaults to 10,007 and create_agent uses 9,999, so set your own for agents, or add call-limit middleware.

A checkpointer saves the graph’s state after every super-step. A thread is one conversation or job, identified by thread_id in config["configurable"]; all its checkpoints are stored under that id. Invoke again with the same thread and the graph continues from the saved state, which gives you conversation memory, recovery after a crash, interrupts and time travel. Use InMemorySaver in tests and a Postgres or SQLite saver in production.

Because every step is checkpointed, you can list past states with get_state_history(config). Invoking with an older checkpoint’s config replays from that point. update_state on an old checkpoint creates a fork you can run forward with different values. It’s used for debugging, for retrying from a bad step, and for letting users edit an earlier turn.

Calling interrupt(value) in a node saves the state and stops the run; the caller gets value under __interrupt__. To continue, invoke the same thread with Command(resume=answer). The catch: the node restarts from its first line and interrupt() then returns answer, so any code before it runs twice. Keep side effects after the interrupt or make them idempotent. Interrupts need a checkpointer.

Short-term memory is the state of one thread, kept by the checkpointer: the messages of this conversation. Long-term memory has to outlive threads, such as user preferences or facts learned earlier, so it goes in a Store: JSON documents under namespaces like ("users", user_id), optionally searchable by meaning. Nodes reach the store through the Runtime argument.

Nodes scheduled in the same super-step run in parallel, so several edges out of one node already give you fixed parallel branches. When the number of branches is only known at run time, a conditional edge returns a list of Send(node, input), and each one runs the node with its own input, the map step of map-reduce. A reducer such as operator.add on the result key collects their outputs.

A compiled graph is a Runnable, so it can be a node inside another graph. If the parent and child share state keys, you add it directly; if not, you call it inside a node and map the state in and out. Subgraphs let you build and test agents separately and then combine them, which is the usual way to build multi-agent systems. A node in a subgraph can route in the parent with Command(graph=Command.PARENT, goto=...).

values streams the full state after each step, and updates streams only what each node returned. messages streams LLM tokens with metadata saying which node produced them, and custom streams whatever nodes send with get_stream_writer(). There are also checkpoints, tasks and debug; pass a list to get several as (mode, data) tuples.

Use astream with stream_mode="messages" to get tokens as they’re generated, often combined with "updates" to show progress such as which tool is running. Forward the chunks to the browser over server-sent events or a WebSocket. Token streaming works even if the node calls invoke, because LangGraph listens to the model’s callbacks.

Indexing: load sources into Documents, split them into chunks with a text splitter, embed the chunks and store them in a vector store. Answering: a retriever finds the most similar chunks for the question, and a chain puts them into the prompt and asks the model to answer from that context only. For harder questions you make retrieval a tool, so an agent can search several times and rewrite its query.

Chunks should be big enough to hold one complete idea and small enough that a retrieved chunk is mostly relevant; a few hundred to a thousand or so tokens is a common start. Overlap, often 10 to 20 percent, repeats text across neighbouring chunks so an idea cut at a boundary is still found. RecursiveCharacterTextSplitter splits on paragraphs, then lines, then words, to keep chunks natural. Then measure retrieval quality on real questions and tune.

A vector store stores embeddings and runs similarity search. A retriever is a Runnable interface: a query string in, a list of Documents out. vector_store.as_retriever(search_kwargs={"k": 4}) wraps a store, but a retriever can also be keyword search, a web search or a hybrid of several. Because it’s a Runnable, it drops into any chain.

Tracing: with LANGSMITH_TRACING=true and an API key, every run is recorded as a tree of steps with inputs, outputs, latency, tokens and errors, so you can see exactly what the model was sent. Evaluation: you keep datasets of examples, run your app over them with code or LLM-judge evaluators, and compare experiments before shipping. It also monitors production traffic and can run online evaluations on it.

runnable.with_retry(stop_after_attempt=3) retries with exponential backoff, and with_fallbacks([backup]) tries another model or chain when it still fails. Pass rate_limiter=InMemoryRateLimiter(requests_per_second=...) to a chat model to stay under provider limits. In agents, ModelRetryMiddleware, ToolRetryMiddleware and ModelFallbackMiddleware do the same jobs.

Use the fake chat models in langchain_core: FakeListChatModel replays strings, and GenericFakeChatModel or FakeMessagesListChatModel replay whole AIMessages, including tool calls. Run the real chain or graph with an InMemorySaver and assert on the messages, the route taken and the final state. That keeps unit tests fast and deterministic; you judge answer quality separately with LangSmith evaluations.

Trim it, keeping the system message and the most recent messages, with trim_messages; summarise older turns into a short message, which SummarizationMiddleware does for agents; or move lasting facts to a Store and retrieve them when needed. When trimming, never split an AIMessage with tool calls from its ToolMessages, or the provider rejects the history.

In a graph, ToolNode by default turns invalid-argument errors into a ToolMessage so the model can correct its call, and re-raises other exceptions. Set handle_tool_errors=True, or pass a function, to send every error back to the model as a message. In create_agent, retry middleware can retry flaky tools first.

It creates a chat model from a string like "provider:model-name", loading the right integration package, such as langchain-openai or langchain-anthropic. Every provider then has the same interface, so you can switch models through configuration without changing code. create_agent also accepts the same string directly.

Each agent is a node or a subgraph with its own prompt and tools. In a supervisor design, one agent routes work to the others, often by calling them as tools; in a handoff design, agents pass control directly with Command(goto=...). Shared state or messages carry the context between them. Start with one agent and add more only when tools or instructions clearly separate.