Search
30 results for “token”
Search results
LLM Fundamentals: Tokens, Context Windows & Sampling
Why a token isn't a word, what actually happens when a request would exceed the context window, and what temperature and top_p each control during generation.
Why does a word count poorly estimate how many tokens a prompt will use?
A token is a commonly-occurring sequence of characters the model was trained on, not a word, common short words are often a single token, while a longer or less common word can split into several tokens ("tokenization" might become " token" plus "ization"). As a rough rule of thumb, one token is about 4 characters or 0.75 words in English, but that ratio shifts with vocabulary, punctuation, and especially non-English text, so a word count is only ever an approximation, not the actual unit the model's context window and pricing are measured in.
In the token bucket algorithm, what do the "rate" and "burst" settings each actually control?
The rate is how many tokens get added to the bucket per second, the steady-state throughput the API is designed to sustain indefinitely. The burst is the bucket's total capacity, the maximum number of tokens that can accumulate and therefore the largest spike of concurrent requests the API will accept in a single instant before it starts rejecting anything further. A request consumes one token; as long as the bucket has a token available, the request proceeds immediately, rate and burst are two independent numbers, not one combined setting, because sustained throughput and tolerance for short spikes are genuinely different things to configure.
Why would a system use a sliding window rate limiter instead of a token bucket, and what problem does it fix?
A naive fixed window (say, "100 requests per minute, resetting on the minute") allows a client to send 100 requests in the last second of one window and another 100 in the first second of the next, 200 requests in a two-second span despite the stated 100/minute limit, an artifact of the window boundary rather than actual demand. A sliding window rate limiter instead evaluates the limit over a continuously moving time range rather than fixed, discrete buckets, which avoids that boundary-doubling effect at the cost of typically more state to track (timestamps of recent requests, not just a single counter) compared to a token bucket's simpler accumulator model.
JWT Decoder
Decode a JWT header and payload; runs entirely in your browser, the signature is not verified since no secret is available client-side.
Rate Limiting Algorithms
How the token bucket algorithm AWS API Gateway actually uses separates a steady-state rate from a burst allowance, and why a request only fails once the bucket is genuinely empty, not the instant the average rate is exceeded.
A request's input already exceeds the model's context window before generation even starts. What happens, versus a request that only exceeds the limit once output is generated?
If the input alone already exceeds the context window, the API rejects the request upfront with a 400 error, generation never starts at all. If the input fits but input tokens plus the requested max output tokens could exceed the window, current models accept the request and generate as far as they can; if generation actually reaches the window limit before finishing, it stops early with a specific stop reason indicating the context window was exhausted, rather than silently truncating or erroring out mid-response. The distinction matters operationally: the first case is a fixable request-construction bug, the second is a signal to reduce the requested output length or the accumulated conversation history.
What is the practical difference between lowering temperature and lowering top_p, and why would you use them together?
Temperature scales the randomness applied across the entire probability distribution of next-token candidates, low temperature makes the model consistently pick its highest-probability tokens, high temperature flattens the distribution so lower-probability tokens get chosen more often. top_p (nucleus sampling) instead restricts the candidate pool itself to the smallest set of tokens whose cumulative probability reaches the given threshold, then samples only from that trimmed set. They're complementary, not redundant: temperature changes how sharply the model prefers likely tokens, top_p changes which tokens are even eligible to be picked, which is why both are typically left at sensible defaults and only tuned together deliberately for advanced use cases, not adjusted independently without understanding the interaction.
A client sends a sudden spike of requests that exceeds the configured rate, but the API doesn't immediately reject any of them. Why not, and when does rejection actually start?
The token bucket has accumulated capacity up to the burst limit, if the client had been under the rate limit recently, unused tokens built up in the bucket, and that reserve absorbs a short spike without any request failing, exactly the point of separating burst capacity from steady-state rate. Rejection (a 429 response) only starts once the bucket is actually empty, every accumulated token has been consumed and the spike is sustained long enough that the rate of token consumption keeps exceeding the rate of token replenishment. This is why a token bucket, unlike a hard per-second cap, tolerates brief bursts gracefully while still enforcing a real steady-state ceiling.
Tool Use & Agents
How a tool call actually completes as a round trip, a tool_use block your code executes and a tool_result you send back, and why every tool definition you provide adds real tokens to every request whether or not it's ever called.
What is the difference between authentication and authorization in a cloud IAM context?
Authentication answers "who is making this request", verifying an identity via credentials, a token, or a federated login. Authorization answers "is this identity allowed to do this specific action on this specific resource", evaluated after authentication succeeds, by checking the identity's attached policies against the requested action. A request can be perfectly authenticated (the caller genuinely is who they claim) and still be denied, because authorization is a separate check against what that identity is actually permitted to do.
When does semantic search (via embeddings) actually outperform traditional keyword search, and when might keyword search still win?
Semantic search wins when a query and the relevant document use different words for the same idea, "car won't start" matching a document about "vehicle fails to ignite", since embeddings compare meaning rather than literal tokens, which keyword search cannot do at all. Keyword search still wins, or at least remains necessary, for exact-match needs, an error code, a product SKU, a specific proper noun, where the literal string matters and a semantically "close" but textually different result is actually the wrong answer. This is why production search systems commonly combine both (hybrid search) rather than treating embeddings as a strict replacement for keyword matching.
What problem does RAG solve that simply pasting an entire knowledge base into the prompt doesn't?
A real knowledge base, product docs, legal filings, support history, routinely exceeds any model's context window, and even when it technically fits, stuffing everything into every request wastes tokens on mostly-irrelevant content and degrades accuracy as context grows. RAG instead retrieves only the specific chunks relevant to the current query at request time, from a knowledge base that's been pre-processed into searchable chunks and embeddings, so the model gets focused, relevant background knowledge sized to the question actually being asked, not the entire corpus every time.
Why does simply including a tool definition in a request add cost even on a turn where the model never actually calls it?
Every tool's name, description, and input schema in the `tools` parameter counts as real input tokens on every single request, and using tools at all triggers an additional fixed system-prompt token overhead that varies by model, regardless of whether the model ends up calling any tool that turn. This means a large library of rarely-used tool definitions has a real, continuous token cost across every request that includes them, which is exactly why techniques like on-demand tool discovery (only loading the tool definitions actually relevant to the current task) exist as a mitigation for applications with many available tools.
Securing Azure Blob Storage with PowerShell: Network Isolation, SAS Access & Immutable Policies (Beginner to Pro)
A hands-on lab automating secure Azure Blob Storage using VNets, subnets, SAS tokens, and immutability.
Why does reusing the same idempotency key with different request parameters return an error instead of just processing the new parameters?
An idempotency key is a promise that a specific, exact operation happened once; if the same key showed up with different parameters, honoring the new parameters would silently violate that promise; either the original operation's recorded result no longer describes what the key represents, or the client made a mistake by rIeusing a key it should have generated fresh for a genuinely different request. Rejecting the mismatched reuse as an error, rather than guessing which parameters were "correct" or silently processing the new ones, surfaces that client-side mistake immediately instead of masking it.
Why does using a monotonically increasing field (a sequential ID, a timestamp) as a shard key concentrate all new writes onto one shard, even with range-based sharding across many shards?
With range-based sharding, chunks are assigned contiguous ranges of shard-key values, and a monotonically increasing key means every new document's value is higher than every previously inserted one, so all new inserts land in whatever chunk currently owns the highest range, which lives on one specific shard. Every other shard, holding older, lower-valued ranges, receives none of the new write traffic at all, the exact "hot shard" problem, all insert load concentrated on a single shard regardless of how many total shards the cluster has.
MongoDB can be configured to behave more like a CP system or more like an AP system. What settings make that choice, and what are they actually trading?
Write concern and read preference are the actual levers: a write concern requiring acknowledgment from a majority of replicas favors consistency, a write isn't considered successful until enough nodes agree, at the cost of availability if too many nodes are unreachable during a partition. A looser write concern, or reading from secondaries that might lag behind the primary, favors availability, operations keep succeeding even when full replica agreement isn't achievable, at the cost of potentially reading or acknowledging data that isn't fully consistent across the cluster yet. This is exactly the CP-versus-AP trade-off CAP describes, expressed as a concrete, tunable configuration rather than an abstract theorem.
Two transactions each update two of the same two accounts, but in opposite order, and deadlock. What's the actual fix, not just for this pair of transactions, but for the application generally?
The deadlock happens because Transaction 1 locks account A then waits for account B, while Transaction 2 locks account B then waits for account A, a circular wait. The general fix isn't retry logic alone, retries only paper over deadlocks that keep recurring, it's acquiring locks on multiple objects in the same, consistent order everywhere in the application (for example, always locking accounts in ascending id order), which makes the circular-wait pattern structurally impossible rather than merely less frequent. Retry logic is still worth having as a safety net, but consistent lock ordering is what actually eliminates this class of deadlock.
Azure Policy, Tags, and Resource Locks Explained: A Complete Governance Guide for Cloud Engineers
Effective governance is essential as cloud environments grow. Without proper controls, organizations often face challenges in visibility, accountability, and cost tracking. In this lab, we explore how to implement governance practices in Azure using Azure Policy , Resource Tags , and Resource Locks…
Why can animating transform or opacity skip layout and paint entirely, while animating top or width cannot?
transform and opacity don't change an element's geometry, its size or position in the document flow, or repaint its actual pixel content, they only change how an already-painted layer is displayed (moved, scaled, faded) during compositing. Because nothing about the element's layout or pixels actually changed, the browser can skip straight to the composite stage, the cheapest, fastest path in the pipeline, and handle it on the compositor thread. `top` and `width` do change geometry, so there's no way to skip layout, the browser has no shortcut for "the size changed but skip figuring out the new size."
What is a shard key, and why does a low-cardinality shard key (few distinct values) cause a hot shard?
A shard key is the field (or fields) a sharded database uses to decide which shard each document or row actually lives on. If that field has few distinct values, say `country`, and the real data is skewed (80% of users in one country), the vast majority of documents route to the same shard regardless of how many shards exist in the cluster, overwhelming it with disproportionate read/write load and storage while other shards sit comparatively idle. Cardinality alone doesn't guarantee even distribution either, the values also need to actually occur with reasonably even frequency in the real data, not just theoretically have many possible values.
What is the default Docker network driver and how does container-to-container communication work on it?
The default is the bridge driver, Docker creates a private virtual network on the host, and each container gets its own network namespace with a virtual ethernet interface connected to that bridge. Containers on the same user-defined bridge network can reach each other by container name, because Docker runs an embedded DNS server that resolves container names to their internal IPs on that network. Containers on the default (unnamed) bridge network do not get this automatic DNS resolution, only user-defined bridge networks provide it.
What is the required contract between equality and hashing for an object used as a dictionary key, and why does the hash table need it specifically?
The contract is one-directional but strict: if two objects compare equal, they must produce the same hash value. A hash table uses an object's hash to pick which bucket to look in, then uses equality only to confirm the exact match within that bucket, so if two equal objects hashed differently, a lookup for one would search the wrong bucket entirely and never even reach the equality check that would have confirmed the match. The hash doesn't have to be unique across unequal objects (collisions are expected and handled), it just has to agree for anything that compares equal, that's the one property the whole lookup mechanism depends on.
Why does sending SIGKILL to a stuck process work when SIGTERM doesn't, and what does that cost you?
SIGTERM asks a process to terminate but can be caught by a signal handler, letting the process run its own cleanup logic (closing files, flushing buffers, releasing locks) before actually exiting, or in a broken process, being caught and never acted on at all. SIGKILL cannot be caught, blocked, or ignored under any circumstances, the kernel terminates the process directly, which is why it works on a process SIGTERM couldn't reach. The cost is that none of that cleanup logic runs, a database connection isn't closed cleanly, a temp file isn't removed, a lock isn't released, so SIGKILL is a last resort after SIGTERM has been given a real chance to work, not a default first move.
Why does Linux split user information across /etc/passwd and /etc/shadow instead of keeping everything, including the password hash, in one file?
/etc/passwd has to be world-readable, ordinary tools and commands need to map UIDs to usernames and look up home directories or login shells for every user on the system. If password hashes lived there too, every local user could copy them out and run an offline cracking attempt. /etc/shadow holds the actual encrypted password (and related aging data) and is readable only by root, while /etc/passwd keeps a placeholder character (commonly `x`) in the password field, preserving the UID-lookup functionality everything else depends on without exposing anything crackable.
What does the kernel actually check when deciding whether a process can access a file, the username or the UID?
The UID. A username is a human-readable label that /etc/passwd maps to a numeric UID, but every permission check, ownership comparison, and process credential the kernel deals with operates on that number, root is UID 0 specifically, not "whichever account is named root." This is why two different usernames can be given identical privileges by sharing UID 0, and why renaming an account changes nothing about what it's allowed to do, only the numeric UID does.
What is the difference between GROUP BY aggregation and a window function like `ROW_NUMBER() OVER (...)`?
`GROUP BY` collapses every row in a group into a single output row per group, you lose access to the individual rows once you've aggregated. A window function computes its result (a rank, a running total, a row number) per row while keeping every individual row in the output, `PARTITION BY` defines the grouping for that calculation, but nothing is collapsed. This is why "give me each order plus that customer's running total" needs a window function, while "give me one row per customer with their total" is a plain `GROUP BY`.
Why can a list not be used as a dictionary key, but a tuple can?
Dictionary keys must be hashable, and hashability requires that an object's hash value never changes for the lifetime of the object, which in turn requires immutability, because a mutable object's contents (and therefore its logical value) could change after being used as a key, silently breaking the hash table's internal bucket placement. Lists are mutable, so Python makes them explicitly unhashable to prevent that class of bug. Tuples are immutable, so as long as every element they contain is also hashable, the tuple itself is hashable and safe to use as a dictionary key or set member.
Why is using a mutable object (like a list) as a default argument value a common bug in Python?
Default argument values are evaluated exactly once, when the function is defined, not on every call, so a mutable default like `def f(items=[])` creates one list object that is shared across every call that doesn't explicitly pass its own `items`. Appending to it in one call leaves those items present the next time the function is called with the default, which looks like inexplicable state leaking between unrelated calls. The fix is defaulting to `None` and creating a new list inside the function body when `items is None`, so every call that needs the default gets its own fresh object.