Search
30 results for “tokens”
Search results
LLM Fundamentals: Tokens, Context Windows & Sampling
Why a token isn't a word, what actually happens when a request would exceed the context window, and what temperature and top_p each control during generation.
Why does a word count poorly estimate how many tokens a prompt will use?
A token is a commonly-occurring sequence of characters the model was trained on, not a word, common short words are often a single token, while a longer or less common word can split into several tokens ("tokenization" might become " token" plus "ization"). As a rough rule of thumb, one token is about 4 characters or 0.75 words in English, but that ratio shifts with vocabulary, punctuation, and especially non-English text, so a word count is only ever an approximation, not the actual unit the model's context window and pricing are measured in.
A request's input already exceeds the model's context window before generation even starts. What happens, versus a request that only exceeds the limit once output is generated?
If the input alone already exceeds the context window, the API rejects the request upfront with a 400 error, generation never starts at all. If the input fits but input tokens plus the requested max output tokens could exceed the window, current models accept the request and generate as far as they can; if generation actually reaches the window limit before finishing, it stops early with a specific stop reason indicating the context window was exhausted, rather than silently truncating or erroring out mid-response. The distinction matters operationally: the first case is a fixable request-construction bug, the second is a signal to reduce the requested output length or the accumulated conversation history.
What is the practical difference between lowering temperature and lowering top_p, and why would you use them together?
Temperature scales the randomness applied across the entire probability distribution of next-token candidates, low temperature makes the model consistently pick its highest-probability tokens, high temperature flattens the distribution so lower-probability tokens get chosen more often. top_p (nucleus sampling) instead restricts the candidate pool itself to the smallest set of tokens whose cumulative probability reaches the given threshold, then samples only from that trimmed set. They're complementary, not redundant: temperature changes how sharply the model prefers likely tokens, top_p changes which tokens are even eligible to be picked, which is why both are typically left at sensible defaults and only tuned together deliberately for advanced use cases, not adjusted independently without understanding the interaction.
Tool Use & Agents
How a tool call actually completes as a round trip, a tool_use block your code executes and a tool_result you send back, and why every tool definition you provide adds real tokens to every request whether or not it's ever called.
When does semantic search (via embeddings) actually outperform traditional keyword search, and when might keyword search still win?
Semantic search wins when a query and the relevant document use different words for the same idea, "car won't start" matching a document about "vehicle fails to ignite", since embeddings compare meaning rather than literal tokens, which keyword search cannot do at all. Keyword search still wins, or at least remains necessary, for exact-match needs, an error code, a product SKU, a specific proper noun, where the literal string matters and a semantically "close" but textually different result is actually the wrong answer. This is why production search systems commonly combine both (hybrid search) rather than treating embeddings as a strict replacement for keyword matching.
In the token bucket algorithm, what do the "rate" and "burst" settings each actually control?
The rate is how many tokens get added to the bucket per second, the steady-state throughput the API is designed to sustain indefinitely. The burst is the bucket's total capacity, the maximum number of tokens that can accumulate and therefore the largest spike of concurrent requests the API will accept in a single instant before it starts rejecting anything further. A request consumes one token; as long as the bucket has a token available, the request proceeds immediately, rate and burst are two independent numbers, not one combined setting, because sustained throughput and tolerance for short spikes are genuinely different things to configure.
A client sends a sudden spike of requests that exceeds the configured rate, but the API doesn't immediately reject any of them. Why not, and when does rejection actually start?
The token bucket has accumulated capacity up to the burst limit, if the client had been under the rate limit recently, unused tokens built up in the bucket, and that reserve absorbs a short spike without any request failing, exactly the point of separating burst capacity from steady-state rate. Rejection (a 429 response) only starts once the bucket is actually empty, every accumulated token has been consumed and the spike is sustained long enough that the rate of token consumption keeps exceeding the rate of token replenishment. This is why a token bucket, unlike a hard per-second cap, tolerates brief bursts gracefully while still enforcing a real steady-state ceiling.
What problem does RAG solve that simply pasting an entire knowledge base into the prompt doesn't?
A real knowledge base, product docs, legal filings, support history, routinely exceeds any model's context window, and even when it technically fits, stuffing everything into every request wastes tokens on mostly-irrelevant content and degrades accuracy as context grows. RAG instead retrieves only the specific chunks relevant to the current query at request time, from a knowledge base that's been pre-processed into searchable chunks and embeddings, so the model gets focused, relevant background knowledge sized to the question actually being asked, not the entire corpus every time.
Why does simply including a tool definition in a request add cost even on a turn where the model never actually calls it?
Every tool's name, description, and input schema in the `tools` parameter counts as real input tokens on every single request, and using tools at all triggers an additional fixed system-prompt token overhead that varies by model, regardless of whether the model ends up calling any tool that turn. This means a large library of rarely-used tool definitions has a real, continuous token cost across every request that includes them, which is exactly why techniques like on-demand tool discovery (only loading the tool definitions actually relevant to the current task) exist as a mitigation for applications with many available tools.
Securing Azure Blob Storage with PowerShell: Network Isolation, SAS Access & Immutable Policies (Beginner to Pro)
A hands-on lab automating secure Azure Blob Storage using VNets, subnets, SAS tokens, and immutability.
MongoDB can be configured to behave more like a CP system or more like an AP system. What settings make that choice, and what are they actually trading?
Write concern and read preference are the actual levers: a write concern requiring acknowledgment from a majority of replicas favors consistency, a write isn't considered successful until enough nodes agree, at the cost of availability if too many nodes are unreachable during a partition. A looser write concern, or reading from secondaries that might lag behind the primary, favors availability, operations keep succeeding even when full replica agreement isn't achievable, at the cost of potentially reading or acknowledging data that isn't fully consistent across the cluster yet. This is exactly the CP-versus-AP trade-off CAP describes, expressed as a concrete, tunable configuration rather than an abstract theorem.
Why does reusing the same idempotency key with different request parameters return an error instead of just processing the new parameters?
An idempotency key is a promise that a specific, exact operation happened once; if the same key showed up with different parameters, honoring the new parameters would silently violate that promise; either the original operation's recorded result no longer describes what the key represents, or the client made a mistake by rIeusing a key it should have generated fresh for a genuinely different request. Rejecting the mismatched reuse as an error, rather than guessing which parameters were "correct" or silently processing the new ones, surfaces that client-side mistake immediately instead of masking it.
Why does sending SIGKILL to a stuck process work when SIGTERM doesn't, and what does that cost you?
SIGTERM asks a process to terminate but can be caught by a signal handler, letting the process run its own cleanup logic (closing files, flushing buffers, releasing locks) before actually exiting, or in a broken process, being caught and never acted on at all. SIGKILL cannot be caught, blocked, or ignored under any circumstances, the kernel terminates the process directly, which is why it works on a process SIGTERM couldn't reach. The cost is that none of that cleanup logic runs, a database connection isn't closed cleanly, a temp file isn't removed, a lock isn't released, so SIGKILL is a last resort after SIGTERM has been given a real chance to work, not a default first move.
What does it mean for a sorting algorithm to be "stable," and why does that matter for sorting by multiple keys in separate passes?
A stable sort guarantees that when two elements compare as equal under the current sort key, their original relative order is preserved rather than left unspecified. This matters directly for multi-key sorting done as a series of single-key sorts: sort by a secondary key first, then stably sort by the primary key, and elements sharing the same primary key retain their secondary-key order from the first pass, correctly producing a combined sort by (primary, secondary) without needing a single comparator that handles both keys at once. An unstable sort would silently scramble that secondary ordering among equal-primary-key elements.
Why does using a monotonically increasing field (a sequential ID, a timestamp) as a shard key concentrate all new writes onto one shard, even with range-based sharding across many shards?
With range-based sharding, chunks are assigned contiguous ranges of shard-key values, and a monotonically increasing key means every new document's value is higher than every previously inserted one, so all new inserts land in whatever chunk currently owns the highest range, which lives on one specific shard. Every other shard, holding older, lower-valued ranges, receives none of the new write traffic at all, the exact "hot shard" problem, all insert load concentrated on a single shard regardless of how many total shards the cluster has.
A custom dropdown built entirely from styled divs passes a visual design review. What is likely still broken for a keyboard-only or screen-reader user, and why doesn't looking right catch it?
Without the correct ARIA roles/states (or, better, a native `<select>`), a div-based dropdown typically has no way to be reached or operated via keyboard alone (no built-in Tab/Enter/Arrow-key handling), and a screen reader has no semantic information telling it "this is a dropdown, it is currently closed, here are its options," so it may announce nothing meaningful at all. A purely visual review can't catch this because the div looks and behaves correctly with a mouse, the missing behavior only surfaces via keyboard navigation or assistive technology, which is exactly why accessibility has to be tested directly with a keyboard and a screen reader, not inferred from how a component looks.
Two transactions each update two of the same two accounts, but in opposite order, and deadlock. What's the actual fix, not just for this pair of transactions, but for the application generally?
The deadlock happens because Transaction 1 locks account A then waits for account B, while Transaction 2 locks account B then waits for account A, a circular wait. The general fix isn't retry logic alone, retries only paper over deadlocks that keep recurring, it's acquiring locks on multiple objects in the same, consistent order everywhere in the application (for example, always locking accounts in ascending id order), which makes the circular-wait pattern structurally impossible rather than merely less frequent. Retry logic is still worth having as a safety net, but consistent lock ordering is what actually eliminates this class of deadlock.
Why does explicitly telling a model it's allowed to say "I don't know" measurably reduce hallucination, and what's a second, complementary technique for the same problem?
Without that explicit permission, a model under an implicit expectation to always produce a confident, complete answer will sometimes fill a real information gap with a plausible-sounding but fabricated one; stating outright that uncertainty is an acceptable answer removes that pressure and lets the model surface "I don't have enough information" instead of guessing. A complementary technique for long documents is asking the model to first extract direct, word-for-word quotes relevant to the task before generating any analysis, grounding its response in verifiable text it actually has in front of it, and then citing which quote supports each claim, rather than generating an answer freely and hoping it stayed faithful to the source.
What does FastAPI provide that Flask does not, and what's the trade-off?
FastAPI is async-first (built on ASGI, not WSGI) and uses Python type hints to automatically generate request validation, serialization, and interactive OpenAPI documentation, a type-annotated function signature becomes both the API contract and its enforcement, with no separate schema to maintain by hand. The trade-off is that FastAPI assumes an async-first mental model and a type-hint-driven style throughout, which is a bigger shift for a team used to Flask's simpler, synchronous, un-opinionated style, and FastAPI has a smaller, younger ecosystem of plugins/extensions compared to Flask or Django's much longer history.
Azure Policy, Tags, and Resource Locks Explained: A Complete Governance Guide for Cloud Engineers
Effective governance is essential as cloud environments grow. Without proper controls, organizations often face challenges in visibility, accountability, and cost tracking. In this lab, we explore how to implement governance practices in Azure using Azure Policy , Resource Tags , and Resource Locks…
How does a Network Security Group (NSG) evaluate traffic rules?
An NSG contains a prioritized list of allow/deny rules, evaluated in priority order (lowest number first) until the first rule matching the traffic's source, destination, port, and protocol is found, that rule's action wins, and no further rules are evaluated. NSGs can attach to a subnet, a network interface, or both, and are stateful, meaning an allowed inbound connection's return traffic is automatically permitted without needing a matching outbound rule. Because evaluation stops at first match, rule priority ordering is the actual logic, a broad allow rule placed before a specific deny rule silently makes that deny unreachable.
What is a shard key, and why does a low-cardinality shard key (few distinct values) cause a hot shard?
A shard key is the field (or fields) a sharded database uses to decide which shard each document or row actually lives on. If that field has few distinct values, say `country`, and the real data is skewed (80% of users in one country), the vast majority of documents route to the same shard regardless of how many shards exist in the cluster, overwhelming it with disproportionate read/write load and storage while other shards sit comparatively idle. Cardinality alone doesn't guarantee even distribution either, the values also need to actually occur with reasonably even frequency in the real data, not just theoretically have many possible values.
What is the default Docker network driver and how does container-to-container communication work on it?
The default is the bridge driver, Docker creates a private virtual network on the host, and each container gets its own network namespace with a virtual ethernet interface connected to that bridge. Containers on the same user-defined bridge network can reach each other by container name, because Docker runs an embedded DNS server that resolves container names to their internal IPs on that network. Containers on the default (unnamed) bridge network do not get this automatic DNS resolution, only user-defined bridge networks provide it.
What is the required contract between equality and hashing for an object used as a dictionary key, and why does the hash table need it specifically?
The contract is one-directional but strict: if two objects compare equal, they must produce the same hash value. A hash table uses an object's hash to pick which bucket to look in, then uses equality only to confirm the exact match within that bucket, so if two equal objects hashed differently, a lookup for one would search the wrong bucket entirely and never even reach the equality check that would have confirmed the match. The hash doesn't have to be unique across unequal objects (collisions are expected and handled), it just has to agree for anything that compares equal, that's the one property the whole lookup mechanism depends on.
Why does Linux split user information across /etc/passwd and /etc/shadow instead of keeping everything, including the password hash, in one file?
/etc/passwd has to be world-readable, ordinary tools and commands need to map UIDs to usernames and look up home directories or login shells for every user on the system. If password hashes lived there too, every local user could copy them out and run an offline cracking attempt. /etc/shadow holds the actual encrypted password (and related aging data) and is readable only by root, while /etc/passwd keeps a placeholder character (commonly `x`) in the password field, preserving the UID-lookup functionality everything else depends on without exposing anything crackable.
What does the kernel actually check when deciding whether a process can access a file, the username or the UID?
The UID. A username is a human-readable label that /etc/passwd maps to a numeric UID, but every permission check, ownership comparison, and process credential the kernel deals with operates on that number, root is UID 0 specifically, not "whichever account is named root." This is why two different usernames can be given identical privileges by sharing UID 0, and why renaming an account changes nothing about what it's allowed to do, only the numeric UID does.
Why are environment variables considered a weaker place to store a secret than a dedicated secrets manager, even though they avoid hardcoding it in source?
Environment variables are readable by the entire process (and often child processes) they're set for, commonly get dumped into crash reports, debugging output, or `/proc` on Linux, and are easy to accidentally log in full, none of which requires a targeted attack, just an ordinary operational mistake. A dedicated secrets manager instead requires an authenticated, audited API call to retrieve a secret, can issue it as short-lived, and centralizes rotation and access logging in one place. Environment variables are a real improvement over hardcoding in source, but they are a stopgap, not the same security posture as centralized, audited secret retrieval.
How to Configure Desktop Backgrounds, Power Settings, and Legal Notices Using Group Policy
In this lab, I implemented key Group Policy configurations to standardize system behavior, improve user experience, and strengthen security awareness across all domain joined devices. The configuration focuses on: Enforcing a consistent and professional desktop environment Preventing unauthorized o…
What is the difference between a reserved/committed-use discount and a spot/preemptible instance, and when does each make sense?
A committed-use discount (reserved instances, savings plans) trades a usage commitment, a fixed amount of spend or capacity over a term, typically one or three years, for a significant price reduction on workloads you know will run continuously. Spot/preemptible instances offer a much steeper discount in exchange for the provider being able to reclaim the capacity with little notice, making them suitable only for interruption-tolerant workloads (batch jobs, stateless workers, CI runners) rather than anything requiring guaranteed uptime. Committed-use addresses predictable steady-state load; spot addresses flexible, interruption-tolerant load, using either for the wrong workload type either wastes the discount or causes outages.