Jekyll2026-10-22T16:58:00-06:00https://bishnubhusal.com.np/feed.xmlBishnu BhusalBishnu Bhusal is a computer science researcher working on differential privacy, formal verification, and privacy-preserving machine learning and LLMs. Publications at ACM CCS, AAAI, and OOPSLA.Why gradient clipping changes Muon’s geometry: a two-dimensional example2026-10-17T09:00:00-06:002026-10-17T09:00:00-06:00https://bishnubhusal.com.np/blog/2026/clipping-geometry-private-muonPer-example gradient clipping changes more than an update’s size. Although clipping multiplies each example’s gradient by a nonnegative scalar, different examples generally receive different scalars. Their average can therefore point in a different direction or, for matrix-valued gradients, have different singular-vector geometry.

Our preprint Inner Momentum for Differentially Private Muon investigates why this matters for Muon and proposes a history-based correction before clipping. This tutorial uses the preprint’s abstract to describe its motivation and reported scope. The calculation below is an independent illustration, not a reproduction of its algorithm, theoretical bounds, or experiments. The source is available on arXiv.

Why a matrix optimizer notices clipping

A matrix A can be written using a singular value decomposition as U Sigma V-transpose. For a nonsingular square matrix, its orthogonal polar factor is Q = U V-transpose. This retains singular-vector information while replacing singular values by one. Muon’s orthogonalization is motivated by an approximation to this polar factor.

Multiplying A by the same positive scalar changes its singular values but preserves Q. That invariance can make clipping seem harmless at first: if clipping just scales the update, why would the orthogonalized update change?

The missing distinction is the order of operations. Private training clips individual examples before averaging them. If their clipping multipliers differ, the clipped mean need not be a scalar multiple of the original mean. Orthogonalization can therefore see a different input geometry even before privacy noise is added.

Two matrices are enough

Consider two illustrative per-example matrix gradients:

\[G_1 = \begin{pmatrix} 4 & 0 \\ 0 & 0 \end{pmatrix}, \qquad G_2 = \begin{pmatrix} 0 & 1 \\ 0 & 1 \end{pmatrix}.\]

Their arithmetic mean is

\[A = \frac{G_1 + G_2}{2} = \begin{pmatrix} 2 & 0.5 \\ 0 & 0.5 \end{pmatrix}.\]

Clip each matrix to Frobenius norm at most 1. The Frobenius norm is the square root of the sum of squared entries. The clipping operation multiplies G by min(1, 1 / norm(G)), leaving a zero matrix unchanged.

The two norms are 4 and square-root-of-2, so the clipping multipliers are 0.25 and approximately 0.707107. After clipping each example and then averaging, the result is

\[B = \begin{pmatrix} 0.5 & 0.353553\ldots \\ 0 & 0.353553\ldots \end{pmatrix}.\]

B is not a common rescaling of A. Its first column shrank more than its second because the examples contributing to those columns had different norms. This reweighting occurs before we add any noise or run any approximate orthogonalization routine.

Two example gradients receive different clipping multipliers: 0.25 and about 0.7071. Their original mean is [[2, 0.5], [0, 0.5]], while the mean after clipping is about [[0.5, 0.3536], [0, 0.3536]].
Figure 1. Clipping each example changes its contribution to the mean. The first example is reduced more strongly, so the two columns do not shrink by a common factor. Values come from the illustrative matrices above.

Compute the polar-factor rotation

For a real 2-by-2 matrix with entries a, b, c, d and positive determinant, its polar factor has a convenient closed form:

\[Q = \frac{1}{\sqrt{(a + d)^2 + (c - b)^2}} \begin{pmatrix} a + d & b - c \\ c - b & a + d \end{pmatrix}.\]

Its rotation angle is atan2(c − b, a + d). Both means in this example have positive determinants, so the formula applies even though the individual example matrices are rank deficient.

Matrix sent to orthogonalization Polar-factor rotation
Original mean A −11.309932 degrees
Mean of clipped examples B −22.500000 degrees
Common rescaling 0.25A −11.309932 degrees

The unequal clipping changes the polar rotation by about 11.19 degrees. Common positive rescaling leaves it unchanged. The Frobenius distance between the two polar factors is approximately 0.275762. These are deterministic properties of the chosen matrices, not measured training improvements.

The original polar factor rotates the first unit vector by minus 11.31 degrees; after per-example clipping, the rotation is minus 22.50 degrees, a difference of 11.19 degrees. Common positive rescaling preserves the original rotation.
Figure 2. Each arrow shows where a polar factor sends the first unit vector. These are unit directions after orthogonalization; their lengths do not represent gradient magnitudes.

Download clipping_polar_geometry.py and run it with Python’s standard library:

python3 clipping_polar_geometry.py

The script prints the clipping multipliers, both means, their rotations, and the common-rescaling control. It intentionally handles only the positive-determinant 2-by-2 case needed here. It is not a general SVD implementation or a private optimizer.

Explore the clipping threshold

Let \(C>0\) be the clipping threshold. For these two matrices, thresholds at or below \(\sqrt{2}\) give both clipped examples norm \(C\) and produce the same polar rotation of −22.5°. Between \(\sqrt{2}\) and 4, only the first example is clipped, so the angle moves toward the original −11.31°. At \(C\geq4\), neither example is changed.

Polar rotation versus clipping threshold C: minus 22.5 degrees up to square root of two, then approaching the original minus 11.31 degrees at C equal to four, where clipping stops. The worked example uses C equal to one.
Figure 3. Changing the threshold changes the geometry of the same two example matrices. The curve is calculated directly from their polar factors.

The connection to inner momentum

The preprint proposes averaging each sampled example’s Muon gradient over the current model and a short history of recent models before clipping. This places the temporal averaging inside the per-example computation. It differs from simply smoothing a batch update after clipping has already reweighted its contributors.

A useful algebraic view separates common scaling from the part that changes geometry. Write a clipped mean as the average of alpha-i times G-i, where alpha-i is an example’s clipping multiplier. If alpha-bar and G-bar are their respective averages, then

\[\frac1n \sum_i \alpha_i G_i = \bar\alpha \bar G + \frac1n \sum_i (\alpha_i - \bar\alpha)(G_i - \bar G).\]

The first term is a common rescaling. The second is a covariance-like residual that captures the association between gradient structure and clipping multipliers. The abstract describes a distortion bound using such a residual. The identity alone does not prove that any particular temporal averaging scheme reduces it.

The abstract also states that a finite Newton–Schulz iteration preserves the input polar factor under specified spectral conditions. That statement needs those conditions; this example’s exact closed form does not test a finite-iteration implementation.

What the preprint reports

The abstract reports GPT-2 fine-tuning on E2E and DART at epsilon values 1, 2, 4, and 8. It states that DP-Muon-IM improves BLEU and ROUGE-L over DP-Muon in every seed-matched comparison in those experiments, and reports 2–4% lower pre-noise polar error in non-private diagnostics. These are the authors’ reported results for the preprint’s evaluation, not a guarantee for other models or datasets.

This illustrative calculation adds no privacy noise and performs no training. Clipping alone is insufficient to establish differential privacy. A private implementation must analyze the full mechanism, including its noise, sampling, history dependence, and privacy accounting.

When to cite this work

Cite Inner Momentum for Differentially Private Muon when discussing its proposed per-example history averaging, its analysis of clipping-induced polar distortion, or comparisons to DP-Muon-IM. Identify it as an arXiv preprint and consult the current version for the exact assumptions and implementation. The elementary matrix example here explains the motivation; it does not establish the proposed method’s effectiveness.

]]>Bishnu BhusalPrivacy-preserving in-context learning: synthetic demonstrations and privacy budgets2026-09-08T09:00:00-06:002026-09-08T09:00:00-06:00https://bishnubhusal.com.np/blog/2026/privacy-preserving-in-context-learningIn-context learning gives a language model examples inside a prompt, allowing it to perform a task without updating its weights. When those examples contain private records, the prompt becomes part of the privacy problem. A model can repeat sensitive phrases, and simply asking it to paraphrase does not provide a mathematical privacy guarantee.

Our AAAI 2026 paper, Privacy Preserving In-Context-Learning Framework for Large Language Models, studies a different workflow: generate synthetic demonstrations through a differentially private mechanism, then reuse those demonstrations for downstream inference. This tutorial explains the mechanism and the boundary of its guarantee. The paper PDF and research code provide the full method.

What is being protected?

Differential privacy bounds how much an output distribution can change when one protected record changes. The paper uses add/remove adjacency: two datasets are neighbors when one contains one additional prompt record. A mechanism A is (epsilon, delta)-differentially private if, for every neighboring pair and every output event S,

\[\Pr[A(D)\in S]\leq e^{\varepsilon}\Pr[A(D')\in S]+\delta.\]

The guarantee concerns distributions, including unlikely outputs. It does not promise that sensitive-looking words can never appear. Delta is an additive allowance in this inequality; it should not be interpreted as the probability that a particular person’s record will be exposed. For the formal definition and post-processing property, see Chapter 2 of Dwork and Roth’s The Algorithmic Foundations of Differential Privacy.

The protected unit matters. If one person contributes many prompt records, a record-level guarantee is not automatically the same as a user-level guarantee. Deployment requires choosing the unit of protection before calibrating the mechanism.

How private demonstrations are generated

The paper’s generator consults several private prompts separately. At each decoding step, the language model produces a vector of logits for each prompt, conditioned on the synthetic prefix generated so far. A logit is a score for a possible next token, before conversion into probabilities.

The algorithm recenters and clips each logit vector, then averages the clipped private vectors. Bounding each vector’s coordinates controls the influence of one private prompt. Recentering and clipping have different effects: a uniform shift leaves softmax probabilities unchanged, while clipping can change them.

A second branch computes clipped logits using a public prompt and the same synthetic prefix. The algorithm averages the public vector with the private aggregate, then samples a token using temperature-scaled softmax. The paper analyzes this sampling through the exponential mechanism. It does not implement this step by adding Gaussian noise to logits.

The sampled token extends the prefix, and generation continues until an end-of-sequence token or a specified maximum length. The resulting synthetic examples become demonstrations for downstream ICL. No fine-tuning of the underlying language model is required by this workflow.

Public blending is useful because a language model’s public context can support task-relevant tokens without another direct consultation of private records. However, blending an unprotected private vector with a public vector does not, by itself, establish privacy. The guarantee depends on the sensitivity bound and randomized sampling together.

Where composition enters

Every new token can depend on the private prompts. Privacy accounting must therefore cover the whole sequence, including its adaptive dependence on earlier tokens.

A conservative calculation uses basic composition, developed alongside stronger composition results in Chapter 3 of Dwork and Roth. If each of T token mechanisms is epsilon-zero DP, with that guarantee holding conditional on every prior transcript, releasing the sequence is at most T times epsilon-zero DP. For example, 40 steps with a per-step budget of 0.02 give an overall bound of 0.8. These numbers illustrate composition; they are not settings or results from the paper.

More refined composition can improve the bound by allowing a nonzero delta. Any implementation must retain the terms and assumptions of the chosen accounting theorem. A useful operating rule is to set a maximum number of private token-generation steps before generation begins. Generating several synthetic examples from the same records also requires accounting across those examples, not just within each one.

Reusing a fixed, privately generated demonstration set is different. If subsequent inference uses only that release and information independent of the protected records, differential privacy’s post-processing property adds no privacy cost for those records. Creating a fresh demonstration set by consulting the records again is a new private computation.

The public branch and the trust boundary

The public branch can depend on the already generated private transcript. For analysis, one fixes that transcript and asks whether the branch makes any additional access to the protected dataset. A public prompt chosen from raw private records, or a model separately fine-tuned on those records without accounting, requires additional analysis.

The mechanism protects what is released through its analyzed output interface. The machine computing raw private logits still sees private prompts. Sending those prompts to a remote inference provider exposes them to that provider unless the deployment establishes a separate protection. Output DP does not make the raw prompts invisible to the service running inference.

Similarly, downstream queries may themselves contain sensitive information. The guarantee for the demonstration records does not automatically cover those queries. Logging raw prompts or intermediate unprotected logits also creates an output channel outside the intended release.

What the experiments establish

The paper evaluates synthetic demonstrations on AGNews, DBPedia, and TREC classification, plus MIT-G and MIT-D extraction tasks. Its main experiments use Gemma 1.1 2B IT and access to full token logits. At epsilon equal to 1, Table 1 reports better accuracy than the two Gemma-based baselines on three of the five tasks; performance is not uniformly best across all tasks.

These results support studying private prediction as a way to obtain useful demonstrations. They do not establish that arbitrary models, prompts, or application domains will have the same utility. An API exposing only generated text or a few token probabilities may also lack the interface required by the described method.

When to cite this work

Cite the AAAI paper when discussing private synthetic demonstrations for ICL, clipped logit aggregation with public-context blending, or reproducing and comparing this framework. Use the published DOI for the article and the repository for implementation details. General statements about differential privacy or composition should also cite their foundational sources.

]]>Bishnu BhusalDipApprox: verifying differential privacy with Gaussian noise2026-08-26T10:00:00-06:002026-08-26T10:00:00-06:00https://bishnubhusal.com.np/blog/2026/dipapprox-gaussian-differential-privacyA program can add Gaussian noise and still fail its intended differential privacy guarantee. The answer depends on what the program releases, how its branches interact, and how it reuses random values. Establishing privacy therefore requires reasoning about the complete computation.

In our CCS 2025 paper, Approximate Algorithms for Verifying Differential Privacy with Gaussian Distributions, we develop a way to automate that reasoning for a defined class of programs. Our tool, DipApprox (styled DiPApprox in the paper), computes probability intervals with certified error bounds and uses them to prove privacy, find a violation, or report that the current precision is insufficient.

I worked on this paper with Rohit Chadha, A. Prasad Sistla, and Mahesh Viswanathan. This visual walkthrough explains the method, its limits, and selected results. Read the paper, browse the code, or download the citation.

From a randomized program to a privacy answer

Consider a program that samples one noisy threshold and compares several independently perturbed queries against it. Even though the noise draws are independent, the comparison outcomes share the threshold. Computing the probability of a particular output sequence requires accounting for that shared random value.

This is the challenge behind examples such as Gaussian variants of the Sparse Vector Technique (SVT). SVT answers threshold questions while limiting how many positive answers it releases. The program’s output is discrete, but the random choices leading to it range over the real line.

DipApprox follows the program’s execution paths, expresses their probabilities as integrals, and encloses those probabilities in intervals. It then checks the differential privacy inequalities using the bounds.

DipApprox pipeline: a program and finite inputs produce execution paths, bounded integrals and tail bounds, probability intervals, and privacy-gap bounds, ending in DP, Not_DP, or Unknown.
Figure 1. The verification pipeline. Both integration error and probability in the tails remain accounted for in the final bounds.

The formal program class, DiPGauss, has finite input and output domains and is loop-free. Fixed bounded loops can be unrolled. Its presented syntax includes Gaussian and Laplace sampling; the underlying probability-approximation argument extends to suitable distributions with computable densities and finite means and variances. The prototype has additional restrictions, including support for comparisons among sampled values. These assumptions define what a successful check establishes.

Why a Gaussian tail cannot simply be ignored

For an elementary example, suppose a program releases whether a query value plus Gaussian noise exceeds a fixed threshold:

\[b=\mathbf{1}\{q+Z>T\},\qquad Z\sim\mathcal{N}(0,\sigma^2).\]

The probability of releasing 1 is an integral of the Gaussian density from \(T-q\) to infinity. More complicated programs introduce multiple random values and branch conditions, producing nested integrals. A numerical estimate alone is insufficient when a privacy inequality is close to its boundary.

Our approach separates a bounded integration region from its tails. It computes the bounded-region contribution with an error bound, then adds a bound on the probability outside that region. For multiple samples, the tail contribution accounts for the event that at least one draw falls outside its allowed window.

A standard Gaussian split at minus two and plus two standard deviations: about 95.45 percent of its mass is in the center, and 4.55 percent is in the two tails combined. The tail mass must be bounded, not discarded.
Figure 2. An illustrative integration window, chosen to make the tails visible. The paper's experiments use a wider Gaussian window of four standard deviations, not the two-standard-deviation window shown here. Tail truncation error is a numerical quantity, separate from the privacy parameter δ.

The theory uses tail inequalities, including Chernoff and Chebyshev bounds. The implementation uses FLINT for rigorous arithmetic and integral computation. A computed value is represented by a ball enclosure—a midpoint together with an error radius—rather than an unqualified floating-point estimate.

The resulting probability interval contains the true probability. The paper’s approximation interface can request an interval of width at most \(2^{-\rho}\), where increasing \(\rho\) asks for finer precision. Numerical approximation therefore does not weaken a certified privacy conclusion: uncertainty remains explicit in the bounds.

Checking privacy without enumerating every output event

Differential privacy compares the program’s output distributions on neighboring inputs. Let \(p_u(o)\) denote the probability of output \(o\) on input \(u\). For a fixed privacy budget \(\varepsilon_{\mathrm{prv}}\), define the directed privacy gap

\[\Delta(u,u')= \sum_o\max\left\{ p_u(o)-e^{\varepsilon_{\mathrm{prv}}}p_{u'}(o),\;0 \right\}.\]

The program satisfies \((\varepsilon_{\mathrm{prv}},\delta)\)-differential privacy when this gap is at most \(\delta\) for every required ordered adjacent pair. With symmetric adjacency, both directions matter.

Why sum only positive terms? The output event that maximizes the privacy violation includes exactly the outputs with positive contributions. This replaces checking every subset of the finite output space with a sum over individual outputs. Lemma 2 in the paper establishes the equivalence.

DipApprox computes lower and upper bounds on this gap. For each pair, an upper bound at or below \(\delta\) proves the inequality; a lower bound above \(\delta\) proves a violation. An interval crossing the boundary is inconclusive at the current precision. Equivalently, the largest gap over all required pairs must be at most \(\delta\); the figure illustrates bounds on that maximum.

Illustrative bounds on the maximum privacy gap against delta equal to 0.10: 0.04 to 0.08 certifies privacy, 0.12 to 0.16 certifies a violation, and 0.08 to 0.13 is inconclusive.
Figure 3. Three possible relationships between a certified interval and the privacy boundary. The numbers illustrate the decision rule; they are not experimental measurements or recommended privacy settings. A program-wide DP answer requires checking every pair in the specified adjacency relation; one violating pair is enough for Not_DP.

The paper distinguishes the mechanism’s noise-controlling parameter \(\varepsilon\) from its claimed privacy budget \(\varepsilon_{\mathrm{prv}}\). They need not be equal. A verification run fixes both, together with \(\delta\) and the adjacency relation.

What “almost decidable” means

The qualifier concerns termination at exact boundaries. For a fixed program, noise parameter, privacy budget, and adjacency relation, the theory identifies a finite set of critical \(\delta\) values. Away from that set, sufficiently precise probability bounds can decide the privacy question. At a critical value, continued refinement may remain inconclusive.

The answers DP and Not_DP are sound under the model’s assumptions. Unknown preserves that soundness when the current bounds cannot resolve the inequality. It is neither a privacy certificate nor evidence of a violation. This is also distinct from approximate differential privacy, where \(\delta\) appears in the guarantee itself.

Making the integrals practical

Nested integration can dominate runtime. The paper uses a dependency graph of integration limits to identify opportunities to separate integrations and reduce nesting while preserving their values. For the studied SVT-Gauss expressions, this optimization limits maximum nesting depth to three, independently of the number of queries.

Paper Table 2, SVT-Gauss with five queries and one selected adjacent pair: optimized integration took 7.9 seconds and reduced maximum nesting depth from six to three; unoptimized integration exceeded the 600-second timeout.
Figure 4. A result from Table 2 of the paper: SVT-Gauss, N = 5, one selected adjacent pair, ε = 0.5, ε_prv = 1.24, and δ = 0.01. The optimized runtime averages three runs on a 1.4 GHz quad-core Intel Core i5 Mac with 8 GB RAM. The unoptimized result is a timeout, not a measured completion time of 600 seconds.

The experiments also distinguish checking one pair from verifying all pairs in a finite domain. For SVT-Gauss with five queries, Table 1 reports 76.7 seconds for all pairs of binary input vectors under the experiment’s adjacency relation. For 25 queries, one selected pair took 441.3 seconds, while the all-pair run exhausted memory. In this benchmark, every pair of vectors in \(\{0,1\}^N\) is considered adjacent; the selected pair is the all-zero vector and the vector with only its final entry set to one.

The tool also detected violations in both Gaussian SVT leaky variants at the parameter settings reported in Table 1. These results demonstrate both proof and falsification on the benchmark programs. They also expose a practical limit: reducing integral depth does not remove the growth in the number of input combinations. The supported mathematical model is not a verification of arbitrary deployed Python or C++ implementations, including their floating-point sampling behavior.

Explore the artifact

The code repository contains the Python/C++ implementation, examples, and reproduction scripts. For the version associated with CCS 2025, use the archived artifact on Zenodo.

The repository documents this Docker workflow:

git clone https://github.com/bhusalb/approximate-dp.git
cd approximate-dp
docker build . -t dpapprox
docker run --rm -it dpapprox

Inside the container, start with python3 src/main.py --help to inspect the verification options. The documented benchmark entry points are scripts/benchmark.py and scripts/benchmark_unoptimized.py; scripts/table_generator.py generates Tables 1 and 2. Consult the artifact README for the current instructions and select the program, parameters, and input pairs deliberately when interpreting a result.

Paper and citation

Bishnu Bhusal, Rohit Chadha, A. Prasad Sistla, and Mahesh Viswanathan. “Approximate Algorithms for Verifying Differential Privacy with Gaussian Distributions.” ACM CCS 2025, pp. 1439–1453. DOI: 10.1145/3719027.3765043.

The author-hosted extended paper contains the definitions, proofs, and experimental details used here. See Sections 5–6 for the decision procedure and probability approximation, and Section 7 and Tables 1–2 for the results. Download BibTeX or RIS.

Cite this work when using its probability-approximation method, studying automated privacy verification for its supported Gaussian-sampling programs, or comparing against DipApprox. For foundational differential privacy definitions and composition, see Dwork and Roth’s The Algorithmic Foundations of Differential Privacy.

]]>Bishnu Bhusal