Sensitive data rarely stays where it was created. It moves into databases, logs, backups, queues, exports, analytics systems, and internal tools. Every copy creates another place where access can be misconfigured, credentials can be stolen, or retention can outlive its original purpose.
Access control decides who may request data. Cryptography constrains what a storage layer, intermediary, or unauthorized reader can learn or modify. Both are necessary. Neither repairs a system that exposes plaintext everywhere.
Encryption is only one form of protection. A system may need to recover a value, search for an exact match, display a small part, compute without decrypting, verify integrity, prove without revealing, commit to a prior choice, or reconstruct a secret under shared custody. Those operations require different techniques.
Each technique preserves a capability. Each also accepts leakage, complexity, or a new trust boundary. This article is a practical guide to choosing the minimum capability that the system must retain.
Unix tools survive because each does one thing and composes with the rest. Cryptographic primitives behave the same way. The failures come from asking one primitive to preserve every operation, or from keeping a capability the system never uses.
Preserve only the operation you need.
Contents↗
- Advanced Data Protection and Infrastructure Controls
- Format-Preserving Encryption
- Reversible Tokenization
- Data Masking
- MAC as a Service
- Blind Indexes
- Homomorphic Encryption
- Cryptographic Commitments
- Zero-Knowledge Proofs
- Authenticated Shamir Secret Sharing
- Encryption as a Service
- Digital Signatures
- Design Guidance
Advanced Data Protection and Infrastructure Controls↗
Advanced Data Protection is not one standard or product. In this article, it means protecting a value close to its application semantics. The protected form keeps only the operations the system can justify: recovery, equality, partial display, encrypted computation, verification without disclosure, opening, or threshold reconstruction.
Infrastructure controls protect different boundaries. TLS protects a channel. Disk encryption protects storage media. KMSs and HSMs protect keys. Secrets managers protect operational credentials. Access control governs actions. DLP tools observe or restrict movement.
These controls are necessary. They do not replace value-level protection, and value-level protection does not replace them.
Infrastructure protects the boundary. Data protection constrains the value.
| Control | Protects | Trust boundary | What it does not solve | Relationship to advanced data protection |
|---|---|---|---|---|
| TLS | Confidentiality and integrity while data moves between endpoints | The authenticated connection | Data at an endpoint, in storage, in logs, or available to an authorized process | Carries protected values but does not decide which operations remain possible |
| Encrypted disks | Lost devices, unmounted volumes, and offline media | The storage device and its unlock mechanism | Queries, malware, or application compromise after the disk is mounted | Complements field encryption, tokenization, and masking |
| Cloud KMS | Key versions, permissions, wrapping, and cryptographic operations | The managed key-control plane | Field semantics, plaintext handling, or the leakage accepted by an application | Supplies and controls keys for encryption, MAC, tokenization, and signature services |
| HSM | Key custody and cryptographic operations inside a hardware boundary | The hardware module and its authenticated clients | Protocol design, data formats, semantic authorization, or exposure policy | Can protect keys behind a KMS, encryption service, MAC service, or signer |
| Secrets manager | Distribution and rotation of passwords, API keys, certificates, and service credentials | Authenticated workloads and secret-delivery paths | General storage of PII, PANs, or application payloads | Protects credentials used to reach data-protection services; it is not a token store |
| Access control | Which identity may perform which action | The identity, policy, and enforcement system | Excessive permissions, an authorized malicious actor, or a compromised application with plaintext access | Restricts encode, decode, tokenize, resolve, sign, verify, and reconstruct operations |
| Database encryption | Database files, pages, backups, or selected columns, depending on the design | The database engine, key boundary, and query interface | TDE does not protect data returned by an authorized query; application-level encryption still exposes data where it decrypts | TDE protects storage; field or application encryption may become Advanced Data Protection when it preserves explicit operations |
| DLP tools | Discovery, classification, monitoring, or blocking of sensitive-data movement | Covered endpoints, applications, and egress paths | Cryptographic protection of stored values or complete detection without false positives and negatives | Observes and constrains exfiltration; it does not replace transformation of the value |
A Layered Example↗
client
|
TLS
v
[ application + access control ] <--- [ secrets manager ]
|
v
[ data protection services ] <------ [ KMS / HSM ]
|
v
[ database encryption ] ---> [ encrypted disk / backups ]
[ DLP observes export and egress paths ]Consider a customer record entering an application. TLS protects the request in transit. Access control decides whether the application may create, view, or recover each field. The secrets manager delivers operational credentials, not the customer data itself.
The data-protection layer then applies the operation required by each field. It may tokenize a PAN, mask an account number, create a blind index for an email, or encrypt a profile with AEAD. A KMS manages wrapping keys and policy. An HSM can keep high-value keys inside a hardware boundary and perform selected operations without exporting them.
Database encryption protects files and backups. Disk encryption protects the underlying media. DLP observes supported export and egress paths. None of those layers prevents a compromised application from exposing plaintext when that application is authorized to decrypt it. Recovery permissions and plaintext lifetime must still be minimized.
Defense in depth is not duplication. Each layer should fail differently.
| Technique | Reversible | Deterministic | Searchable | Integrity-focused | Format-preserving | Main leakage or limit |
|---|---|---|---|---|---|---|
| Format-Preserving Encryption | Yes | Yes, for a fixed key and tweak | Equality with the same key and tweak | No | Yes | Reveals equality and frequency; does not preserve semantic rules |
| Reversible Tokenization | Yes | Policy-dependent | Token lookup; equality with persistent tokens | No | Optional | The secure token store becomes a trust boundary; persistent tokens permit correlation |
| Data Masking | No, from the masked output | Yes, for a fixed policy | No | No | Often | The source value remains sensitive and accessible elsewhere |
| MAC as a Service | Not applicable | Yes, for the same key, context, and message | No | Yes, within a shared-key domain | Not applicable | Does not hide the message or identify an individual key holder |
| Blind Indexes | No | Yes, for a fixed key, context, and canonical input | Equality | No | No | Reveals equality and frequency; small domains permit guessing attacks |
| Homomorphic Encryption | Yes, by the key holder | No; encryption is randomized | Computation-specific, not general search | No; evaluation is malleable by design | No | High computation and ciphertext expansion; leaks parameters, metadata, and access patterns |
| Cryptographic Commitments | Opened later | No; secure schemes use a random opening | No | Binding, not source authenticity | No | Requires retention of the opening and careful handling of low-entropy values |
| Zero-Knowledge Proofs | Not applicable | Scheme-dependent | No | Statement validity, not source authenticity | No | Reveals public inputs, the proved relation, and protocol metadata |
| Authenticated Shamir Secret Sharing | Yes, with t valid shares | No; share generation is randomized | No | Share integrity, if authentication is correctly composed | No | Ceremony complexity; share authentication is not dealer verifiability |
| Encryption as a Service | Yes, when authorized | No, with proper nonce-based encryption | No | Yes, when AEAD is required | No | Key-service trust, availability, authorization, and metadata leakage |
| Digital Signatures | Not applicable | Algorithm-dependent | No | Yes, with public-key verification | No | Does not encrypt the payload; identity depends on key lifecycle and context |
Format-Preserving Encryption↗
Legacy fields often accept only a fixed length and alphabet. Format-Preserving Encryption (FPE) solves that integration problem. FF1 can map a 16-digit input to another 16-digit value without changing the database schema.
Definition: Domain and tweak. The domain is the complete set of valid inputs. A tweak is non-secret context that changes the FF1 permutation under the same key; it provides separation, but it does not replace the key.
PAN FF1(key, tweak) same-format value database
411111...1111 -> [ FPE service ] ----> 109876...9876 --------> [ store ]
^ |
+------------ authorized decrypt --------------------------+The values are illustrative, not an FF1 test vector.
Format is not semantics. FF1 preserves length and alphabet, but it does not automatically preserve a Luhn checksum, an issuer prefix, a valid range, or other field rules. Those constraints need an explicit domain design. Encrypting only selected digits also reduces the domain and can weaken security.
For a fixed key, tweak, and plaintext, FF1 is deterministic. Repeated values therefore produce repeated ciphertexts. Equality survives; frequency leaks.
NIST SP 800-38G is the current final recommendation. A 2019 Initial Public Draft of Revision 1 first turned the domain-size guidance for FF1 and FF3-1 into a requirement of one million. The 2025 Second Public Draft goes further: it removes FF3-1 entirely, raises the required FF1 domain size again, and disallows both the inverse-cipher shortcut and floating-point arithmetic. Treat this revision as draft guidance until it becomes final, but treat small domains as a real design risk now.
FPE protects confidentiality. Integrity needs a separate MAC, authenticated storage layer, or signed record.
Typical use cases
- PANs stored in legacy numeric columns.
- Fixed-length account numbers exchanged with older systems.
- Mainframe identifiers constrained by a fixed alphabet and width.
Example
Introduction. A payment platform is migrating its key management but cannot change a 16-digit field shared by several legacy systems.
Problem. Conventional ciphertext does not fit the schema. A schema migration would require coordinated changes across systems that cannot be deployed at the same time.
Context. The field must remain numeric. Its issuer prefix and Luhn check digit have application meaning. Repeated PANs must be handled under an explicit leakage model.
Solution. Apply FF1 only over a sufficiently large, well-defined portion of the PAN. Bind tenant and data class through the tweak. Preserve the issuer prefix only when routing requires it, and recompute the check digit outside FF1. Protect record integrity separately. This keeps the schema stable, but repeated inputs under the same key and tweak remain linkable.
Pros
- Preserves length and alphabet.
- Avoids coordinated schema changes.
- Supports authorized recovery of the original value.
Cons
- Reveals equality and frequency under a fixed context.
- Requires careful domain, tweak, and checksum design.
- Keeps reversible key material inside the trust boundary.
Reversible Tokenization↗
Most applications do not need the sensitive value. They need a stable handle for it.
Tokenization stores the plaintext in a secure token store and returns an unpredictable reference. Applications and databases keep the token. Only an authorized path can resolve it.
Definition: Token and secure token store. A token is an unpredictable reference with no useful plaintext meaning. The secure token store holds and protects the reversible mapping between that reference and the original value.
plaintext -> [ secure token store ] -> random token -> app/database
^ |
+---------------------+
controlled resolveA persistent token maps the same scoped value to the same token. It supports correlation across permitted workflows. A one-time token is limited to one use or a narrow lifetime and is invalidated by policy after that use.
Random-looking text is not enough. Confidentiality comes from the token store, unpredictable tokens, authorization, rate limits, encryption, and audit logs. Creation and detokenization should be separate permissions.
Tokenization moves exposure. It does not remove it.
Typical use cases
- Payment references used by billing and order services.
- PII identifiers shared across microservices without sharing plaintext.
- One-time tokens for narrow, short-lived retrieval workflows.
Example
Introduction. A billing system needs to charge the same payment method every month. Order, support, and analytics services only need a reference to it.
Problem. Copying the PAN into every service multiplies plaintext exposure and makes access control difficult to reason about.
Context. Recurring billing needs a stable reference. Support may display the last four digits, but only the payment service may recover the PAN.
Solution. Store the PAN once in a secure token store and issue a persistent, merchant-scoped token. Give billing permission to resolve it. Give other services only the token and separately stored display metadata. Audit and rate-limit every resolution. The token reduces distribution of plaintext, while the token store remains critical infrastructure.
Pros
- Removes plaintext from most application databases.
- Centralizes recovery policy and audit evidence.
- Supports persistent and one-time workflows.
Cons
- Adds a highly sensitive stateful service.
- Persistent tokens permit correlation inside their scope.
- Availability and authorization failures affect resolution.
Data Masking↗
Sometimes the operation is only display. A support agent may need the last four digits of an account, not the account itself.
4111-1111-1111-1111 -> [ keep last 4 ] -> ****-****-****-1111Masking reduces exposure in consoles, logs, screenshots, dashboards, and exports. The original value may still exist in a database or source system.
Note. Masking changes visibility, not custody. It does not protect the original value wherever that value remains accessible.
Typical use cases
- Support consoles that show only the last four digits.
- Application logs that redact credentials and personal identifiers.
- Operational exports that expose only the fields required by a recipient.
Example
Introduction. A support agent needs to confirm that a customer selected the correct account during a call.
Problem. Displaying the full account number gives the agent more information than the task requires and exposes it in screenshots and screen recordings.
Context. The support service can read an account record, but it never needs to submit the full number to another system.
Solution. Apply a display policy that returns only the last four digits and an account label. Keep the full value behind a separate authorization boundary. Mask logs and exports independently because masking one interface does not cover other paths.
Pros
- Reduces routine human exposure.
- Applies cleanly at display and export boundaries.
- Preserves the small amount of context needed for identification.
Cons
- Does not protect the source value.
- Can be defeated by combining several partially masked views.
- Requires consistent policies across every output path.
MAC as a Service↗
A Message Authentication Code (MAC) detects modification and authenticates a message inside a shared-key domain. HMAC and KMAC are common choices.
message + profile -> [ HMAC/KMAC service ] -> tag
message + tag -> [ verify same profile ] -> valid / rejectEvery holder of the shared key can create a valid tag. A MAC can show that a message came from that trust domain. It cannot identify which key holder created it, and it does not provide third-party verification.
Definition: Domain separation. Domain separation binds a MAC to one protocol and purpose through distinct derived keys, an unambiguous authenticated context, or an algorithm mechanism such as the KMAC customization string.
A signed profile can protect policy metadata such as the algorithm, key version, tag length, encoding, and intended domain. It does not create domain separation by itself.
The service must enforce separation cryptographically. With HMAC, use distinct derived keys or include an unambiguous context in the authenticated input. With KMAC, use its customization string. Canonical encoding matters: two fields must not be confused with one concatenated byte string.
MACs do not hide messages. Use encryption when the message is sensitive, or use AEAD when confidentiality and integrity belong to the same operation.
Typical use cases
- Webhook authentication between services that share a secret.
- Integrity protection for messages on an internal queue.
- Authentication tags for records processed inside one trust domain.
Example
Introduction. A service sends payment-status webhooks to another internal system.
Problem. TLS protects the connection, but the receiver also needs to reject modified, replayed, or cross-protocol requests.
Context. Both services can share a key. The request method, path, timestamp, event identifier, and body all affect its meaning.
Solution. Define a signed MAC profile and a canonical encoding for every field. Derive a webhook-specific HMAC key, or use a KMAC customization string. Authenticate the timestamp and event identifier with the request. Verify the tag in constant time, enforce a short time window, and reject duplicate event IDs. The MAC authenticates the shared-key domain; replay defense remains protocol state.
Pros
- Provides fast integrity and shared-key authentication.
- Keeps raw MAC keys out of application processes.
- Centralizes algorithm, domain, and tag-length policy.
Cons
- Cannot identify which shared-key holder created a tag.
- Requires key distribution or a highly available MAC service.
- Does not provide confidentiality or replay protection by itself.
Blind Indexes↗
Encrypted values are difficult to query. A blind index preserves one narrow operation: equality search.
First canonicalize the value. Then apply a keyed pseudorandom function, usually a MAC, with a purpose-specific key and context. Store the result beside the encrypted record.
Definition: Canonicalization and keyed PRF. Canonicalization maps one logical value to one byte representation. A keyed pseudorandom function (PRF) then maps those bytes deterministically to an output that appears random without the key.
email -> [ canonicalize ] -> [ keyed PRF ] -> blind index -> equality query
query -> [ same process ] -----------------> lookup valueCanonicalization is part of the protocol. Case folding, Unicode normalization, whitespace, and field encoding must produce the same bytes during writes and queries. Index truncation also needs a collision budget and an explicit failure strategy.
Determinism buys lookup by spending privacy.
Equal plaintexts produce equal indexes. Frequency remains visible. Small domains can permit dictionary attacks if the key is exposed or the indexing service can be queried as an oracle.
A blind index does not encrypt the record. It is not Private Information Retrieval, Fully Homomorphic Encryption, or a general encrypted-query system.
Typical use cases
- Exact lookup of encrypted records by email or phone number.
- Uniqueness checks without storing the comparison value in plaintext.
- Deduplication within a defined tenant or data class.
Example
Introduction. A customer service must find an encrypted profile from an email address supplied during login.
Problem. Randomized encryption produces a different ciphertext each time, so the database cannot use it for equality lookup.
Context. Only exact search is required. Email normalization rules are stable, and profiles are already protected with AEAD.
Solution. Canonicalize the email, compute an HMAC with a tenant- and field-specific index key, truncate it according to a collision budget, and store the index beside the ciphertext. Include a key version and use dual indexes during rotation. This enables equality search but exposes repeated emails inside the index scope.
Pros
- Adds indexed equality search to randomized encrypted records.
- Keeps the search key separate from the encryption key.
- Supports scoped equality and uniqueness checks.
Cons
- Reveals equality and frequency.
- Requires exact, durable canonicalization rules.
- Makes rotation and collision handling part of the data model.
Homomorphic Encryption↗
Homomorphic Encryption evaluates a function over ciphertext. After decryption, the result is equivalent to evaluating that function over the plaintext.
Definition: Circuit. A circuit is a fixed sequence of arithmetic or Boolean operations that represents a computation. Circuit depth is the longest chain of dependent operations that must be evaluated.
plaintext -> [ encrypt ] -> ciphertext
|
v
[ evaluate f ]
|
v
encrypted result -> [ decrypt ] -> f(plaintext)Definition: Noise budget and bootstrapping. Homomorphic operations consume a ciphertext’s noise budget; decryption fails when noise exceeds the selected parameters. Bootstrapping refreshes a ciphertext so more operations can be evaluated, at substantial computational cost.
Schemes can be classified by the computations they support:
- Partially Homomorphic Encryption preserves one operation, such as addition or multiplication.
- Somewhat Homomorphic Encryption supports limited combinations of operations before exhausting its noise budget.
- Leveled Homomorphic Encryption evaluates circuits up to a selected depth without bootstrapping.
- Fully Homomorphic Encryption supports arbitrary circuits, usually by using bootstrapping to control accumulated noise.
They can also be classified by their computation model:
- Exact modular arithmetic for counts and discrete aggregates.
- Approximate arithmetic for statistics, vectors, and machine learning.
- Boolean or gate-oriented operations for comparisons and logic.
These categories overlap, and terminology varies between schemes. The useful question is not whether a system is “fully” homomorphic. It is whether the required computation, depth, precision, and security parameters fit the operational budget.
A blind index preserves equality lookup. Homomorphic Encryption preserves a class of computations. It is not Private Information Retrieval or a general private query mechanism.
NIST is supporting the development of FHE toward future standards. There is no general FIPS for Homomorphic Encryption.
Typical use cases
- Aggregations over encrypted values.
- Private inference with a fixed model.
- Delegated computation on infrastructure that must not receive plaintext.
- Processing regulated data while keeping the decryption key elsewhere.
Example
Introduction. A medical organization needs to calculate a risk score from clinical data using external compute infrastructure.
Problem. The compute provider must not receive the clinical records in plaintext.
Context. The scoring function is fixed and can be expressed with arithmetic supported by the selected scheme. The organization retains the decryption key.
Solution. Encrypt the inputs, evaluate the scoring circuit over ciphertext, and return an encrypted score. Only the key holder can decrypt the result. The design still needs authenticated inputs and outputs, parameter governance, and controls for metadata and access patterns.
Pros
- Computes without giving plaintext to the evaluator.
- Separates possession of data from the ability to process it.
- Can reduce plaintext exposure in analytics and model inference.
Cons
- Adds substantial computation, memory use, and ciphertext expansion.
- Requires deliberate parameter, depth, and numeric-representation choices.
- Approximate arithmetic introduces controlled error.
- Is malleable by design and does not authenticate the result by itself.
- Does not automatically hide sizes, circuits, timing, or access patterns.
Cryptographic Commitments↗
A commitment records a choice without revealing it. Later, the committer reveals the value and a random opening so anyone can verify the original choice.
Definition: Opening. An opening is the secret randomness and any auxiliary data needed to reveal a committed value and verify it against the commitment. It is not the committed plaintext by itself.
value + random opening -> [ commitment scheme ] -> commitment C
value + opening + C -> [ verify opening ] ----> valid / rejectHiding prevents early disclosure. Binding prevents the same commitment from being opened to a different value. The exact guarantees depend on the scheme.
Use a reviewed commitment construction. A casual
hash(value || random_value) design is not automatically safe: low-entropy
values permit guessing, weak randomness harms hiding, and ambiguous encodings
can change what was committed. Use a high-entropy opening, canonical encoding,
and domain separation.
Binding proves consistency with an earlier commitment. It does not authenticate who created it. Identity needs a signature or an authenticated channel.
Typical use cases
- Sealed bids that open after a deadline.
- Commit-reveal protocols for choices or randomness contributions.
- Audit workflows that disclose a value only after an approval stage.
Example
Introduction. Participants in an auction must submit bids before a deadline without revealing them to other participants.
Problem. Publishing bids changes later behavior. Keeping them only in a private database gives participants no evidence that bids were fixed before the deadline.
Context. Bid values come from a small domain and are therefore guessable. Each participant already has an authenticated identity.
Solution. Use a reviewed commitment scheme with a high-entropy random opening and canonical bid encoding. Publish the commitment before the deadline. After closing, reveal the bid and opening and verify them against the published commitment. Sign the commitment when participant identity must also be proven. Define what happens when a participant refuses to open.
Pros
- Separates choosing a value from revealing it.
- Provides verifiable hiding and binding properties.
- Allows commitments to be published on an untrusted log.
Cons
- Requires secure retention of the opening until disclosure.
- Does not authenticate the committer by itself.
- Needs an explicit policy for missing or invalid openings.
Zero-Knowledge Proofs↗
A Zero-Knowledge Proof (ZKP) lets one party prove a claim while disclosing less data to the verifier.
Definition: Witness and public statement. The witness is the private data used to construct the proof. The public statement is the claim and public input that the verifier checks without learning that witness.
Three properties define the model:
- Completeness: an honest proof for a valid statement can be accepted.
- Soundness: a false statement should not produce an accepted proof.
- Zero knowledge: the proof reveals nothing about the witness beyond the validity of the statement.
private witness + public statement
|
v
[ prover ] -> proof
|
public statement + proof -> [ verifier ] -> valid / rejectDefinition: Trusted setup. A trusted setup generates public parameters for a proof system. If its secret trapdoor material survives or becomes known to an attacker, the system’s soundness may fail.
Proof systems differ along independent axes:
- Interactive or non-interactive: the proof requires a dialogue with the verifier or can be transferred as one artifact.
- Specialized or general-purpose: the system proves one relation or a computation represented as a circuit.
- Trusted setup or transparent: security depends on generated parameters or avoids that setup assumption.
- Succinct or proof-size tolerant: the design prioritizes small proofs and fast verification or accepts larger artifacts for other properties.
SNARK and STARK are useful family labels, but they are neither universal nor mutually exclusive. Concrete systems make different choices about setup, assumptions, proof size, prover cost, and post-quantum security.
A commitment fixes a value for a later opening. A ZKP proves a statement about a witness without revealing it. The proof establishes the encoded relation; it does not establish that an issuer, sensor, or external input was truthful.
NIST is supporting the development of ZKP systems toward future standards. There is no general FIPS for Zero-Knowledge Proofs.
Typical use cases
- Proving age, residence, or eligibility without revealing full identity data.
- Proving knowledge of a key or credential without exposing it.
- Demonstrating correct execution of a computation.
- Proving that a committed or encrypted value satisfies a rule.
Example
Introduction. A user needs to prove that they meet a minimum-age requirement.
Problem. Sharing a birth date reveals more information than the verifier needs.
Context. A trusted authority issued a signed credential containing the birth date. The verifier trusts that issuer and accepts the selected proof system.
Solution. Prove that the credential has a valid issuer signature and that the hidden birth date satisfies the age threshold. Reveal only the required public statement. The proof does not repair a false credential, an untrusted issuer, or an incorrect age rule encoded in the circuit.
Pros
- Verifies statements with minimal disclosure.
- Proves knowledge, membership, or correct execution without exposing a witness.
- Composes with credentials, commitments, and encrypted computation.
Cons
- Proof generation can be expensive.
- A valid proof can enforce an incorrectly specified circuit.
- Some systems require a trusted setup.
- Public inputs, metadata, and persistent identifiers can enable linkability.
- Post-quantum security depends on the proof system; ZKP does not imply it.
Authenticated Shamir Secret Sharing↗
Shamir Secret Sharing distributes recovery authority across multiple holders. Reconstruction succeeds only after enough compatible fragments are combined.
Definition: Share and threshold. A share is one randomized fragment produced by the split. The threshold
tis the minimum number of compatible shares required to reconstruct the secret.
secret
|
[ split: n=4, t=3 ]
/ | | \
S1* S2* S3* S4* * authenticated share
\ | /
[ any 3 valid shares ]
|
reconstructed secret“Authenticated Shamir” is a composition, not one universal construction. A share can be packaged with a MAC, a dealer signature, or authenticated metadata. The authentication key must be independent of the share and available when the share is checked.
Note. Share authentication detects corrupted or substituted shares. It does not prove that a dealer created one consistent polynomial and distributed compatible shares; that requires Verifiable Secret Sharing or another robust protocol.
The operational problem is the ceremony: holder identity, isolated custody, share refresh, lost shares, reconstruction devices, and audit evidence. The threshold is only useful if the process preserves it.
Typical use cases
- Offline recovery of a root encryption or signing key.
- Break-glass credentials held by independent custodians.
- Backup of a master secret without a single recoverable copy.
Example
Introduction. An organization needs a recovery path for an offline root key without giving one administrator unilateral access.
Problem. A single backup creates one point of compromise. Requiring every custodian makes recovery fragile when someone is unavailable.
Context. Five custodians work in separate teams. Recovery is rare, occurs on an isolated device, and must tolerate two unavailable shares.
Solution. Create a 3-of-5 Shamir split and package each share with a dealer
signature or independently verifiable authentication metadata. Store shares in
separate physical and administrative domains. Verify each package before
reconstruction, record the ceremony, and destroy reconstructed plaintext after
use. If a malicious dealer is in scope, use verifiable secret sharing rather
than assuming share authentication proves consistency.
Pros
- Removes any single custodian as a recovery authority.
- Tolerates loss or unavailability below the threshold.
- Makes recovery an explicit, auditable event.
Cons
- Introduces ceremony, holder, and share-lifecycle complexity.
- Exposes the complete secret at the reconstruction endpoint.
- Needs a separate design for authentication and dealer verifiability.
Encryption as a Service↗
Applications need encryption, but they should not each invent key management. An encryption service can enforce algorithms, key versions, authorization, and audit policy.
Definition: AEAD and AAD. Authenticated Encryption with Associated Data (AEAD) protects ciphertext confidentiality and integrity. Associated Data (AAD) is authenticated with the ciphertext but remains visible.
Definition: DEK and envelope encryption. A Data Encryption Key (DEK) encrypts the payload. Envelope encryption stores that ciphertext with a DEK wrapped under a separate key managed by a KMS or HSM boundary.
central API:
plaintext + AAD -> [ AEAD service ] -> nonce + ciphertext + tag -> storage
envelope encryption:
plaintext -> [ app encrypts with DEK ] -> ciphertext
DEK -> [ KMS/HSM wraps key ] ----> wrapped DEK -> storageThe models have different trust boundaries. A central API sees plaintext. With client-side envelope encryption, the KMS or HSM can wrap the DEK without receiving the payload.
AEAD is the baseline for new designs. It binds ciphertext to associated data such as tenant, record type, schema version, or object identifier. The stored envelope must carry enough metadata to recover the algorithm and key version. Nonce generation must follow the selected AEAD requirements.
Decryption must fail closed when authentication fails. Availability, authorization, rotation, revocation, and audit are part of the cryptographic boundary because every recovery path passes through them.
Typical use cases
- Field-level encryption of PII in application databases.
- Envelope encryption for documents, backups, and object storage.
- Tenant-scoped encryption in multi-tenant platforms.
Example
Introduction. A multi-tenant application stores customer profiles containing personal and contractual data.
Problem. Each service has implemented encryption differently. Keys, nonces, associated data, and rotation policy are inconsistent.
Context. Payloads can be encrypted inside the application. The key service should enforce tenant isolation without receiving profile plaintext.
Solution. Generate a DEK in the application, encrypt the profile with AEAD, and bind tenant ID, record ID, and schema version as associated data. Ask the KMS/HSM boundary to wrap the DEK under a tenant-scoped key. Store the wrapped DEK, nonce, ciphertext, tag, algorithm, and key version together. Audit unwrap operations and fail closed on any authentication error.
Pros
- Standardizes algorithms, key versions, and authorization policy.
- Supports rotation without re-encrypting every payload immediately.
- Keeps payload plaintext outside the wrapping-key service.
Cons
- Adds latency and availability dependencies to key operations.
- Still exposes plaintext and the DEK inside the application boundary.
- Leaks envelope metadata such as tenant, size, timing, and key version.
Digital Signatures↗
Digital signatures provide integrity and public-key authentication. Anyone with the trusted public key can verify a signature; only the private-key holder should be able to create one.
Model↗
EdDSA is a classical signature family built over Edwards-form elliptic curves. NIST FIPS 186-5 specifies Ed25519 and Ed448. Their implementations are mature, their keys and signatures are compact, and their performance fits online protocols.
ML-DSA and SLH-DSA address a different threat. They are post-quantum signature algorithms standardized in NIST FIPS 204 and FIPS 205. ML-DSA is based on module lattices. SLH-DSA is based on hash functions.
NIST standardizes the individual algorithms. It does not standardize an EdDSA + ML-DSA or EdDSA + SLH-DSA bundle. A hybrid signature is therefore a protocol profile, not a new NIST algorithm.
Definition: Canonical transcript. A canonical transcript is the exact, unambiguous byte representation covered by every signature. It binds the payload to its protocol context, version, algorithm suite, and policy.
In a hybrid profile, both algorithms sign that same canonical transcript. The verifier requires both signatures and rejects missing, substituted, duplicated, or unknown components.
payload + context + suite
| |
v v
[ EdDSA ] [ ML-DSA ]
| |
+----+-----+
v
[ bundle: algorithms + key IDs + signatures ]
|
v
[ require both valid ]The intended property is simple: a forgery must satisfy both verification checks. That property depends on canonical encoding, key binding, strict verification, and downgrade-resistant negotiation. Concatenating two signatures without defining those rules is not a secure hybrid protocol.
Note. Downgrade resistance prevents a verifier from silently accepting a weaker signature policy. Crypto agility permits deliberate, versioned changes to algorithms without making that fallback implicit.
Why EdDSA with ML-DSA↗
For most general-purpose systems, EdDSA + ML-DSA is the preferred default.
EdDSA preserves a compact and widely deployed classical path. ML-DSA adds a NIST-standardized post-quantum path with a more practical balance of signature size and throughput than SLH-DSA for frequent operations. The combination fits APIs, release systems, policy distribution, and infrastructure protocols where latency and bandwidth still matter.
Requiring both is designed to preserve authenticity while at least one component remains unforgeable. This is conditional, not automatic: the bundle format and verification policy are part of the security argument.
The cost is structural. A hybrid system operates two key lifecycles, emits two signatures, performs two verifications, and needs an explicit migration and failure policy.
Risk-Based Selection↗
| Risk profile | Classical signature | Post-quantum signature | Typical fit |
|---|---|---|---|
| General-purpose | Ed25519 | ML-DSA-44 | APIs, frequent releases, and latency-sensitive protocols |
| Higher assurance | Ed448 | ML-DSA-65 | Durable artifacts, infrastructure, and high-value policy |
| Maximum PQ margin | Ed448 | ML-DSA-87 | Roots and long-retention archives where additional cost is acceptable |
| Hash-based diversity | Ed25519 or Ed448 | Matching SLH-DSA profile | Infrequent signing where hash-based assumptions take priority |
This is a deployment matrix, not a claim that security categories and elliptic curve strength are interchangeable. Select a profile from the threat model, artifact lifetime, signing frequency, bandwidth, and verifier capacity.
ML-DSA and SLH-DSA↗
ML-DSA is the general-purpose post-quantum choice. Its module-lattice design offers a better size and performance balance for services that sign or verify often. This makes it the natural partner for EdDSA in most hybrid profiles.
SLH-DSA makes a different trade. Its security is built from hash-based constructions, providing cryptographic diversity from lattice assumptions. Its signatures are much larger, and signing can be substantially more expensive. That cost can be reasonable when signatures are rare and artifacts live for a long time.
SLH-DSA parameter names ending in s prioritize smaller signatures. Names
ending in f prioritize faster signature generation at the cost of larger
signatures. Neither family is universally stronger; they optimize different
constraints.
SLH-DSA is a better fit for:
- Offline root keys and trust anchors that sign infrequently.
- Firmware or software releases produced in controlled signing ceremonies.
- Long-lived artifacts where hash-based cryptographic diversity matters.
- Systems that can absorb large signatures but want to avoid a lattice-only post-quantum strategy.
NIST SP 800-230 proposes additional SLH-DSA parameter sets for
limited-signature use cases such as software, firmware, and certificates. They
trade a hard limit of 2^24 signatures per key for faster verification and
smaller signatures. It is an Initial Public Draft, not a general-purpose final
standard.
A signature proves control of a key. Identity comes from the system around it.
Signatures can provide evidence to a third party, but non-repudiation also depends on identity proofing, key custody, revocation, timestamps, and operating procedure. Signatures do not encrypt the payload.
Typical use cases
- APIs and policy bundles that need frequent hybrid verification.
- Software releases and update manifests retained for years.
- Offline roots, firmware, and trust anchors that benefit from hash-based diversity.
Example
Introduction. A software publisher wants release artifacts to remain verifiable while its ecosystem migrates toward post-quantum signatures.
Problem. EdDSA is widely deployed, but it does not provide post-quantum security. Replacing it immediately would break existing integrations.
Context. Every release already has a canonical manifest containing artifact hashes and version metadata. Releases are frequent, bandwidth matters, and the artifacts may remain trusted for years. The balanced profile in the risk matrix fits better than a larger hash-based signature.
Solution. Define a versioned hybrid transcript that binds the manifest, release context, verification policy, and algorithm suite. Sign it with Ed25519 and ML-DSA-44. Put algorithm IDs, key IDs, and both signatures in one bundle. Hybrid-aware verifiers require both. During a bounded migration, legacy clients may consume a separately identified Ed25519 signature under an explicit sunset policy. A high-assurance or low-frequency release process can select another profile from the matrix instead of changing the transcript format.
Pros
- Enables public verification without a shared secret.
- Combines a mature classical path with a standardized post-quantum path.
- Makes the algorithm suite explicit and replaceable through crypto agility.
- Uses ML-DSA where size and performance matter, while preserving an SLH-DSA option for hash-based diversity.
Cons
- Increases signature size, verification cost, and protocol complexity.
- Requires interoperable encoding and strict downgrade-resistant verification.
- Doubles key generation, protection, rotation, revocation, and audit concerns.
- Still depends on identity proofing and trusted public-key distribution.
Design Guidance↗
Start with the operation that must survive.
| Problem | Choice | Tradeoff |
|---|---|---|
| Store a PAN in a legacy numeric field | FPE/FF1 | Fixed format for equality leakage and domain constraints |
| Remove plaintext from application databases | Tokenization | Controlled resolution through a token store dependency |
| Show an account reference to support staff | Data masking | Partial display while the source remains sensitive |
| Authenticate a webhook inside one trust domain | MAC as a Service | Shared-key verification without public attribution |
| Search for an exact email | Blind index | Exact lookup for equality and frequency leakage |
| Compute a defined function without exposing inputs | Homomorphic Encryption | Encrypted computation for performance, parameter, and metadata costs |
| Prove a value existed before disclosure | Commitment | Later opening without identity proof |
| Prove that a private value satisfies a rule | Zero-Knowledge Proof | Minimal disclosure with proof-system and circuit complexity |
| Remove any single root-key custodian | Authenticated Shamir Secret Sharing | Threshold recovery with ceremony complexity |
| Standardize application encryption | Encryption service | Policy-controlled recovery through a trusted service |
| Authenticate releases across organizations | Digital signatures | Public verification tied to identity and key lifecycle |
No primitive covers every property. Composition is where most design mistakes appear: deterministic data without a leakage model, ciphertext without authentication, signatures without context, or recovery paths without policy.
The primitive matters. The boundary around it matters more.