Encryption is not a new concept to MongoDB. Encryption may occur in-transit (with TLS) and at-rest (with the encrypted storage engine). But MongoDB 4.2 introduces support for Client Side Encryption, ensuring the most sensitive data is encrypted before ever leaving the client application. Even full access to your MongoDB servers is not enough to decrypt this data. And better yet, Client Side Encryption can be enabled at the "flick of a switch". This session covers using Client Side Encryption in your applications. This includes the necessary setup, how to encrypt data without sacrificing queryability, and what trade-offs to expect.

2. Introducing…

4. db.coll.insert({
_id: 1,
name: "Doris",
ssn: "457-55-5462"
})

6. doc = db.coll.find_one({
ssn: "457-55-5462"
})

8. print (doc)

9. {
_id: 1
name: "Doris",
ssn: "457-55-5462"
}

12. db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
{
insert: "coll",
documents: [{
name: "Doris",
ssn: BinData(6, "a10x…")
}]
}
You see: MongoDB sees:
Encrypt before sending

13. {
_id: 1
name: "Doris",
ssn: BinData(6, "a10x…")
}
Driver receives: You see:
{
_id: 1
name: "Doris",
ssn: "457-55-5462"
}
Decrypt after receiving

14. How does this differ from…?
•… encryption in-transit (TLS)
•… encryption at-rest (encrypted storage engine)

15. Attacker
query
App
(Client)
Disk
insert write
MongoDB
Auth
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })

16. Disk
insert write
MongoDB
Attacker
snoop
TLS
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

17. Disk
insert write
MongoDB
Attacker
insert
TLS
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

18. Disk
insert write
MongoDB
Attacker
steal
ESE
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

19. Disk
insert write
MongoDB
Attacker
login
Client Side Encryption
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

20. Disk
insert write
MongoDB
Boundaries of unencrypted data
App
(Client)

21. Disk
insert write
MongoDB
… with Encrypted Storage Engine
App
(Client)

22. Disk
insert write
MongoDB
… and TLS
App
(Client)

23. Disk
insert write
MongoDB
with Client Side Encryption
App
(Client)

25. Disk
insert write
MongoDB
ssn: BinData(6, "a10x…")
App
(Client)

26. db.coll.update({}, {
$set: { ssn: "457-55-5462" }
})
{
update: "coll",
updates: [{
q:{},
u: {
$set: { ssn: BinData(6, "a10x…") }
}
}]
}
You see: MongoDB sees:
Update that overwrites value

27. db.coll.aggregate([{
$project: { name_ssn: {$concat: [ "$name", " - ", "$ssn" ] } }
}]
Aggregate acting on the data

28. Find with equality query
* For deterministic encryption
db.coll.find({ssn: "457-55-5462" }) {
find: "coll",
filter: { ssn: BinData(6, "a10x…") }
}
You see: MongoDB sees:

29. Find with equality query
* For deterministic encryption
db.test.find(
{
$and: [
{
$or: [
{ ssn : { $in : [ "457-55-5462", "153-96-2097" ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
)
You see:

30. Find with equality query
* For deterministic encryption
MongoDB sees:
{
find: "coll",
filter: {
$and: [
{
$or: [
{ ssn : { $in : [ BinData(6, "a10x…"), BinData(6, "8dk1…") ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
}

31. MongoDB
Attacker
Login

33. Destroy the key
Provably delete all user data.
GDPR "right-to-be-forgotten"

36. Doris
Private stuff in storage

37. PoliceDoris
Private stuff in storage

38. Vault key
Held only by you
Vault

40. Encrypted Data
MongoDB
Encryption Key

42. { _id: 1, ssn: BinData(0, "A81…"), name: "Kevin" }
{ _id: 2, ssn: BinData(0, "017…"), name: "Eric" }
{ _id: 3, ssn: BinData(0, "5E1…"), name: "Albert" }
…

44. client = MongoClient(
auto_encryption_opts=opts)

45. Not sensitive
{

46. One key for all vaults

47. One key per vault

48. {
name: "Doris"
ssn: "457-55-5462",
email: "Doris@gmail.com",
credit_card: "4690-6950-9373-8791",
comments: [ …. ],
avatar: BinData(0, "0fi8…"),
profile: { likes: {…}, dislikes: {…} }
}

51. Describes JSON
{
bsonType: "object",
properties: {
a: {
bsonType: "int"
maximum: 10
}
b: { bsonType: "string" }
},
required: ["a", "b"]
}
{
a: 5,
b: "hi"
}
{
a: 11,
b: false
}
JSON Schema

52. {
bsonType: "object",
properties: {
ssn: {
encrypt: { … }
}
},
required: ["ssn"]
}
JSON Schema "encrypt"

53. encrypt: {
keyId: (…),
algorithm: (…),
bsonType: (…)
}
bsonType indicates the type of underlying data.
algorithm indicates how to encrypt (Random or Deterministic).
keyId indicates the key used to encrypt.

54. opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> }
…)

55. Remote Schema Fallback
db.createCollection("coll", { validator: { $jsonSchema: … } } )
Misconfigured
Client insert "457-55-5462"
error, that should be
encrypted
MongoDB

56. What if
… the server lies about the schema?
Misconfigured
Client insert "457-55-5462"
Evil MongoDB
ok :)

57. schema_map
Sub-options

59. Key vault
Key vault key
Held only by you

62. Stores encrypted keys

63. opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keyvault"
…)

64. schema_map
Sub-options
key_vault_namespace

65. What if
… attacker drops key vault collection?

66. Keep at home
opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keyvault",
key_vault_client = <client>
…)

67. schema_map
Sub-options
key_vault_namespace
key_vault_client

69. (Key Management Service)

70. Protects keys Stores keys
KMS
Key vault key
Key vault
Key vault
collection

71. Decryption requires

72. opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keys",
kms_providers = <creds>
…)

73. schema_map
Sub-options
key_vault_namespace
key_vault_client
kms_providers

75. db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Get encrypted key
Decrypt the key with KMSDecrypt the key with KMS
Encrypt 457-55-5462
Send insert
Compare to JSON schema

77. You need…
•MongoDB 4.2 server
•Beta client (shell, Java, Python, NodeJS, Go, C#)
•Enterprise license for auto encryption
•Community for explicit encryption

79. *

81. Authenticated Encryption with Associated Data using the Advance

82. AEAD_AES_256_CBC_HMAC_SHA_512
Provides confidentiality + integrity

83. AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic
AEAD_AES_256_CBC_HMAC_SHA_512-Random

84. coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
You see: MongoDB stores:
coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "f991…") }
…Random

85. coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
You see: MongoDB stores:
coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
…Deterministic

86. Can be queried
doc = db.coll.find({
ssn: "457-55-5642"
})
…Deterministic

87. Only for binary comparable types.
db.coll.find({ a: NumberDecimal("1.2")})
{ a: NumberDecimal("1.2") }
{ a: NumberDecimal("1.20") }
MongoDB returns:
…Deterministic

88. { a: BinData(6, "b515…") }
{ a: BinData(6, "801f…") }
Encrypted as:
{ a: NumberDecimal("1.2") }
{ a: NumberDecimal("1.20") }
Value:

89. db.coll.find({ a: NumberDecimal("1.2") })
{ a: NumberDecimal("1.2") }
MongoDB returns:
"a" encrypted

90. Encrypt-able values
Deterministic encryption valid for…
•String
•Binary
•ObjectID
•Date
•Regex
•DBPointer
•Javascript
•Symbol
•Int
•Timestamp
•Long
Random encryption valid for…
•(all of deterministic)
•Document
•Array
•JavascriptWithScope
•Double
•Decimal128
•Bool

92. { ssn: BinData(6, "AWNkTYTCw89Ss1DPzV3/2pSRDNGNJ9NB" }
New binary subtype
Older drivers and older MongoDB will treat as a black box.

93. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Ciphertext

94. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
key_id + algorithm describes how to decrypt.
No JSON Schema necessary!
Ciphertext

95. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Provides extra server-side validation.
But prohibits single-value types (MinKey, MaxKey, Undefined, Null)
Ciphertext

96. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Payload includes encoded IV and padding block, and HMAC.
Ciphertext adds between 66 to 82 bytes of overhead.
Ciphertext

98. ce = ClientEncryption(opts)
encrypted = ce.encrypt("457-55-5462", opts))
decrypted = ce.decrypt(BinData(6, "a10x…")))
id = ce.createDataKey(opts)

100. db.test.find({
$or: [
{ ssn : { $in : [ "457-55-5462", "153-96-2097" ]} },
{ name: "Doris" }
]
})
db.test.find({
$or: [
{ ssn : { $in : [ BinData(6, "a10x…"), BinData(6, "8dk1…") ]} },
{ name: "Doris" }
]
})

101. Limitations
If we cannot parse…
or it is impossible…
we err for safety.

102. { passport_num: "281-301-348", ssn: "457-55-5462" }
{ passport_num: "390-491-482", ssn: "482-38-5899" }
{ passport_num: "104-201-596" }
passport_num and ssn encrypted with different keys

103. db.test.aggregate([
])
{ $project: { identifier: { $ifNull: ["$ssn", "$passport_num" ] } } },
{ $match: { identifier: "457-55-5462" } }
How do we encrypt 457-55-5462?

104. opts = AutoEncryptionOptions(
bypass_auto_encryption = True
…)
client = MongoClient(auto_encryption_opts=opts)
(Decryption still occurs)

105. ce = ClientEncryption(opts)
db.test.aggregate([
])
{ $project: { identifier: { $ifNull: ["$ssn", "$passport_num" ] } } },
{ $match: { identifier: ce.encrypt("457-55-5462", opts) } }

107. One key
ce = ClientEncryption(opts)
id = ce.createDataKey(keyopts)
Create with ClientEncryption

108. One key
Specify with JSONSchema
…
ssn: {
encrypt: {
keyId: [id],
algorithm: (…),
bsonType: (…)
}
}

109. One key
db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Query key vault by _id
Compare to JSON schema

110. Labeled keys
ce = ClientEncryption(opts)
id = ce.createDataKey(keyopts, keyAltNames=["Doris"])
Create with ClientEncryption

111. Labeled keys
Specify JSON Pointer with JSONSchema
…
ssn: {
encrypt: {
keyId: "/name",
algorithm: (…),
bsonType: (…)
}
}

112. db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Query key vault by label "Doris"
Compare to JSON schema
Labeled keys

113. db.coll.insert({
name: "Kevin",
ssn: "457-55-5462"
})
Query key vault by label "Kevin"
Compare to JSON schema
Labeled keys

115. DorisDB
MongoDB Cloud Hosting - By Doris ™

118. App Server
User
insert
MongoDB
(Key Vault)
fetch key
decrypt key DorisDB
(Storage)
encrypted insert

119. {
email: (encrypted),
pwd: (encrypted)
}
Email deterministic, pwd random, uses collection key

120. {
user_id: "…",
title: (encrypted),
body: (encrypted)
}
Encrypted randomly with per-user key

121. User
MongoDB
(Key Vault)
delete user key
DorisDB
(Storage)
delete all posts
App Server
"Delete my data"
… but was it really deleted?

124. Users :) Latency :(

125. EAST
DICTATORLAND
Global Shards

126. EAST
DICTATORLAND

127. EAST
DICTATORLAND
{ _id: 1, body: BinData(6, "A81…") }
{ _id: 2, body: BinData(6, "017…") }
{ _id: 3, body: BinData(6, "5E1…") }
…