Since the European Court of Justice nullified the Privacy Shield agreement in July 2020, a huge uncertainty came up for EU companies processing Personally Identifiable Information (PII) in the US, for example by using Amazon Web Services (AWS). Privacy Shield ensured that Personally Identifiable Information of EU citizens transferred to and processed in the US were protected by EU data protection laws. US companies registered on a list for compliant data processing and should be watched by the government. EU citizens could file complaints against US companies for data protection violation and had claims against those companies.
The European Court of Justice terminated the agreement after Max Schrems filed a complaint that Facebook transferred data to the US and made data accessible for the NSA and FBI. Laws like the Foreign Intelligence Surveillance Act (FISA) allow access to data of non-us-citizens without court order and this is in violation to EU data protection laws.
Now that Privacy Shield is gone it is difficult to know if you are still in compliance with EU laws when transferring or processing data in the US. AWS claims that customers are still compliant, even when using US regions. AWS processes all data in conformity to GDPR and “standard contractual clauses” (SCCs). The SCCs are not yet revoked by the European Court of Justice and can still be used as a data protection framework though they have no legal effect anymore. AWS customers transferring and processing data in AWS regions inside of the EU are not affected.
Things you can do to be more compliant with EU law is to only host in EU AWS regions though this is also not 100% safe by now. Make sure you have SCC agreements with AWS and make sure you implement all GDPR requirements.
Not conforming with EU data protection laws may result in customers or data protection agencies complaining and forcing you to stop processing data within the US. Fines may also be imposed which can be up to 4% of your annual turnover though this is very unlikely since the whole situation is currently very unclear. Warning letters and consequences in regards of competition law may arise, though.
The non-profit trade association “Cloud Infrastructure Services Providers in Europe“ (CISPE) released the “CISPE Data Protection Code of Conduct” to help IaaS providers to offer GDPR compliant services. AWS’s CISPE conform services include EC2, RDS, S3, IAM and EBS. It is your job now to make sure that you use those services in conformity with GDPR requirements as well. The division of work is the same as with security which means AWS handles data protection of the cloud while you handle data protection in the cloud.
Stored data in S3 containers can be encrypted by default. AWS managed KMS keys enable S3 encryption easily and AWS promises that the KMS keys never leave the region they are stored in. If you need more control over your master keys you can also create your own KMS CMKs and use your own managed keys for S3 encryption. You can also encrypt with your own data encryption keys though building a custom key management solution is quite an overhead and as the saying goes: “Don’t Roll Your Own Cryptography”.
AWS services like Macie, which is available in EU regions since May 2020, can help you protect Personally Identifiable Information with machine learning concepts. The service scans S3 buckets for PII patterns and warns you of any data that is accessible or shared outside your AWS Organization.
Furthermore, you can encrypt your business data with the help of AWS KMS CMKs. The key management service allows you to create customer managed master keys (CMK) that are used to encrypt the data encryption keys which are also provided by KMS. You have control over your CMKs and can define access controls with the help of policies and roles (IAM). With the AWS KMS API you can then request data encryption keys for envelope encryption and decryption of your data. KMS is not managing data encryption keys so make sure to store the encrypted data encryption keys together with your encrypted data. To decrypt your business data, you must decrypt your data encryption key via the AWS KMS API and then decrypt your data with the cleartext data encryption key. There is also the AWS Encryption SDK available which you may want to look at.
Example of KMS encryption of email addresses
Here is a little example on how to encrypt the email address of a user model in Ruby on Rails 5.2. This example is functional but by no means complete. It demonstrates the easy use of the AWS KMS API to implement encryption on the application side.
Execute the following commands in your terminal to create the example project that will encrypt a users email address transparently for a user model:
Step 1 – Create an IAM User
Create a new IAM user that will be used for programmatic access to the AWS KMS API. Note the key_id and access_key! The access_key can not be retrieved after the user is created!
Step 2 – Create your KMS CMK
You can create a new KMS CMK for this demo in the AWS Console under Key Management Service. The wizard will guide you through this process. In the final steps select your personal User as the keys administrator and the new created IAM user account as the key user. Note the KMS key ARN.
Step 3 – Create your rails project
% rails new encryption_example % cd encryption_example
Step 4 – Edit your Gemfile and add the following GEM
gem 'aws-sdk-kms'
Then install the GEMs with bundler.
% bundle install
Step 5 – Create the User model with an email address attribute and migrate the database
% rails g model user email:text % bundle exec rake db:migrate
Step 6 – Configure your AWS credentials by creating config/app_config.yml
aws: aws_region: <your-aws-region> aws_key_id: <your-aws-key-id> aws_access_key: <your-aws-access-key> aws_kms_key_id: <your-kms-key-arn>
Step 7 – Create the AWS KMS API wrapper at app/models/concerns/aws_kms.rb
module AwsKms
extend self
# Read AWS credentials from app_config.yml
mattr_accessor :config, default: YAML.load(File.read(File.join(Rails.root, 'config/app_config.yml')))['aws']
# Initialize AWS KMS client with AWS credentials.
mattr_accessor :client, default: Aws::KMS::Client.new(
region: config['aws_region'],
access_key_id: config['aws_key_id'],
secret_access_key: config['aws_access_key']
)
# Decrypt data encryption key
def decrypt_data_key(ciphertextblob)
client.decrypt(ciphertext_blob: ciphertextblob)
end
# Get a new data encryption key
def generate_data_key(key_id = config['aws_kms_key_id'], key_spec = 'AES_256')
client.generate_data_key({
key_id: key_id,
key_spec: key_spec
})
end
end
Step 8 – Create the attribute encryption helper that transparently encrypts you Model attributes under app/models/concerns/crypted_attribute.rb
require 'json'
require 'aws-sdk-kms'
module CryptedAttribute
extend ActiveSupport::Concern
included do
extend ClassMethods
end
module ClassMethods
# Dyanmic code that generates attribute accessors for encrypted attributes
def crypted_attribute(attribute)
class_eval(%(
# Transperently decrypt configured model attribute
def #{attribute}
# Read crypted attribute and parse its JSON content.
encrypted_attribute = JSON.parse(read_attribute(:#{attribute}))
# Encrypted AWS KMS data key that was used to encrypt the attribute
data_key = Base64.decode64(encrypted_attribute['data_key']).force_encoding('BINARY')
# Initialization vector of the encrypted attribute
iv = Base64.decode64(encrypted_attribute['iv']).force_encoding('BINARY')
# The encrypted attribute
cipher_text = Base64.decode64(encrypted_attribute['cipher_text']).force_encoding('BINARY')
# Decryption of the stored encrypted data key
decrypted_key = AwsKms.decrypt_data_key(data_key)['plaintext'].force_encoding('BINARY')
# Decryption of the crypted attributed with the decrypted data key
AesCryptoProvider.decrypt(cipher_text, iv, decrypted_key)
end
# Transparently encrypt configured model attribute
def #{attribute}=(value)
# Request a new data encryption key from AWS KMS
data_key = AwsKms.generate_data_key
# Encrypt the assigned value with the plaintext data encryption key
crypted_attribute, iv = AesCryptoProvider.encrypt(value, data_key['plaintext'])
# Store a JSON document in the attribute column containing the key_id or ARN of the KMS CMK, the encrypted attribute, it's IV and the encrypted data encryption key all Base64 encoded.
write_attribute(:#{attribute},
{
key_id: data_key['key_id'],
cipher_text: Base64.encode64(crypted_attribute),
iv: Base64.encode64(iv),
data_key: Base64.encode64(data_key['ciphertext_blob'])
}.to_json
)
end
))
end
end
end
Step 9 – Create the AES encryption helper under app/models/concerns/aes_crypto_provider.rb
module AesCryptoProvider
extend self
# Encrypt a value with a AWS KMS data encryption key. Returns the crypted value and its IV as array.
def encrypt(value, key)
cipher = OpenSSL::Cipher.new('aes-256-cbc')
cipher.encrypt
cipher.key = key
cipher.iv = iv = cipher.random_iv.force_encoding('BINARY')
crypted_attribute = cipher.update(value).force_encoding('BINARY')
crypted_attribute << cipher.final.force_encoding('BINARY')
[crypted_attribute, iv]
end
# Decrypt a cipher text with its IV and the decrypted AWS KMS data encryption key
def decrypt(cipher_text, iv, key)
cipher = OpenSSL::Cipher.new('aes-256-cbc')
cipher.decrypt
cipher.key = key
cipher.iv = iv
decrypted_attribute = cipher.update(cipher_text)
decrypted_attribute << cipher.final
decrypted_attribute.force_encoding('UTF-8')
decrypted_attribute
end
end
Step 10 – Finally modify your user model at app/models/user.rb and include the attribute encryption helper and define the email attribute as to be encrypted
class User < ApplicationRecord # Include our attribute encryption helper include CryptedAttribute # and tell it to encrypt the email attribute crypted_attribute :email end
You should now be able to create an User in the rails console and by assigning the email address the attribute should contain a JSON document with the encrypted email address, initialization vector, key ID and the encrypted data encryption key that will later be used to decrypt the email attribute.
% bundle exec rails c
Running via Spring preloader in process 23206
Loading development environment (Rails 5.2.4.4)
2.6.5 :001 > u = User.new
2.6.5 :002 > u.email = ‘mail@example.org’
2.6.5 :003 > pp u.read_attribute(:email)
{\"key_id\":\"arn:aws:kms:eu-central-1:123456789012:key/5bd75943-aad8-4c7d-8d90-f884bb21090f\",\"cipher_text\":\"QrNWoBUZSPjuLtPs56nBNtAvkYJcs2JHaJjj57YrxtU=\\n\",\"iv\":\"KT7fZ8e01PjfIUncPff0dQ==\\n\",\"data_key\":\"K8H10z/n+0ssL6a8xBQgzn6WI1s5t3Wcd8YnNUCgy/uxZbmAdrRWJdqzzdWJ\\nP2LEf0hcB7+o/ZMU7eGcNSlsUtcNcdVxBhaSsJaFAt7RHCkk8UklksRRSum6\\n5ZqSbZI+lg6WAZB2Wxk3AWqALgUI71fEZquXnrn02J+uyEulqHNbhpqraJ4s\\n4X6Zbdwu31DJV3kHgB7+TwsRPL0saHiqN4PQDQDN1mON+kVzFU4Z8SNjQxpa\\nj6/LtODSn6vm4sQcgNE=\\n\"}"
=> {\"key_id\":\"arn:aws:kms:eu-central-1:123456789012:key/5bd75943-aad8-4c7d-8d90-f884bb21090f\",\"cipher_text\":\"QrNWoBUZSPjuLtPs56nBNtAvkYJcs2JHaJjj57YrxtU=\\n\",\"iv\":\"KT7fZ8e01PjfIUncPff0dQ==\\n\",\"data_key\":\"K8H10z/n+0ssL6a8xBQgzn6WI1s5t3Wcd8YnNUCgy/uxZbmAdrRWJdqzzdWJ\\nP2LEf0hcB7+o/ZMU7eGcNSlsUtcNcdVxBhaSsJaFAt7RHCkk8UklksRRSum6\\n5ZqSbZI+lg6WAZB2Wxk3AWqALgUI71fEZquXnrn02J+uyEulqHNbhpqraJ4s\\n4X6Zbdwu31DJV3kHgB7+TwsRPL0saHiqN4PQDQDN1mON+kVzFU4Z8SNjQxpa\\nj6/LtODSn6vm4sQcgNE=\\n\"}"
2.6.5 :004 > pp u.email
2.6.5 :005 > “mail@example.org”
If you need assistance in dealing with the Privacy Shield topic on AWS just contact Alice&Bob!
Disclaimer: This article serves non-binding information purposes and does not constitute legal advice in the actual sense.