PolyBase: support for encrypted Azure Blob Storage (Storage Service Encryption)
There is a requirement to never leave unencrypted data anywhere. Inbound data files are encrypted but will have to be decrypted before Polybase can read them. In order to satisfy this requirement, Polybase will have to have ability to read encrypted files.
Starting April 2016, Azure Storage Service Encryption (SSE) for Azure Blob Storage is in Public Preview. With this feature, Azure Storage automatically encrypts your data prior to persisting to blob storage and decrypts prior to retrieval, providing encryption at rest. We have tested this feature and PolyBase supports reading/writing data to encryption-enabled Azure Blob Storage. To learn more about Azure Storage Service Encryption, refer to https://azure.microsoft.com/en-us/documentation/articles/storage-service-encryption/.
Thanks again for sharing your feedback with us. 6891055
I know there are options on stream-encrypt/decrypt the blobs in Azure. But thats bit of hacky and you cant really use it for selective records (specially that you have external table options). Having decryption support / functions within EDW will be great advantage to customers (am having to put in a hacky solution at the moment, which is all over the places...)
Dan Cieslak commented
I agree with Michael and Greg
Michael Tamlyn commented
When evaluating technologies, encryption at rest has become one of the few features my organization checks for immediately. If it's not supported, the technology is immediately dismissed due to the heavy security requirements we work with everyday. We are consistently surprised how encryption-at-rest, with TDE being particularly attractive, is often a major reason we go with one solution over another.
Robert Downey commented
I agree that this is key. Many high-security applications have an absolute encryption-at-rest requirement. Supporting TDE in the warehouse itself is great, but if much of our data lives in blob storage, this is a show stopper.
Greg Galloway commented
Similarly, when Polybase writes to Azure Blob Storage, it would be nice to have the option of writing to an encrypted file or files
Greg Galloway commented
Specifically we are encrypting the flat files and storing them in Azure Blob Storage using the following Azure Key Vault encryption/decryption approach:
If Polybase could integrate with Azure Key Vault for decryption that would be ideal.