Support staging/production scenario as in Windows Azure
This could be done with the future cloning technique or perhaps a more sophisticated one?
The idea of a Staging environment is a great idea, but the implementation is severely flawed. If you rely on Azure for the majority of your stack, the problem is that you have a SQL Azure database that is eventually going to need to be updated and changed. You can embed SQL commands directly into the deployment process to make these changes to the database, but then you're still left with a situation where you have a single database that serves both your production and staging systems. Unfortunately, this means your production system is very likely going to break because you just introduced database schema changes that the code wasn't ready for.
One way around this would be to use a Staging database which is copied from production, but then you run into other problems. First, things like billing code would still execute, so you risk billing customers twice while this second database is being used in your staging environment. Second, you still have to figure out how to modify the connection string so that in staging, the application hits the staging database and in production, the application hits the production database. Since the configuration is not intended to be modified once you've pushed it to staging, then this isn't possible.
The only other solution I can see would be to back up the database to a blob and make it accessible across Azure accounts, then restore it into a different account. Publish your code, and see if everything works in an automated fashion. If not, then try the process again.
There are a lot of different ways that you can sort of hack around these problems, but it would really be best if Azure provided a true mechanism for testing a full-blown staging environment. It's pretty rare from my experience where you can make DB schema changes and they don't significantly affect the application. And in that case, how would you ever be able to roll back the code because the database changes need to be made in lock-step with the code changes?
Today, there just isn't a good solution for this. Allowing the users to apply some specific data transforms to customer data as the Azure DB is being copied from production would be really helpful here, so as to obfuscate a lot of the data and make sure that developers are working with anonymous sets of data, rather than live data from customers. Otherwise, who knows what they might see? This part is a serious problem for developers creating solutions aimed at financial or health care related customers.
Customers and Partners need a "lab-in-the-cloud" that showcases upcoming features of SQLAzure. This cluster could be made available a few months before code is released to production. It helps customers to validate features and develop functionality on early bits and stay prepared for cloud upgrades.
Can you provide some details of your scenario and what you would like to see in this area? Thanks.
Dan Siegel commented
The process should be fairly straight forward like deployment slots are for Azure Websites (App Services).
1) I should be able to manage one or more versions of the database for scenarios like Production, Staging, Development
2) I should be able to choose between user requested and an automatic process. This could be daily, weekly, etc., and perform a clone anytime I need.
3) I should be able to choose schema only, full db, or select tables to sync
4) I should be able to run any Sql files, PS scripts, or compiled apps I may need. For example I may want to change user passwords to a known password for testing, or generate a random password and email the developer.
5) I should be able to kick off this process using any stored backup and not just the live database.
Shannon Lowder commented
Yeah, the built-in db copy could be used to clone the db, just restore it to your db_staging like mxrss suggests. You could even use powershell to automate the restores with some work.
Chris Schaller commented
I'm not sure how much of this is "Azure's" responsibility.
It is not the platform's role to identify all of the services that your solution might be using, nor are you confined to only consuming services in Azure.
Firstly, there are common staging vs production paradigms that we as developers have been implementing long before Azure, I have to ask:
Why are you treating your deployment to Azure differently to your normal in-house or client test vs live environments.
Secondly, Visual Studio has provided awesome support for deployment to staging vs production.
Our App uses two separate databases, multiple services outside of azure and 3 roles within.
In our Visual Studio solution, we have the two Azure deployment profiles, one for staging, one for production, we then have two solution configurations... you may have heard of them, Debug and Release.
We have two rules in-house. Staging deployments MUST be debug builds, Production deployments MUST be release builds. The Deployment profiles handle this for us so it's not something that you need to be overly aware of.
We toyed with the idea of changing the solution configuration names to Staging and Production but the name is arbitrary.
We use mainly web.config transformations (visit http://go.microsoft.com/fwlink/?LinkId=125889) to handle the fact that settings need to be changed between the two environments and some compiler directives.
To Handle our Azure DB changes we could use DAT packs, however we have an in-house solution were the service roles detect changes and verify DB schema on version upgrades, as this is the solution we have been using for many years now, it all still functions the same in Azure.
Because we integrate with a number of on-premise services, there is no way that Azure could automatically migrate or indeed affect the state of these services. So either way we would have to manage this ourselves, as you would expect, however the web.config transformations really are instrumental in managing our service proxy configuration.
Cant this be done by either
1) Creating a new DB Server
2) Creating a copy of the databases like stagging_db