Lower scale points for Dev/Test environments
SQL Data Warehouse scale stepping should be much smaller (10 DWU or even 1 DWU), than 100 DWU!
Please fix that ASAP!
Thank you for all the feedback folks. We are currently evaluating options for lower click stops for SQL Data Warehouse. Please continue to vote and comment on your scenario below. Thank you for your patience.
Steve Powell commented
The use of a shared dev environment as no local dev is available at a reasonable price is crippling. If you use SSDT projects in visual studio and CI/CD pipelines then the shared dev environment is impossible to sync with via a schema compare when pulling code back into the repo. the use of CTAS means schema changes occur constantly and the environment is rarely quiet enough to get a sync completed. The bugs in SSDT DB project and CI/CD deployment support for Synapse databases make this process incredibly painful (IDENTITY columns, issues with statistics etc.)
The existing dev model is geared towards empty isolated databases. Data Warehouse development inevitably is as much about data as it is about code. It's the conjunction of the two. Providing an environment that only supports 1 single session as a dev configuration with limited performance would go a long way to helping.
Simon D'Morias commented
A local emulator would be very nice for basic coding.
This is a barrier for us to be able to use this. Why should we pay for 100 DWU when we might only need that much processing for a fraction of the day? Pay on actual use would be better.
Rob Barat commented
Also, for Gen2 Data Warehouse, the minimum is now 500dwu. There is no option to restore to a Gen1 Data warehouse either so once the migration is done, we're stuck with 500dwu minimum.
For dev scenario, preferred solution would be to develop against a visual studio database project in local host. An option to use less redundant storage in a basic configuration would be desirable. If I’m testing, I might be willing to live to forgo geo-redundant storage on ssd. Local only would be enough in many contexts.
Henko Kors commented
This would be great for a dev scenario.
Stanisław Wawszczak commented
That is absolutelly Right!! I have done some initial play with SQL Data Warehouse on small data set (about 20GB) and never I have reached the performance limit. After over two weeks of work I have paid over 200euro.
Any development process on Data Warehouse is completely over budget...
Luis Delgado commented
10DWU should be available for Dev environments that don't require much compute. 100 DWUs as the minimum price is overly expensive for Dev /test environments
i agree, for dev there needs to be the option to scale down to around 10 DWU, otherwise it is too expensive
I echo this. For proof of concept work, dev and general training it would be useful to have a lower entry-level price point, which can then be quickly scaled up to normal levels as required. As suggested anything between 10-50 DWU's would be great, I can actually see this increasing DW usage as it means we can try out more things with it, without being inhibited by the high initial cost.
Jeffrey van Binsbergen commented
10 DWU would be great for testing/development purposes. I've got some database ideas and stored procedures I want to test but I'm not 100% sure if it is all supported. I don't really want to turn on/off an expensive system if I only want to test a design (or is there another way?).
Also it would nice to have my data warehouse always available for a low cost and just scale up when really needed.
I agree that current pricing ~£357 per month is just too much. Either reduce cost or offer 50 DWU usage point.
Matt Usher [MSFT] commented
What about the 100 DWU scale unit is concerning to you? Can you explain a little more detail about what you'd like to see and why? Thanks.