Permit bulk loading of documents from SQL server
We have about 2.5 million documents, currently stored in a table in a SQL database on Azure.
We would like to insert these documents into an Azure search service, hosted in the same datacentre.
Currently the only way to do this is by reading each row and inserting using the REST API.
Something like the request feature "Azure Search and SQL Server integration" would help in this case.
Although that request is asking for the data to be stored in SQL, in our case we want the data to be moved from SQL.
An alternative is for the REST API to support a "bulkInsert" method that accepts a database connection string and either a stored procedure or some raw SQL. Once the request is received by the Azure Search Service, it could open a connection to the database, execute the query and create documents from the resulting rows, which would need to be strings representing the required JSON document.
We do have a batch API, that allows you to upload multiple documents to Azure Search. There is some more information on batching here: http://msdn.microsoft.com/en-us/library/azure/dn798930.aspx
Also, we are looking at ways of providing tighter integration with Azure SQL databases.