Allow user to define the sample data set for Custom Field mapping with FastExtract
Log Analytics provides for modeling custom fields in custom logs using FastExtract.
The mapping tool limits the user to the top 100 log entries when doing the modeling of custom fields. This can be seen in the interface when doing a field extract by clicking the "hide tips" link in the right hand side. The tips scroll away revealing the Condition section which indicates a "take 100" limit.
I've struggled to create fields that contain all possible values for a field or column in my custom logs due to this arbitrary limit.
A much more useful implementation would be to allow the user to define their own dataset to use with the FastExtract algorithms. Then the user can ensure the sample dataset contains all possible values for a given field.
In my example, I am creating custom fields for ProFTPd logs. A given file transfer may be done using "ftp", "sftp" or "ftps". I modeled my custom field, and in the 100 rows allowed, FastExtract only saw "ftp". So all my data is either classified as "ftp" or empty string.
As a follow-up note, in Chrome at least, the "Remove" link for each custom field is off to the right of the visible viewport. It's necessary to side-scroll to find it.