Custom Logs (import and delete) and add custom timestamps
One amazing idea is create custom fields on custom log sample process. Another good idea is add more timestamp samples (like ISO 8601 format, YYYYMMDDThhmmss.fffK where YYYY: Year, MM: Month, DD: Day in month, T: Delimiter, hh: Hour, mm: Minutes, ss: Seconds, fff: Milliseconds, K: Time zone offset) or add the possobility to create a custom timestamp.
It will be possible delete some imported custom logs to make some tests?
We’re planning on allowing you to import/export Custom Logs & Fields via the UI & ARM Templates. We’re currently implementing the ARM support today for most of Settings in OMS.
Thanks for sharing some of the timestamps you need. Feel free to e-mail them to me here: evanhi(at)microsoft.com
We’re actively planning way for you to specify timestamps yourselves.
Steeve Roy commented
We are also looking for custom timestamp to fit our logs timestamp delimiter. I send you an example of log by email.
Is this still being developed? Also, I see two ideas here......specifically I am interested in being able to create custom fields based upon a delimiter. For instance the Network Policy Server logs use a comma as a delimiter and it would be great to be able to use those for custom fields upon ingestion.
The problem we are experiencing is that OMS is not delimiting records correctly imported as custom logs when the date format uses the ISO 8601 date time convention with a comma as opposed to “your” predefined setting for YYYY-MM-DD HH:MM:SS
We use log4net as a log parser which uses the ISO 8601 standard https://en.wikipedia.org/wiki/ISO_8601 “ISO 8601:2004(E), ISO, 2004-12-01, 18.104.22.168 ... the decimal fraction shall be divided from the integer part by the decimal sign specified in ISO 31-0, i.e. the comma [,] or full stop [.]. Of these, the comma is the preferred sign.”
In order for us to change our log format to use decimal points we would have to reconfigure logging for thousands of records, this would include updating our product, deployment scripts and require interruption of service to 100,000’s of customers worldwide as IIS would cause an app recycle. Additionally, we would need to change these logs at one time, as it would be necessary to reconfigure hundreds of log settings within OMS to change the date time separator.
This is causing us major concern as currently we had removed the log settings in an attempt to improve our reporting, however, with the logs breaking on incorrect lines, we are unable to trust the data. Our business relies us on proactively monitoring the health of our clients and swiftly reacting when certain alerts get generated in our products logs. Without this functionality, OMS is not a viable option for us and we will have to investigate alternatives such as SPLUNK.
I appreciate your help, and would be happy to demonstrate.