How to store Internal log4j2 Logs within the Application that is uploaded and run-ed by Azure HDInsight
Currently i'm using a HDInsight for my Data Processing Pipeline with multiple nodes required. The application that I upload into Azure HDInsight has its own log4j2 logging within the application itself.
When I upload this application and run it on the HDInsight Cluster, it works. However, I need to store these log4j2 log results into a possible storage container, which in this case, it is most convenient to use Azure Blob Storage. How do I store it in an Azure Blob Storage with my Application?
Here's an example of the problem to upload internal log4j2 inside the application, where the application is uploaded into HDInsight and run-ed. It is similar to this problem, displayed on github.
Currently, I am using scala spark intellij for development.