Openshift Advanced Stacktrace Log Parse Issue
We foorwarded logs of application and cluster by collecting collector (which is logging application on Openshift) and forwarded them through logging api (which is also provided by Openshift) to external Logstash and ELK. We noticed that Java log traces cannot be seen on Kibana. We opened a case to Red Hat about this issue and they responded as below.
They suggest that adding \n or \r character on each line on trace. Could you please guide us is it possible to make this on app level?
If you need a meeting we can also arrange.
It's important to highlight:
- Multiline works only for English, not other languages
- The lines should match the regular expressions from [0] for the first lines and for the next lines from a multiline Java stack trace the ones on java_after_exception regular expressions.
- In the logs provided, it's visible different formats of logs and dates where some match a Java stack trace and other lines are mostly errors they are not respecting what's expected from a native Java stack trace application. The solution should be:
Going through the logs to review what logs are not being detected as multiline and normalize them
Please let me know if you have additional questions or concerns.
Refs
[0] https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions/blob/fcaeb43ed5248df533d270ceae60c94c6f2e6174/lib/fluent/plugin/exception_detector.rb#L54[1] https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions/blob/fcaeb43ed5248df533d270ceae60c94c6f2e6174/lib/fluent/plugin/exception_detector.rb#L56