In release 1.2.0 of SAS Viya Monitoring for Kubernetes in June 2022, we got a great new way to export SAS Viya log messages to a file, the Reporting > Generate CSV function in OpenSearch Dashboards:
Reporting > Generate CSV menu button
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
At the risk of sounding like click-bait, with four simple tips it's easy to export the SAS Viya log messages you're interested in. You can include just the fields of contextual information you want, and your exported messages can span multiple services over multiple days. These tips will help you work around a bit of quirky behavior, and get the most out of this excellent new(-ish) feature.
As a brief aside while on the topic of exporting logs from SAS Viya, I last wrote about it in this post in April 2021. A couple of things have evolved about the methods I surveyed back then:
Now, on to the main subject of this post: the Reporting > Generate CSV function in OpenSearch Dashboards' Discover page. This will export log messages as seen in the table below the chart on the discover page.
You need to know four things to get best results:
Tip: you should not overwrite the 'Log Messages' saved search. It is provided with your SAS Viya Monitoring for Kubernetes logging stack during its deployment. This saved search is used in a number of dashboards, and changing it (e.g. by adding a filter or search) will impact dashboards based on it. Because dashboards display charts, the effect of changes to the 'Log Messages' saved search may not be immediately obvious.
Leave 'Log Messages' alone and save your search and filter criteria as a new saved search before exporting log data to a CSV with the Reporting > Generate CSV function.
Let's see why each of these things matters.
Here's a simple saved search I created by opening the Log Messages saved search, adding a filter for 'logsource = cas', and saving it as a new saved search named 'Log Messages - CAS':
On the Discover page of OpenSearch Dashboards, the leftmost column of the table of log documents is always Time
When I choose Reporting > Generate CSV, the resulting CSV data does not include the Time column, and have not found any option that you can change to include it:
The Time column is not included in a generated CSV based on the same saved search
The solution is to add the @timestamp column to the table in your saved search...
Add @timestamp column to the table
...then remember to save it and lastly export the CSV again:
Now the CSV has a timestamp column… wherever it was in the table. Here it is the right-most column, which looks a bit strange.
It's nice to have the @timestamp column first in the CSV. To achieve this, click the little double left chevrons ('«') in the column heading for the @timstamp column - they appear when you hover your mouse pointer over them:
Left double chevrons (‘«’) in the @timestamp column heading. Click to move column to the left.
Keep doing this until the @timestamp column is as far to the left as it will go. This is the second-left most column. The 'Time' column which is not exported in the CSV, is always the left-most column:
The timestamp column moved to the left as far as it will go
Remember again to save your changes to your saved search, and choose Reporting > Generate CSV again. The @timestamp column will now be at the left of the CSV:
CSV with @timestamp column on the left
In the tips above, I reminded you to save your changes (but don't overwrite the out-of-the-box Log Messages saved search) before exporting a CSV. There are two reasons for this.
First, if you just open the Discover page in OpenSearch Dashboards, without opening or saving your view as a saved search, you will see that the Reporting > Generate CSV function is disabled:
No saved search is open in the Discover page. The Reporting > Generate CSV option is disabled.
Second, after you change any of the filters, search string, columns, sort order or anything else about the Discover page, if you do not then save your changes to update your saved search, and choose Reporting > Generate a CSV, the CSV will reflect the filters, search string, columns, sort order etc. as they were when you last saved them to the saved search object, and will NOT reflect your changes.
If you try to generate a CSV without saving recent changes, there is no prompt asking if you want to save your changes. This is not great UI design, but it's not too difficult to get into the habit of first saving, and then generating a CSV.
Some initialization happens behind the scenes in OpenSearch the first time you generate a CSV this way.
SAS Viya Monitoring for Kubernetes configures a 'non administrator' user in OpenSearch called logadm, which you can use when you don't need full administrator privileges, to reduce the risk of you making an accidental configuration change.
It seems that in SAS Viya Monitoring for Kubernetes prior to version 1.2.7 the admin user has all the permissions it requires to complete the initialization steps for generating a CSV, while the logadm user does not.
So, if you first try to generate a CSV in a new deployment of the SAS Viya Monitoring for Kubernetes logging stack as logadm, it fails like this:
Generating a CSV as the logadm user first fails. Generate one as admin first.
The error message says "Error generating report", "Insufficient permissions. Reach out to your OpenSearch Dashboards Administrator".
The solution is to sign in as the admin user, and generate a CSV from any saved search. This should succeed, and without you seeing any evidence of it the initialization steps are performed successfully. Then sign out and sign in again as the logadm user. Now generating CSVs as logadm should succeed too.
I hope these tips have been helpful. Please let me know in the comments below if you have any more tips for generating CSVs, or for any other feature in SAS Viya Monitoring for Kubernetes' logging stack (OpenSearch and OpenSearch Dashboards).
See you next time!
Find more articles from SAS Global Enablement and Learning here.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.