Four Tips for Exporting logs from OpenSearch Dashboards’ Generate CSV function
In release 1.2.0 of SAS Viya Monitoring for Kubernetes in June 2022, we got a great new way to export SAS Viya log messages to a file, the Reporting > Generate CSV function in OpenSearch Dashboards:
Reporting > Generate CSV menu button
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
At the risk of sounding like click-bait, with four simple tips it's easy to export the SAS Viya log messages you're interested in. You can include just the fields of contextual information you want, and your exported messages can span multiple services over multiple days. These tips will help you work around a bit of quirky behavior, and get the most out of this excellent new(-ish) feature.
As a brief aside while on the topic of exporting logs from SAS Viya, I last wrote about it in this post in April 2021. A couple of things have evolved about the methods I surveyed back then:
- Mirantis Lens is still great, but now requires you to create an account and sign in before you can use it. A near equivalent project called OpenLens does not require you to have an account or sign in. You either have to download and trust their unsigned binary installation files, or you have to clone the source code and build/make it yourself. But we like it because in our workshops, not having the attendees need to create an account and sign in removes a little bit of friction.
- The experimental getlogs.sh script in the SAS Viya Monitoring for Kubernetes project has not received much maintenance in the past year or so, and as a result some key features, like searching and filtering, no longer seem to work. When I last tried using the getlogs.sh script with search or filtering parameters in release 1.2.6, it returned no results when I think it should have done. It always was a little quirky, and was always clearly categorized as EXPERIMENTAL, so I doubt this effects many users. Given the range of good alternatives available and competing development priorities for the team who maintain it, I don't expect to see the getlogs.sh utility become production ready any time soon. You should plan to use something else for the short to medium term.
Four Tips for using the Generate CSV function
Now, on to the main subject of this post: the Reporting > Generate CSV function in OpenSearch Dashboards' Discover page. This will export log messages as seen in the table below the chart on the discover page.
You need to know four things to get best results:
- The Time column in the table is not exported. I don't know why not, but its omission from CSV exports appears to be a deliberate choice by the developers. You should add the @timestamp column to the table if you want a timestamp in your CSV export - and why wouldn't you?
- You likely prefer to move the @timestamp column to the left of the CSV results. But the Time column (which is not exported) is fixed as the first column on the left side of the table. So, you will likely prefer to move the @timestamp column left as far as it will go, which will be the second column on the left side of the table, one to the right of the Time column.
- OpenSearch generates the CSV results based on the filter and search criteria, and the table columns in a saved search, not based on what you currently see on screen. This can be unintuitive. Changes you make to the Discover page will have no effect on the content of the exported CSV unless you first save them as a saved search.
Tip: you should not overwrite the 'Log Messages' saved search. It is provided with your SAS Viya Monitoring for Kubernetes logging stack during its deployment. This saved search is used in a number of dashboards, and changing it (e.g. by adding a filter or search) will impact dashboards based on it. Because dashboards display charts, the effect of changes to the 'Log Messages' saved search may not be immediately obvious.
Leave 'Log Messages' alone and save your search and filter criteria as a new saved search before exporting log data to a CSV with the Reporting > Generate CSV function.
- If you are unable to generate a CSV as the logadm user, generate one as the admin user, just once. Afterwards, the logadm user should be able to generate CSVs.
Let's see why each of these things matters.
Tip 1: Time column not exported; add @timestamp column
Here's a simple saved search I created by opening the Log Messages saved search, adding a filter for 'logsource = cas', and saving it as a new saved search named 'Log Messages - CAS':
On the Discover page of OpenSearch Dashboards, the leftmost column of the table of log documents is always Time
When I choose Reporting > Generate CSV, the resulting CSV data does not include the Time column, and have not found any option that you can change to include it:
The Time column is not included in a generated CSV based on the same saved search
The solution is to add the @timestamp column to the table in your saved search...
Add @timestamp column to the table
...then remember to save it and lastly export the CSV again:
Now the CSV has a timestamp column… wherever it was in the table. Here it is the right-most column, which looks a bit strange.
Tip 2: Move @timestamp column as far left as it can go, to the second column from the left
It's nice to have the @timestamp column first in the CSV. To achieve this, click the little double left chevrons ('«') in the column heading for the @timstamp column - they appear when you hover your mouse pointer over them:
Left double chevrons (‘«’) in the @timestamp column heading. Click to move column to the left.
Keep doing this until the @timestamp column is as far to the left as it will go. This is the second-left most column. The 'Time' column which is not exported in the CSV, is always the left-most column:
The timestamp column moved to the left as far as it will go
Remember again to save your changes to your saved search, and choose Reporting > Generate CSV again. The @timestamp column will now be at the left of the CSV:
CSV with @timestamp column on the left
Tip 3: Save the filter and query state of the Discover page as a Saved Search before Generating a CSV
In the tips above, I reminded you to save your changes (but don't overwrite the out-of-the-box Log Messages saved search) before exporting a CSV. There are two reasons for this.
First, if you just open the Discover page in OpenSearch Dashboards, without opening or saving your view as a saved search, you will see that the Reporting > Generate CSV function is disabled:
No saved search is open in the Discover page. The Reporting > Generate CSV option is disabled.
Second, after you change any of the filters, search string, columns, sort order or anything else about the Discover page, if you do not then save your changes to update your saved search, and choose Reporting > Generate a CSV, the CSV will reflect the filters, search string, columns, sort order etc. as they were when you last saved them to the saved search object, and will NOT reflect your changes.
If you try to generate a CSV without saving recent changes, there is no prompt asking if you want to save your changes. This is not great UI design, but it's not too difficult to get into the habit of first saving, and then generating a CSV.
Tip 4: Generate a CSV as the admin user first before trying to do it as the logadm user
Some initialization happens behind the scenes in OpenSearch the first time you generate a CSV this way.
SAS Viya Monitoring for Kubernetes configures a 'non administrator' user in OpenSearch called logadm, which you can use when you don't need full administrator privileges, to reduce the risk of you making an accidental configuration change.
It seems that in SAS Viya Monitoring for Kubernetes prior to version 1.2.7 the admin user has all the permissions it requires to complete the initialization steps for generating a CSV, while the logadm user does not.
So, if you first try to generate a CSV in a new deployment of the SAS Viya Monitoring for Kubernetes logging stack as logadm, it fails like this:
Generating a CSV as the logadm user first fails. Generate one as admin first.
The error message says "Error generating report", "Insufficient permissions. Reach out to your OpenSearch Dashboards Administrator".
The solution is to sign in as the admin user, and generate a CSV from any saved search. This should succeed, and without you seeing any evidence of it the initialization steps are performed successfully. Then sign out and sign in again as the logadm user. Now generating CSVs as logadm should succeed too.
I hope these tips have been helpful. Please let me know in the comments below if you have any more tips for generating CSVs, or for any other feature in SAS Viya Monitoring for Kubernetes' logging stack (OpenSearch and OpenSearch Dashboards).
See you next time!
Find more articles from SAS Global Enablement and Learning here.
