Import CSV to CAS via the web interface does not detect dates

[farshidhss]

Hello!

I have a very simple CSV file:

id,date

1,01-08-2018

2,03-08-2018

...

The problem is when I import the CSV file via the "Add data source" option in Visual Analytics then the date column is always detected as a "char".

I have a few requirements that makes it impossible for me to manipulate the data after import:

1. I need the data to be imported automatically via a scheduled job.

2. I don't want to use Parse to convert the char to a date type because the size of the data is very large and I don't want the report to get slowed down by Parse when a user interacts with it.

I have full control on exporting the csv contents so I can at least change the date format written inside the CSV file. I tried to change the date to many different formats and I repeated the import process but the end result is always a char data type instead of date type. Do you have any idea how to import this CSV file so that the date is correctly detected?

I'm using Visual Analytics 8.3.1 if that matters. 

[Reeza]

Use a data plan to convert the data as it's loaded, or part of the 'scheduled job' to load data can often include a transformation. 

If you're using a scheduled job, sounds like a server though, so why not have VA connect directly to the server instead which should import the data correctly?

---

@farshidhss wrote:

Hello!

I have a very simple CSV file:

id,date

1,01-08-2018

2,03-08-2018

...

The problem is when I import the CSV file via the "Add data source" option in Visual Analytics then the date column is always detected as a "char".

I have a few requirements that makes it impossible for me to manipulate the data after import:

1. I need the data to be imported automatically via a scheduled job.

2. I don't want to use Parse to convert the char to a date type because the size of the data is very large and I don't want the report to get slowed down by Parse when a user interacts with it.

I have full control on exporting the csv contents so I can at least change the date format written inside the CSV file. I tried to change the date to many different formats and I repeated the import process but the end result is always a char data type instead of date type. Do you have any idea how to import this CSV file so that the date is correctly detected?

I'm using Visual Analytics 8.3.1 if that matters. 

---

[farshidhss]

Thanks for your reply. How do we transform data at the same time as the import? I can't seem to find any information on that. I did find that you can create a data plan job to transform data and write into another sas table but don't know how to add the data import to the same job.

Yes we do have a VA server, and we would love to connect it directly to our backend but I don't know how this can be done. We use Java spring in the backend.

[Aiman]

Hi I encounter the same issues as yours, have you solve it? Mind to share on how you tackle the issue. Thank you