topic Re: [Visual Investigator 10.6] How to know when my attachment's content is extracted/indexed? in Fraud, AML and Security Intelligence

[Visual Investigator 10.6] How to know when my attachment's content is extracted/indexed?

JeanDo — Fri, 29 May 2020 09:52:31 GMT

Hello,

I wrote a script to automatically load a bunch of files as attachments, using two API calls to first upload files to the file service and then attach this file from file service to a VI document. Unfortunately, my script crash after 5 iterations, because the File service goes down, each time for the same reason: "java.lang.OutOfMemoryError: Java heap space"

After a lot of exploration, I figured out, by requesting the fdhdata.dh_file table at the same time than uploading the files, that the script goes down because I generate a lot of requests without waiting for the content to be indexed. So my question is: which SAS service can I request to kown if (or when) the files' content is indexed?

Thank you for your help,

Re: [Visual Investigator 10.6] How to know when my attachment's content is extracted/indexed?

mtlppe — Fri, 29 May 2020 13:53:39 GMT

Hi Jean,

The aforementioned table does contain the "content_extracted_flg" which would indicate the extraction has been performed. In the VI 10.7 release, a request parameter "includeExtractedContent=true|false" has been added to the files endpoint that returns the extracted content, which would suggest the extraction is completed.

I would also suggest attempting to increase XMS/XMX settings for the Files service so that there is more memory available to the service.

Cheers,
Paulius.

Re: [Visual Investigator 10.6] How to know when my attachment's content is extracted/indexed?

JeanDo — Fri, 29 May 2020 16:48:17 GMT

Hi Paulius,

Thank you for the response! In fact, in my fdhdata schema, the dh_file table does contain "content_extracted_flg" and "extracted_content_txt" variable (I'm using a 10.6 version of VI); I see it when requesting the PG database directly.

I rose the java_option_xmx to -Xmx2048m and it's actually working fine.

Cheers,