Create Loading Job Using File Parts

GSQL Loading
#1

Apologies if this has previously been covered, I wasnt able to find any previous solutions.

I have extracted files from a teradata warehouse using mulitple (4) instances. This has caused the files to be written in parts.

file.txt-1
file.txt-2
file.txt-3
file.txt-4

Is Tigergraph able to deal with the files in parts, if so is there are a way to use a wildcard or another way when defining the FILENAME

DEFINE FILENAME lf= “/file.txt*”

I dont think I can just define the directory because there are multiple files extracted for different nodes

Have I missed something?

Thanks for any assistance

#2

Hi Nate,

Currently, we don’t have a feature to use a wildcard to specify multiple local files to load from in a single file source. However, what you can do is to use the USING keyword to specify the specific file to load from.

For instance, suppose your loading job is defined like this:

CREATE LOADING JOB load_file_in_part FOR GRAPH ... {
    DEFINE FILENAME lf;
    ...
}

You can then do the following GSQL commands so that you would load the file one at a time. Notice that you can run the same loading job multiple times with a different file by utilizing the USING keyword. Some of the examples of the USING keyword can be seen here in this loading job doc.

RUN LOADING JOB load_file_in_part USING lf=".../file.txt-1"

RUN LOADING JOB load_file_in_part USING lf=".../file.txt-2"

RUN LOADING JOB load_file_in_part USING lf=".../file.txt-3"

RUN LOADING JOB load_file_in_part USING lf=".../file.txt-4"

I hope this helps! Please let us know if there’s any further questions from your end.