Hi,
I am trying to follow the guidance here: https://docs.tigergraph.com/dev/data-loader-user-guides/s3-loader-user-guide to load in csv data stored in an s3 bucket. I’m using the developer edition locally inside a Docker container.
For now I am just using the csvs from GSQL101, but I cannot successfully create a loading job.
I’ve defined 2 data sources and pointed them to my s3.config JSON file:
GSQL-Dev > SHOW DATA_SOURCE *
Data Sources:
- s3 friendship_src ({"file.reader.settings.fs.s3a.access.key":"AKIA********","file.reader.settings.fs.s3a.secret.key":"0exj********"})
- s3 person_src ({"file.reader.settings.fs.s3a.access.key":"AKIA********","file.reader.settings.fs.s3a.secret.key":"0exj********"})
I’ve also created the s3 files config file, one for each csv (bucket path replaced), files_person.config.json and files_friendship.config.json:
{
"tasks.max": 1,
"file.uris": "s3://path/to/csv/person.csv",
"file.regexp": ".*",
"file.recursive": false,
"[file.scan.interval.ms](http://file.scan.interval.ms/)": 60000,
"file.reader.type": "text",
"file.reader.batch.size": 10000,
"file.reader.text.archive.type": "auto",
"file.reader.text.archive.extensions.tar": "tar",
"file.reader.text.archive.extensions.zip": "zip",
"file.reader.text.archive.extensions.gzip": "tar.gz,tgz"
}
When I try to create the loading job using both files simultaneously:
**GSQL-Dev >** USE GRAPH social
Using graph 'social'
**GSQL-Dev >** BEGIN
**GSQL-Dev >** CREATE LOADING JOB load_social FOR GRAPH social {
**GSQL-Dev >** DEFINE FILENAME f1 = "$person_src:/home/tigergraph/data/files_person.config.json";
**GSQL-Dev >** DEFINE FILENAME f2 = "$friendship_src:/home/tigergraph/data/files_friendship.config.json";
**GSQL-Dev >** LOAD f1 TO VERTEX person VALUES ($"name", $"name", $"age", $"gender", $"state") USING header="true", separator=",";
**GSQL-Dev >** LOAD f2 TO EDGE friendship VALUES ($0, $1, $2) USING header="true", separator=",";
**GSQL-Dev >** }
**GSQL-Dev >** END
I get the error:
Semantic Check Fails: The data source in the CREATE LOADING JOB is not unique, or mixed with regular file loading.
The job load_social could not be created!
And when I try to only load f1 on it’s own, I get a different error:
Semantic Check Fails: Exception occurred trying to read $person_src:/home/tigergraph/data/files_person.config.json, error: java.lang.RuntimeException: The loading job used file HEADER but there is no local path in the provided “$person_src:/home/tigergraph/data/files_person.config.json”
The job load_social could not be created!
Could someone help point me in the right direction here? It seems like it isn’t recognising that it should go to look for the data in S3, but otherwise I’m not sure where I’m going wrong.
Thanks!
