Hi,
It seems that there is a timeout of only 3 seconds when starting a Kafka load job for the job
to fetch topic metadata. I think there is a lot of topic metadata, 3 seconds might not be enough time.
How do I override that timeout? I didn’t see any option for it for RUN LOADING JOB.
I get no log output file either, just this:
GSQL > @load_jobs/v_id_load_job.gsql
Using graph ‘cool_data’
The job kafka_v_id_load is dropped!
The job kafka_v_id_load is created.
Try to list topic metadata from Kafka broker ‘kool-kafka.com:443’, timeout: 3 sec …
Timeout expired while fetching topic metadata
GSQL >
I know there is nothing wrong with the broker because that is getting used everywhere at our company.
The broker has SSL, so I have those props as well in the kafka config:
{
“broker”: “kool-kafka.com:443”,
“kafka_config”: {
“group.id”: “some-group”,
“ssl.protocol”: “TLSv1.2”,
“ssl.truststore.location”: “some_truststore.jks”,
“ssl.truststore.password”: “some_password”,
“ssl.keystore.location”: “some_keystore.jks”,
“ssl.keystore.password”: “some_password”,
“ssl.key.password”: “some_password”
}
}
the topic config I’m using is:
{
“topic”: “kool.kafka.tg-vertex-id-type”,
“default_start_offset”: -2
}
and the load job is:
USE GRAPH cool_data
DROP DATA_SOURCE k1
CREATE DATA_SOURCE KAFKA k1 FOR GRAPH cool_data
SET k1 = “/some/path/kafka_config.json”
DROP JOB kafka_v_id_load
define the loading job
CREATE LOADING JOB kafka_v_id_load FOR GRAPH cool_data {
DEFINE FILENAME f1 = “$k1:/some/path/v_id_topic_config.json”;
LOAD f1
TO VERTEX Id VALUES ($0, $1, $2)
USING SEPARATOR="|";
}
load the data
RUN LOADING JOB kafka_v_id_load USING EOF=“true”
kool-kafka.com:443 is a load balancer F5, and the actual keystore password has a + and / in it, do those maybe need to be escaped?
Anyways, I think there might be a lot of metadata, and it can’t be obtained in 3 seconds before timing out,
is there any way to increase the timeout?