layerai-archive / examples Goto Github PK

may i use multi-gpus to train t5 model? the example just support simple gpu training.

Hi~
when i layer.login(), I received this error message:

gaierror Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/aiohttp/connector.py in _create_direct_connection(self, req, traces, timeout, client_error)
998 # across all connections.
--> 999 hosts = await asyncio.shield(host_resolved)
1000 except asyncio.CancelledError:

23 frames
gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

ClientConnectorError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/aiohttp/connector.py in _create_direct_connection(self, req, traces, timeout, client_error)
1009 # in case of proxy it is not ClientProxyConnectionError
1010 # it is problem of resolving proxy ip itself
-> 1011 raise ClientConnectorError(req.connection_key, exc) from exc
1012
1013 last_exc = None # type: Optional[Exception]

ClientConnectorError: Cannot connect to host app.layer.ai:443 ssl:default [Name or service not known]

I want to train t5 from scratch, and use my own vocabulary.

the model i can load like this:
config = T5Config.from_json_file(config_file)
model = T5ForConditionalGeneration(config)

the vocabulary is like this below, it seems the tokenizer cannot load this vocab. how should i load this to a proper tokenizer?
{
"": 0,
"": 1,
"": 2,
"": 3,
"": 4,
"，": 5,
"的": 6,
"？": 7,
"了": 8,
.....
.....
.....
"<s_181>": 33786,
"<s_182>": 33787,
"<s_183>": 33788,
"<s_184>": 33789,
"<s_185>": 33790,
"<s_186>": 33791,
"<s_187>": 33792,
"<s_188>": 33793,
"<s_189>": 33794
}