Sending a request to the forward method of a model that expects more than one vector #1164
Description
Hi,
I'm trying to make a request to the bert model whose trace I have taken. I've tried this in different ways, I've also tried with K value args but I always get an error.
As you know, the Bert model expects 3 input vectors. Ids, attention_mask and token_type_ids. I'm making a test request as follows.
IValueVector inputs = new IValueVector();
inputs.push_back(new IValue(ones(1,128)));
inputs.push_back(new IValue(ones(1,128)));
inputs.push_back(new IValue(ones(1,128)));
Tensor output = module.forward(inputs).toTensor();
And I am getting this error. I think I got the trace of the model correctly. Because I can use the model on cpp.
Exception in thread "main" java.lang.RuntimeException: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/__torch__/transformers/models/bert/modeling_bert.py", line 21, in forward
_2 = torch.to(extended_attention_mask, 6)
attention_mask0 = torch.mul(torch.rsub(_2, 1.), CONSTANTS.c0)
_3 = (embeddings).forward(input_ids, token_type_ids, )
~~~~~~~~~~~~~~~~~~~ <--- HERE
_4 = (encoder).forward(_3, attention_mask0, )
return (_4, (pooler).forward(_4, ))
File "code/__torch__/transformers/models/bert/modeling_bert.py", line 49, in forward
_6 = torch.slice(position_ids, 0, 0, 9223372036854775807)
input = torch.slice(_6, 1, 0, _5)
_7 = (word_embeddings).forward(input_ids, )
~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
_8 = (token_type_embeddings).forward(token_type_ids, )
embeddings = torch.add(_7, _8)
File "code/__torch__/torch/nn/modules/sparse.py", line 10, in forward
input_ids: Tensor) -> Tensor:
weight = self.weight
inputs_embeds = torch.embedding(weight, input_ids, 0)
~~~~~~~~~~~~~~~ <--- HERE
return inputs_embeds
Traceback of TorchScript, original code (most recent call last):
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/functional.py(2183): embedding
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/modules/sparse.py(158): forward
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py(214): forward
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py(989): forward
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/jit/_trace.py(958): trace_module
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/torch/jit/_trace.py(741): trace
<ipython-input-7-6f0adf2a89a3>(1): <module>
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3417): run_code
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3337): run_ast_nodes
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3145): run_cell_async
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/IPython/core/async_helpers.py(68): _pseudo_sync_runner
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/IPython/core/interactiveshell.py(2922): _run_cell
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/IPython/core/interactiveshell.py(2876): run_cell
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/ipykernel/zmqshell.py(536): run_cell
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/ipykernel/ipkernel.py(306): do_execute
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/gen.py(209): wrapper
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/ipykernel/kernelbase.py(543): execute_request
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/gen.py(209): wrapper
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/ipykernel/kernelbase.py(268): dispatch_shell
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/gen.py(209): wrapper
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/ipykernel/kernelbase.py(365): process_one
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/gen.py(748): run
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/gen.py(787): inner
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/ioloop.py(743): _run_callback
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/ioloop.py(690): <lambda>
/home/mehmet/anaconda3/envs/training/lib/python3.8/asyncio/events.py(81): _run
/home/mehmet/anaconda3/envs/training/lib/python3.8/asyncio/base_events.py(1859): _run_once
/home/mehmet/anaconda3/envs/training/lib/python3.8/asyncio/base_events.py(570): run_forever
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/tornado/platform/asyncio.py(149): start
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/ipykernel/kernelapp.py(612): start
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/traitlets/config/application.py(845): launch_instance
/home/mehmet/anaconda3/envs/training/lib/python3.8/site-packages/ipykernel_launcher.py(16): <module>
/home/mehmet/anaconda3/envs/training/lib/python3.8/runpy.py(87): _run_code
/home/mehmet/anaconda3/envs/training/lib/python3.8/runpy.py(194): _run_module_as_main
RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got CPUFloatType instead (while checking arguments for embedding)
at org.bytedeco.pytorch.JitModule.forward(Native Method)
at src.main.java.com.example.JavacppSampleApplication.main(JavacppSampleApplication.java:71)
It works fine when I request a model with a forward method that expects a single variable. But it doesn't work on bert. Can you share an example usage for models with multi-variable forward method? Or what is the reason for the error I get?
Thanks in advance.