Can I map multiple mappings to one model?

0
I'm dealing with a task of matching user sentence to pre-defined sentences. I used ml model to vectorize user sentence and predefined sentences using Mendix ML Kit. Since I only want to vectorize predefined sentences once - I used one mapping with batch size of predefined sentences and second mapping with batch size = 1. That I am handling when user inputs the sentence.   Therefore I have 2 model mappings that create 2 exact same model onnx files - how can I only point to one model file?
asked
2 answers
0

Hi Stefan,

 

Great question! Am I understanding correctly that the only difference between the two model objects is the batch size? If that's the case, you may have some success using Dynamic Tensor Shapes.

 

Otherwise, if you're comfortable sharing a basic reproduction of the issue in a sample application, we'd be happy to take a look at it.

answered
0

Hello,

 

first thing - sorry for taking my time with the reproduction!

I created a very simple project that should show the issue - https://github.com/dudaskostefan/Reproduction138376

 

The app should show that we need 2 identical model files to handle 2 interactions with the ML Mapping that only differs in batch_size.

Knowledgebase interactions embeds to vector space 300 knowledgebase items (sequences) and user input handles batch_size = 1 because we vectorize user question. If I tried to rely on dynamic tensor shape instead of static tensor shape (in knowledge base preprocessor I changed number of simulated sequences from 300 to 200), I got the error :

image.png

 

It might have been the the current onnx model cannot fallback to dynamic tensor shape but relies on static tensor shape but I tried number of models and still got the same issue, I believe the mapping definition is correct:

image.png

 

Reason, why I cannot update it to batch_size = 301 is because I only need to vectorize knowledgebase once, so I don't want to vectorize 300 KB items all the time. When I tried to only use the model with batch_size = 300 also for user question and PAD the other 299 sequences with nulls, performance was the same as if I was handling 300 items, so this workaround would not work. Also, I was not able to just copy-paste another MLMapping from the first one and just change batch_size because the mapping creates and wires to a new entity (ML_{Input/Output}_Entity_Multibatch_sequence_tokenizer).

 

Thank you, if you need additional details, I'm happy to provide it!

Stefan

answered