Is there any evaluation metric nodes for neural machine translations in KNIME. I was looking for something out of the box and could not find any.
Presently outside of KNIME I’m running Python Scripts using the COCO evaluation metrics from this site: coco-caption/pycocoevalcap at master · lvapeab/coco-caption · GitHub and generating graphs like these:
Is there a way we can include these COCO scripts in my workflow for evaluating translations and showing graphs as a results. Ideally love to have them as a KNIME Deep Learning node (none with multiple options, possible enhancement request to Deep Learning Team ? @christian.dietz@nemad )
currently we don’t have any specialized nodes for machine translation but you can use our Python nodes (e.g. Python View) to incorporate the python scripts you mentioned.
You could even use the DL Python Network Learner node to monitor those metrics during training by implementing them as custom metric for Keras but they might be a bit too complex (and expensive) to evaluate after each epoch (but I am not an expert on those metrics so I might be mistaken).
Thanks nemad. I don’t think they should be expensive for calculation if you use the ‘coco’ evaluation package.
Meanwhile I will try to put them all in python nodes and try.
