COCO Evaluation Metrics for Neural Machine Translation

Hi There,

Is there any evaluation metric nodes for neural machine translations in KNIME. I was looking for something out of the box and could not find any.
Presently outside of KNIME I’m running Python Scripts using the COCO evaluation metrics from this site: https://github.com/lvapeab/coco-caption/tree/master/pycocoevalcap/ and generating graphs like these:
epoch_100

Is there a way we can include these COCO scripts in my workflow for evaluating translations and showing graphs as a results. Ideally love to have them as a KNIME Deep Learning node (none with multiple options, possible enhancement request to Deep Learning Team ? @christian.dietz @nemad )

Appreciate any help !

Thanks !

Mohammed Ayub

Hello Mhammed,

currently we don’t have any specialized nodes for machine translation but you can use our Python nodes (e.g. Python View) to incorporate the python scripts you mentioned.
You could even use the DL Python Network Learner node to monitor those metrics during training by implementing them as custom metric for Keras but they might be a bit too complex (and expensive) to evaluate after each epoch (but I am not an expert on those metrics so I might be mistaken).

Cheers,

nemad

Thanks nemad. I don’t think they should be expensive for calculation if you use the ‘coco’ evaluation package.
Meanwhile I will try to put them all in python nodes and try.

Mohammed Ayub